SPB Git

spb/modelmap Public License

Internal cartography of local LLMs on Apple Silicon — registered, gated, negative-first. Public atlas at modelmap.io.

🔗 https://modelmap.io
interpretabilityllmapple-siliconmlxreproducibility
Python 66.3% JavaScript 24.5% CSS 8.1% Shell 0.7%

expA run #1: validity gate FAILED — first atlas entry is a negative result

- src/modelmap/capture: MLX Tap layer + random-init architecture twin
- promptsets v1 (6x240, sha256 manifest) — now the positive-control corpus
- Result: ceiling everywhere (acc 1.000); twin ALSO at 1.00 on all 28
  layers (max sel 0.56-0.88 vs registered 0.05 gate) — probe map reads
  tokenizer+architecture prior, zero trained-model signal. Shuffled-label
  control would not have caught it; only the twin null did.
- Published atlas/qwen3-0.6b-4bit/probes/v1 (negative_result=true,
  Level 1, replication 1.00) via publish.py gate
- Doctrine: probe maps only as REAL-TWIN differentials from now on

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Simon-Pierre Boucher committed 3 h ago (Aug 12, 2026) parent 3935e79

Showing 19 changed files with +16,873 and −1

added atlas/qwen3-0.6b-4bit/probes/v1/confidence.md +39 −0
@@ -0,0 +1,39 @@
1 +---
2 +project: modelmap
3 +document: qwen3-0.6b-4bit/probes/v1 — confidence
4 +author: Simon-Pierre Boucher
5 +contact: contact@spboucher.ai
6 +website: https://modelmap.io
7 +created: 2026-08-12
8 +status: reviewed
9 +---
10 +
11 +# Confidence — qwen3-0.6b-4bit / probes / v1
12 +
13 +```text
14 +Level : 1
15 +Seeds : 5
16 +Prompt sets: 6 (2 disjoint template families per property)
17 +Methods in agreement : 1 (linear probes only — Level 2 requires a second method)
18 +Causal verification : none (observational; Level 3 requires intervention)
19 +```
20 +
21 +Per-property evidence:
22 +- lang_id: maxAcc A/B = 1.000/1.000, seed SD 0.0000, dataset shift 0.0000, twin max selectivity 0.808, replication(top-5) 1.00/1.00
23 +- code_prose: maxAcc A/B = 1.000/1.000, seed SD 0.0000, dataset shift 0.0000, twin max selectivity 0.883, replication(top-5) 1.00/1.00
24 +- arith: maxAcc A/B = 1.000/1.000, seed SD 0.0000, dataset shift 0.0000, twin max selectivity 0.558, replication(top-5) 1.00/1.00
25 +
26 +**This is a published NEGATIVE result (Level 1 for the negative claim).**
27 +The random-init architecture twin matches the trained model at ceiling
28 +(accuracy 1.00, 28/28 layers FDR-significant, for the twin as for the real
29 +model; mean real-minus-twin selectivity within +/-0.06). By the validity
30 +criterion registered in hypothesis.md BEFORE the run (twin selectivity must
31 +stay < 0.05), this probing harness is INVALID for localization claims on
32 +these promptsets: it measures the tokenizer + architecture prior, not
33 +learned computation. The negative claim itself is controlled and replicated
34 +(5 seeds, 2 disjoint promptsets, 3 properties) - hence Level 1.
35 +
36 +Consequences adopted: (1) probe maps are only publishable as REAL-MINUS-TWIN
37 +differentials; (2) promptsets v2 must remove lexical separability (shared
38 +vocabulary across classes); (3) the seed-vs-dataset variance hypothesis is
39 +untestable at ceiling and moves to run #2.
added atlas/qwen3-0.6b-4bit/probes/v1/map.json +2341 −0
@@ -0,0 +1,2341 @@
1 +{
2 + "author": "Simon-Pierre Boucher",
3 + "contact": "contact@spboucher.ai",
4 + "website": "https://modelmap.io",
5 + "map_type": "probes",
6 + "model_id": "mlx-community/Qwen3-0.6B-4bit",
7 + "properties": {
8 + "lang_id": {
9 + "summary": {
10 + "mean_seed_sd": 0.0,
11 + "mean_dataset_shift": 0.0,
12 + "layers_shift_gt_seed_sd": 0,
13 + "n_layers": 28,
14 + "max_task_acc_A": 1.0,
15 + "max_task_acc_B": 1.0,
16 + "twin_max_selectivity": 0.8083333333333333,
17 + "replication_topk_A": 1.0,
18 + "replication_topk_B": 1.0
19 + },
20 + "per_layer": {
21 + "A": [
22 + {
23 + "layer": 0,
24 + "task_acc_mean": 1.0,
25 + "task_acc_seed_sd": 0.0,
26 + "selectivity_mean": 0.3166666666666667,
27 + "selectivity_ci": [
28 + 0.14166666666666666,
29 + 0.5750000000000001
30 + ],
31 + "fdr_significant": true
32 + },
33 + {
34 + "layer": 1,
35 + "task_acc_mean": 1.0,
36 + "task_acc_seed_sd": 0.0,
37 + "selectivity_mean": 0.3,
38 + "selectivity_ci": [
39 + 0.13333333333333336,
40 + 0.5416666666666667
41 + ],
42 + "fdr_significant": true
43 + },
44 + {
45 + "layer": 2,
46 + "task_acc_mean": 1.0,
47 + "task_acc_seed_sd": 0.0,
48 + "selectivity_mean": 0.4625,
49 + "selectivity_ci": [
50 + 0.16666666666666669,
51 + 0.7583333333333334
52 + ],
53 + "fdr_significant": true
54 + },
55 + {
56 + "layer": 3,
57 + "task_acc_mean": 1.0,
58 + "task_acc_seed_sd": 0.0,
59 + "selectivity_mean": 0.2875,
60 + "selectivity_ci": [
61 + 0.11250000000000002,
62 + 0.575
63 + ],
64 + "fdr_significant": true
65 + },
66 + {
67 + "layer": 4,
68 + "task_acc_mean": 1.0,
69 + "task_acc_seed_sd": 0.0,
70 + "selectivity_mean": 0.4333333333333334,
71 + "selectivity_ci": [
72 + 0.14166666666666666,
73 + 0.7333333333333334
74 + ],
75 + "fdr_significant": true
76 + },
77 + {
78 + "layer": 5,
79 + "task_acc_mean": 1.0,
80 + "task_acc_seed_sd": 0.0,
81 + "selectivity_mean": 0.7291666666666667,
82 + "selectivity_ci": [
83 + 0.4000000000000001,
84 + 0.9125
85 + ],
86 + "fdr_significant": true
87 + },
88 + {
89 + "layer": 6,
90 + "task_acc_mean": 1.0,
91 + "task_acc_seed_sd": 0.0,
92 + "selectivity_mean": 0.625,
93 + "selectivity_ci": [
94 + 0.325,
95 + 0.9
96 + ],
97 + "fdr_significant": true
98 + },
99 + {
100 + "layer": 7,
101 + "task_acc_mean": 1.0,
102 + "task_acc_seed_sd": 0.0,
103 + "selectivity_mean": 0.675,
104 + "selectivity_ci": [
105 + 0.3666666666666667,
106 + 0.8708333333333333
107 + ],
108 + "fdr_significant": true
109 + },
110 + {
111 + "layer": 8,
112 + "task_acc_mean": 1.0,
113 + "task_acc_seed_sd": 0.0,
114 + "selectivity_mean": 0.6875,
115 + "selectivity_ci": [
116 + 0.4458333333333333,
117 + 0.9
118 + ],
119 + "fdr_significant": true
120 + },
121 + {
122 + "layer": 9,
123 + "task_acc_mean": 1.0,
124 + "task_acc_seed_sd": 0.0,
125 + "selectivity_mean": 0.5791666666666667,
126 + "selectivity_ci": [
127 + 0.3166666666666666,
128 + 0.8416666666666668
129 + ],
130 + "fdr_significant": true
131 + },
132 + {
133 + "layer": 10,
134 + "task_acc_mean": 1.0,
135 + "task_acc_seed_sd": 0.0,
136 + "selectivity_mean": 0.7708333333333334,
137 + "selectivity_ci": [
138 + 0.6458333333333334,
139 + 0.8958333333333333
140 + ],
141 + "fdr_significant": true
142 + },
143 + {
144 + "layer": 11,
145 + "task_acc_mean": 1.0,
146 + "task_acc_seed_sd": 0.0,
147 + "selectivity_mean": 0.7458333333333333,
148 + "selectivity_ci": [
149 + 0.4958333333333334,
150 + 0.8875
151 + ],
152 + "fdr_significant": true
153 + },
154 + {
155 + "layer": 12,
156 + "task_acc_mean": 1.0,
157 + "task_acc_seed_sd": 0.0,
158 + "selectivity_mean": 0.6458333333333333,
159 + "selectivity_ci": [
160 + 0.5083333333333333,
161 + 0.7666666666666666
162 + ],
163 + "fdr_significant": true
164 + },
165 + {
166 + "layer": 13,
167 + "task_acc_mean": 1.0,
168 + "task_acc_seed_sd": 0.0,
169 + "selectivity_mean": 0.475,
170 + "selectivity_ci": [
171 + 0.3125,
172 + 0.6375
173 + ],
174 + "fdr_significant": true
175 + },
176 + {
177 + "layer": 14,
178 + "task_acc_mean": 1.0,
179 + "task_acc_seed_sd": 0.0,
180 + "selectivity_mean": 0.475,
181 + "selectivity_ci": [
182 + 0.275,
183 + 0.6791666666666666
184 + ],
185 + "fdr_significant": true
186 + },
187 + {
188 + "layer": 15,
189 + "task_acc_mean": 1.0,
190 + "task_acc_seed_sd": 0.0,
191 + "selectivity_mean": 0.55,
192 + "selectivity_ci": [
193 + 0.3375,
194 + 0.7458333333333333
195 + ],
196 + "fdr_significant": true
197 + },
198 + {
199 + "layer": 16,
200 + "task_acc_mean": 1.0,
201 + "task_acc_seed_sd": 0.0,
202 + "selectivity_mean": 0.6791666666666666,
203 + "selectivity_ci": [
204 + 0.4708333333333333,
205 + 0.8291666666666668
206 + ],
207 + "fdr_significant": true
208 + },
209 + {
210 + "layer": 17,
211 + "task_acc_mean": 1.0,
212 + "task_acc_seed_sd": 0.0,
213 + "selectivity_mean": 0.6166666666666668,
214 + "selectivity_ci": [
215 + 0.4333333333333334,
216 + 0.75
217 + ],
218 + "fdr_significant": true
219 + },
220 + {
221 + "layer": 18,
222 + "task_acc_mean": 1.0,
223 + "task_acc_seed_sd": 0.0,
224 + "selectivity_mean": 0.5916666666666666,
225 + "selectivity_ci": [
226 + 0.3375,
227 + 0.8083333333333333
228 + ],
229 + "fdr_significant": true
230 + },
231 + {
232 + "layer": 19,
233 + "task_acc_mean": 1.0,
234 + "task_acc_seed_sd": 0.0,
235 + "selectivity_mean": 0.4541666666666667,
236 + "selectivity_ci": [
237 + 0.20833333333333334,
238 + 0.7
239 + ],
240 + "fdr_significant": true
241 + },
242 + {
243 + "layer": 20,
244 + "task_acc_mean": 1.0,
245 + "task_acc_seed_sd": 0.0,
246 + "selectivity_mean": 0.625,
247 + "selectivity_ci": [
248 + 0.3375,
249 + 0.825
250 + ],
251 + "fdr_significant": true
252 + },
253 + {
254 + "layer": 21,
255 + "task_acc_mean": 1.0,
256 + "task_acc_seed_sd": 0.0,
257 + "selectivity_mean": 0.5624999999999999,
258 + "selectivity_ci": [
259 + 0.2833333333333333,
260 + 0.8416666666666666
261 + ],
262 + "fdr_significant": true
263 + },
264 + {
265 + "layer": 22,
266 + "task_acc_mean": 1.0,
267 + "task_acc_seed_sd": 0.0,
268 + "selectivity_mean": 0.5166666666666666,
269 + "selectivity_ci": [
270 + 0.22916666666666669,
271 + 0.7916666666666666
272 + ],
273 + "fdr_significant": true
274 + },
275 + {
276 + "layer": 23,
277 + "task_acc_mean": 1.0,
278 + "task_acc_seed_sd": 0.0,
279 + "selectivity_mean": 0.3833333333333334,
280 + "selectivity_ci": [
281 + 0.16250000000000003,
282 + 0.6875000000000001
283 + ],
284 + "fdr_significant": true
285 + },
286 + {
287 + "layer": 24,
288 + "task_acc_mean": 1.0,
289 + "task_acc_seed_sd": 0.0,
290 + "selectivity_mean": 0.19583333333333333,
291 + "selectivity_ci": [
292 + 0.09166666666666667,
293 + 0.36249999999999993
294 + ],
295 + "fdr_significant": true
296 + },
297 + {
298 + "layer": 25,
299 + "task_acc_mean": 1.0,
300 + "task_acc_seed_sd": 0.0,
301 + "selectivity_mean": 0.14999999999999997,
302 + "selectivity_ci": [
303 + 0.0833333333333333,
304 + 0.225
305 + ],
306 + "fdr_significant": true
307 + },
308 + {
309 + "layer": 26,
310 + "task_acc_mean": 1.0,
311 + "task_acc_seed_sd": 0.0,
312 + "selectivity_mean": 0.15416666666666667,
313 + "selectivity_ci": [
314 + 0.08750000000000002,
315 + 0.19583333333333336
316 + ],
317 + "fdr_significant": true
318 + },
319 + {
320 + "layer": 27,
321 + "task_acc_mean": 1.0,
322 + "task_acc_seed_sd": 0.0,
323 + "selectivity_mean": 0.4,
324 + "selectivity_ci": [
325 + 0.10416666666666666,
326 + 0.6958333333333334
327 + ],
328 + "fdr_significant": true
329 + }
330 + ],
331 + "B": [
332 + {
333 + "layer": 0,
334 + "task_acc_mean": 1.0,
335 + "task_acc_seed_sd": 0.0,
336 + "selectivity_mean": 0.4458333333333333,
337 + "selectivity_ci": [
338 + 0.10833333333333331,
339 + 0.7833333333333334
340 + ],
341 + "fdr_significant": true
342 + },
343 + {
344 + "layer": 1,
345 + "task_acc_mean": 1.0,
346 + "task_acc_seed_sd": 0.0,
347 + "selectivity_mean": 0.5833333333333334,
348 + "selectivity_ci": [
349 + 0.2833333333333333,
350 + 0.8833333333333334
351 + ],
352 + "fdr_significant": true
353 + },
354 + {
355 + "layer": 2,
356 + "task_acc_mean": 1.0,
357 + "task_acc_seed_sd": 0.0,
358 + "selectivity_mean": 0.3916666666666666,
359 + "selectivity_ci": [
360 + 0.04583333333333332,
361 + 0.7374999999999999
362 + ],
363 + "fdr_significant": true
364 + },
365 + {
366 + "layer": 3,
367 + "task_acc_mean": 1.0,
368 + "task_acc_seed_sd": 0.0,
369 + "selectivity_mean": 0.55,
370 + "selectivity_ci": [
371 + 0.20833333333333331,
372 + 0.8916666666666666
373 + ],
374 + "fdr_significant": true
375 + },
376 + {
377 + "layer": 4,
378 + "task_acc_mean": 1.0,
379 + "task_acc_seed_sd": 0.0,
380 + "selectivity_mean": 0.6208333333333333,
381 + "selectivity_ci": [
382 + 0.3583333333333333,
383 + 0.8666666666666666
384 + ],
385 + "fdr_significant": true
386 + },
387 + {
388 + "layer": 5,
389 + "task_acc_mean": 1.0,
390 + "task_acc_seed_sd": 0.0,
391 + "selectivity_mean": 0.6083333333333333,
392 + "selectivity_ci": [
393 + 0.3291666666666667,
394 + 0.85
395 + ],
396 + "fdr_significant": true
397 + },
398 + {
399 + "layer": 6,
400 + "task_acc_mean": 1.0,
401 + "task_acc_seed_sd": 0.0,
402 + "selectivity_mean": 0.2791666666666667,
403 + "selectivity_ci": [
404 + 0.03750000000000002,
405 + 0.5791666666666667
406 + ],
407 + "fdr_significant": true
408 + },
409 + {
410 + "layer": 7,
411 + "task_acc_mean": 1.0,
412 + "task_acc_seed_sd": 0.0,
413 + "selectivity_mean": 0.4250000000000001,
414 + "selectivity_ci": [
415 + 0.1416666666666667,
416 + 0.7208333333333334
417 + ],
418 + "fdr_significant": true
419 + },
420 + {
421 + "layer": 8,
422 + "task_acc_mean": 1.0,
423 + "task_acc_seed_sd": 0.0,
424 + "selectivity_mean": 0.48749999999999993,
425 + "selectivity_ci": [
426 + 0.21666666666666662,
427 + 0.7583333333333333
428 + ],
429 + "fdr_significant": true
430 + },
431 + {
432 + "layer": 9,
433 + "task_acc_mean": 1.0,
434 + "task_acc_seed_sd": 0.0,
435 + "selectivity_mean": 0.4333333333333333,
436 + "selectivity_ci": [
437 + 0.1958333333333333,
438 + 0.6874999999999999
439 + ],
440 + "fdr_significant": true
441 + },
442 + {
443 + "layer": 10,
444 + "task_acc_mean": 1.0,
445 + "task_acc_seed_sd": 0.0,
446 + "selectivity_mean": 0.4583333333333333,
447 + "selectivity_ci": [
448 + 0.20416666666666666,
449 + 0.7249999999999999
450 + ],
451 + "fdr_significant": true
452 + },
453 + {
454 + "layer": 11,
455 + "task_acc_mean": 1.0,
456 + "task_acc_seed_sd": 0.0,
457 + "selectivity_mean": 0.27083333333333337,
458 + "selectivity_ci": [
459 + 0.07916666666666668,
460 + 0.5583333333333333
461 + ],
462 + "fdr_significant": true
463 + },
464 + {
465 + "layer": 12,
466 + "task_acc_mean": 1.0,
467 + "task_acc_seed_sd": 0.0,
468 + "selectivity_mean": 0.5625,
469 + "selectivity_ci": [
470 + 0.2625,
471 + 0.8583333333333334
472 + ],
473 + "fdr_significant": true
474 + },
475 + {
476 + "layer": 13,
477 + "task_acc_mean": 1.0,
478 + "task_acc_seed_sd": 0.0,
479 + "selectivity_mean": 0.4583333333333333,
480 + "selectivity_ci": [
481 + 0.15416666666666665,
482 + 0.7625
483 + ],
484 + "fdr_significant": true
485 + },
486 + {
487 + "layer": 14,
488 + "task_acc_mean": 1.0,
489 + "task_acc_seed_sd": 0.0,
490 + "selectivity_mean": 0.5458333333333333,
491 + "selectivity_ci": [
492 + 0.3041666666666666,
493 + 0.8041666666666666
494 + ],
495 + "fdr_significant": true
496 + },
497 + {
498 + "layer": 15,
499 + "task_acc_mean": 1.0,
500 + "task_acc_seed_sd": 0.0,
501 + "selectivity_mean": 0.4333333333333334,
502 + "selectivity_ci": [
503 + 0.11666666666666667,
504 + 0.7500000000000001
505 + ],
506 + "fdr_significant": true
507 + },
508 + {
509 + "layer": 16,
510 + "task_acc_mean": 1.0,
511 + "task_acc_seed_sd": 0.0,
512 + "selectivity_mean": 0.6625,
513 + "selectivity_ci": [
514 + 0.375,
515 + 0.8625
516 + ],
517 + "fdr_significant": true
518 + },
519 + {
520 + "layer": 17,
521 + "task_acc_mean": 1.0,
522 + "task_acc_seed_sd": 0.0,
523 + "selectivity_mean": 0.7125,
524 + "selectivity_ci": [
525 + 0.4039583333333335,
526 + 0.9041666666666666
527 + ],
528 + "fdr_significant": true
529 + },
530 + {
531 + "layer": 18,
532 + "task_acc_mean": 1.0,
533 + "task_acc_seed_sd": 0.0,
534 + "selectivity_mean": 0.7625,
535 + "selectivity_ci": [
536 + 0.4833333333333334,
537 + 0.9083333333333334
538 + ],
539 + "fdr_significant": true
540 + },
541 + {
542 + "layer": 19,
543 + "task_acc_mean": 1.0,
544 + "task_acc_seed_sd": 0.0,
545 + "selectivity_mean": 0.4708333333333334,
546 + "selectivity_ci": [
547 + 0.19583333333333336,
548 + 0.7458333333333333
549 + ],
550 + "fdr_significant": true
551 + },
552 + {
553 + "layer": 20,
554 + "task_acc_mean": 1.0,
555 + "task_acc_seed_sd": 0.0,
556 + "selectivity_mean": 0.2791666666666667,
557 + "selectivity_ci": [
558 + 0.07916666666666668,
559 + 0.5416666666666667
560 + ],
561 + "fdr_significant": true
562 + },
563 + {
564 + "layer": 21,
565 + "task_acc_mean": 1.0,
566 + "task_acc_seed_sd": 0.0,
567 + "selectivity_mean": 0.4333333333333334,
568 + "selectivity_ci": [
569 + 0.11250000000000002,
570 + 0.7583333333333334
571 + ],
572 + "fdr_significant": true
573 + },
574 + {
575 + "layer": 22,
576 + "task_acc_mean": 1.0,
577 + "task_acc_seed_sd": 0.0,
578 + "selectivity_mean": 0.6083333333333334,
579 + "selectivity_ci": [
580 + 0.3541666666666667,
581 + 0.8416666666666666
582 + ],
583 + "fdr_significant": true
584 + },
585 + {
586 + "layer": 23,
587 + "task_acc_mean": 1.0,
588 + "task_acc_seed_sd": 0.0,
589 + "selectivity_mean": 0.5041666666666667,
590 + "selectivity_ci": [
591 + 0.2375,
592 + 0.7708333333333333
593 + ],
594 + "fdr_significant": true
595 + },
596 + {
597 + "layer": 24,
598 + "task_acc_mean": 1.0,
599 + "task_acc_seed_sd": 0.0,
600 + "selectivity_mean": 0.2458333333333333,
601 + "selectivity_ci": [
602 + 0.054166666666666675,
603 + 0.5375
604 + ],
605 + "fdr_significant": true
606 + },
607 + {
608 + "layer": 25,
609 + "task_acc_mean": 1.0,
610 + "task_acc_seed_sd": 0.0,
611 + "selectivity_mean": 0.4208333333333333,
612 + "selectivity_ci": [
613 + 0.12916666666666665,
614 + 0.7125
615 + ],
616 + "fdr_significant": true
617 + },
618 + {
619 + "layer": 26,
620 + "task_acc_mean": 1.0,
621 + "task_acc_seed_sd": 0.0,
622 + "selectivity_mean": 0.5083333333333333,
623 + "selectivity_ci": [
624 + 0.2541666666666666,
625 + 0.7666666666666666
626 + ],
627 + "fdr_significant": true
628 + },
629 + {
630 + "layer": 27,
631 + "task_acc_mean": 1.0,
632 + "task_acc_seed_sd": 0.0,
633 + "selectivity_mean": 0.3124999999999999,
634 + "selectivity_ci": [
635 + 0.09166666666666663,
636 + 0.6333333333333333
637 + ],
638 + "fdr_significant": true
639 + }
640 + ]
641 + },
642 + "twin_null_per_layer_A": [
643 + {
644 + "layer": 0,
645 + "selectivity_mean": 0.4458333333333333,
646 + "fdr_significant": true
647 + },
648 + {
649 + "layer": 1,
650 + "selectivity_mean": 0.5291666666666667,
651 + "fdr_significant": true
652 + },
653 + {
654 + "layer": 2,
655 + "selectivity_mean": 0.2625,
656 + "fdr_significant": true
657 + },
658 + {
659 + "layer": 3,
660 + "selectivity_mean": 0.4000000000000001,
661 + "fdr_significant": true
662 + },
663 + {
664 + "layer": 4,
665 + "selectivity_mean": 0.7833333333333334,
666 + "fdr_significant": true
667 + },
668 + {
669 + "layer": 5,
670 + "selectivity_mean": 0.8083333333333333,
671 + "fdr_significant": true
672 + },
673 + {
674 + "layer": 6,
675 + "selectivity_mean": 0.7375,
676 + "fdr_significant": true
677 + },
678 + {
679 + "layer": 7,
680 + "selectivity_mean": 0.2041666666666667,
681 + "fdr_significant": true
682 + },
683 + {
684 + "layer": 8,
685 + "selectivity_mean": 0.6416666666666667,
686 + "fdr_significant": true
687 + },
688 + {
689 + "layer": 9,
690 + "selectivity_mean": 0.6166666666666667,
691 + "fdr_significant": true
692 + },
693 + {
694 + "layer": 10,
695 + "selectivity_mean": 0.35416666666666663,
696 + "fdr_significant": true
697 + },
698 + {
699 + "layer": 11,
700 + "selectivity_mean": 0.5791666666666666,
701 + "fdr_significant": true
702 + },
703 + {
704 + "layer": 12,
705 + "selectivity_mean": 0.5958333333333333,
706 + "fdr_significant": true
707 + },
708 + {
709 + "layer": 13,
710 + "selectivity_mean": 0.3041666666666666,
711 + "fdr_significant": true
712 + },
713 + {
714 + "layer": 14,
715 + "selectivity_mean": 0.22916666666666669,
716 + "fdr_significant": true
717 + },
718 + {
719 + "layer": 15,
720 + "selectivity_mean": 0.2416666666666666,
721 + "fdr_significant": true
722 + },
723 + {
724 + "layer": 16,
725 + "selectivity_mean": 0.4416666666666666,
726 + "fdr_significant": true
727 + },
728 + {
729 + "layer": 17,
730 + "selectivity_mean": 0.2625,
731 + "fdr_significant": true
732 + },
733 + {
734 + "layer": 18,
735 + "selectivity_mean": 0.5916666666666666,
736 + "fdr_significant": true
737 + },
738 + {
739 + "layer": 19,
740 + "selectivity_mean": 0.4916666666666667,
741 + "fdr_significant": true
742 + },
743 + {
744 + "layer": 20,
745 + "selectivity_mean": 0.2333333333333333,
746 + "fdr_significant": true
747 + },
748 + {
749 + "layer": 21,
750 + "selectivity_mean": 0.4291666666666667,
751 + "fdr_significant": true
752 + },
753 + {
754 + "layer": 22,
755 + "selectivity_mean": 0.4625,
756 + "fdr_significant": true
757 + },
758 + {
759 + "layer": 23,
760 + "selectivity_mean": 0.4083333333333334,
761 + "fdr_significant": true
762 + },
763 + {
764 + "layer": 24,
765 + "selectivity_mean": 0.22916666666666669,
766 + "fdr_significant": true
767 + },
768 + {
769 + "layer": 25,
770 + "selectivity_mean": 0.6291666666666667,
771 + "fdr_significant": true
772 + },
773 + {
774 + "layer": 26,
775 + "selectivity_mean": 0.4083333333333333,
776 + "fdr_significant": true
777 + },
778 + {
779 + "layer": 27,
780 + "selectivity_mean": 0.4625,
781 + "fdr_significant": true
782 + }
783 + ]
784 + },
785 + "code_prose": {
786 + "summary": {
787 + "mean_seed_sd": 0.0,
788 + "mean_dataset_shift": 0.0,
789 + "layers_shift_gt_seed_sd": 0,
790 + "n_layers": 28,
791 + "max_task_acc_A": 1.0,
792 + "max_task_acc_B": 1.0,
793 + "twin_max_selectivity": 0.8833333333333332,
794 + "replication_topk_A": 1.0,
795 + "replication_topk_B": 1.0
796 + },
797 + "per_layer": {
798 + "A": [
799 + {
800 + "layer": 0,
801 + "task_acc_mean": 1.0,
802 + "task_acc_seed_sd": 0.0,
803 + "selectivity_mean": 0.5083333333333333,
804 + "selectivity_ci": [
805 + 0.2,
806 + 0.775
807 + ],
808 + "fdr_significant": true
809 + },
810 + {
811 + "layer": 1,
812 + "task_acc_mean": 1.0,
813 + "task_acc_seed_sd": 0.0,
814 + "selectivity_mean": 0.4958333333333333,
815 + "selectivity_ci": [
816 + 0.2541666666666666,
817 + 0.7291666666666667
818 + ],
819 + "fdr_significant": true
820 + },
821 + {
822 + "layer": 2,
823 + "task_acc_mean": 1.0,
824 + "task_acc_seed_sd": 0.0,
825 + "selectivity_mean": 0.4291666666666666,
826 + "selectivity_ci": [
827 + 0.14583333333333331,
828 + 0.7208333333333332
829 + ],
830 + "fdr_significant": true
831 + },
832 + {
833 + "layer": 3,
834 + "task_acc_mean": 1.0,
835 + "task_acc_seed_sd": 0.0,
836 + "selectivity_mean": 0.6708333333333333,
837 + "selectivity_ci": [
838 + 0.4,
839 + 0.8916666666666668
840 + ],
841 + "fdr_significant": true
842 + },
843 + {
844 + "layer": 4,
845 + "task_acc_mean": 1.0,
846 + "task_acc_seed_sd": 0.0,
847 + "selectivity_mean": 0.5083333333333334,
848 + "selectivity_ci": [
849 + 0.275,
850 + 0.7416666666666667
851 + ],
852 + "fdr_significant": true
853 + },
854 + {
855 + "layer": 5,
856 + "task_acc_mean": 1.0,
857 + "task_acc_seed_sd": 0.0,
858 + "selectivity_mean": 0.5291666666666666,
859 + "selectivity_ci": [
860 + 0.2583333333333333,
861 + 0.8
862 + ],
863 + "fdr_significant": true
864 + },
865 + {
866 + "layer": 6,
867 + "task_acc_mean": 1.0,
868 + "task_acc_seed_sd": 0.0,
869 + "selectivity_mean": 0.3541666666666667,
870 + "selectivity_ci": [
871 + 0.1166666666666667,
872 + 0.6085416666666635
873 + ],
874 + "fdr_significant": true
875 + },
876 + {
877 + "layer": 7,
878 + "task_acc_mean": 1.0,
879 + "task_acc_seed_sd": 0.0,
880 + "selectivity_mean": 0.3999999999999999,
881 + "selectivity_ci": [
882 + 0.10416666666666666,
883 + 0.6958333333333333
884 + ],
885 + "fdr_significant": true
886 + },
887 + {
888 + "layer": 8,
889 + "task_acc_mean": 1.0,
890 + "task_acc_seed_sd": 0.0,
891 + "selectivity_mean": 0.45,
892 + "selectivity_ci": [
893 + 0.175,
894 + 0.7250000000000001
895 + ],
896 + "fdr_significant": true
897 + },
898 + {
899 + "layer": 9,
900 + "task_acc_mean": 1.0,
901 + "task_acc_seed_sd": 0.0,
902 + "selectivity_mean": 0.5625000000000001,
903 + "selectivity_ci": [
904 + 0.27083333333333337,
905 + 0.8541666666666667
906 + ],
907 + "fdr_significant": true
908 + },
909 + {
910 + "layer": 10,
911 + "task_acc_mean": 1.0,
912 + "task_acc_seed_sd": 0.0,
913 + "selectivity_mean": 0.4208333333333334,
914 + "selectivity_ci": [
915 + 0.16250000000000003,
916 + 0.6791666666666668
917 + ],
918 + "fdr_significant": true
919 + },
920 + {
921 + "layer": 11,
922 + "task_acc_mean": 1.0,
923 + "task_acc_seed_sd": 0.0,
924 + "selectivity_mean": 0.4958333333333334,
925 + "selectivity_ci": [
926 + 0.225,
927 + 0.7666666666666667
928 + ],
929 + "fdr_significant": true
930 + },
931 + {
932 + "layer": 12,
933 + "task_acc_mean": 1.0,
934 + "task_acc_seed_sd": 0.0,
935 + "selectivity_mean": 0.4666666666666666,
936 + "selectivity_ci": [
937 + 0.2208333333333333,
938 + 0.7166666666666666
939 + ],
940 + "fdr_significant": true
941 + },
942 + {
943 + "layer": 13,
944 + "task_acc_mean": 1.0,
945 + "task_acc_seed_sd": 0.0,
946 + "selectivity_mean": 0.5416666666666667,
947 + "selectivity_ci": [
948 + 0.31666666666666665,
949 + 0.7666666666666667
950 + ],
951 + "fdr_significant": true
952 + },
953 + {
954 + "layer": 14,
955 + "task_acc_mean": 1.0,
956 + "task_acc_seed_sd": 0.0,
957 + "selectivity_mean": 0.475,
958 + "selectivity_ci": [
959 + 0.2458333333333333,
960 + 0.7000000000000001
961 + ],
962 + "fdr_significant": true
963 + },
964 + {
965 + "layer": 15,
966 + "task_acc_mean": 1.0,
967 + "task_acc_seed_sd": 0.0,
968 + "selectivity_mean": 0.4541666666666667,
969 + "selectivity_ci": [
970 + 0.2,
971 + 0.7000000000000001
972 + ],
973 + "fdr_significant": true
974 + },
975 + {
976 + "layer": 16,
977 + "task_acc_mean": 1.0,
978 + "task_acc_seed_sd": 0.0,
979 + "selectivity_mean": 0.425,
980 + "selectivity_ci": [
981 + 0.15000000000000002,
982 + 0.7124999999999999
983 + ],
984 + "fdr_significant": true
985 + },
986 + {
987 + "layer": 17,
988 + "task_acc_mean": 1.0,
989 + "task_acc_seed_sd": 0.0,
990 + "selectivity_mean": 0.4166666666666667,
991 + "selectivity_ci": [
992 + 0.1375,
993 + 0.6958333333333334
994 + ],
995 + "fdr_significant": true
996 + },
997 + {
998 + "layer": 18,
999 + "task_acc_mean": 1.0,
1000 + "task_acc_seed_sd": 0.0,
1001 + "selectivity_mean": 0.4416666666666667,
1002 + "selectivity_ci": [
1003 + 0.13333333333333333,
1004 + 0.775
1005 + ],
1006 + "fdr_significant": true
1007 + },
1008 + {
1009 + "layer": 19,
1010 + "task_acc_mean": 1.0,
1011 + "task_acc_seed_sd": 0.0,
1012 + "selectivity_mean": 0.7916666666666666,
1013 + "selectivity_ci": [
1014 + 0.5416666666666666,
1015 + 0.9625
1016 + ],
1017 + "fdr_significant": true
1018 + },
1019 + {
1020 + "layer": 20,
1021 + "task_acc_mean": 1.0,
1022 + "task_acc_seed_sd": 0.0,
1023 + "selectivity_mean": 0.8625,
1024 + "selectivity_ci": [
1025 + 0.8208333333333332,
1026 + 0.9125
1027 + ],
1028 + "fdr_significant": true
1029 + },
1030 + {
1031 + "layer": 21,
1032 + "task_acc_mean": 1.0,
1033 + "task_acc_seed_sd": 0.0,
1034 + "selectivity_mean": 0.6791666666666667,
1035 + "selectivity_ci": [
1036 + 0.425,
1037 + 0.8541666666666667
1038 + ],
1039 + "fdr_significant": true
1040 + },
1041 + {
1042 + "layer": 22,
1043 + "task_acc_mean": 1.0,
1044 + "task_acc_seed_sd": 0.0,
1045 + "selectivity_mean": 0.5791666666666666,
1046 + "selectivity_ci": [
1047 + 0.2875,
1048 + 0.8416666666666666
1049 + ],
1050 + "fdr_significant": true
1051 + },
1052 + {
1053 + "layer": 23,
1054 + "task_acc_mean": 1.0,
1055 + "task_acc_seed_sd": 0.0,
1056 + "selectivity_mean": 0.675,
1057 + "selectivity_ci": [
1058 + 0.4083333333333334,
1059 + 0.8416666666666668
1060 + ],
1061 + "fdr_significant": true
1062 + },
1063 + {
1064 + "layer": 24,
1065 + "task_acc_mean": 1.0,
1066 + "task_acc_seed_sd": 0.0,
1067 + "selectivity_mean": 0.8041666666666666,
1068 + "selectivity_ci": [
1069 + 0.7041666666666666,
1070 + 0.9
1071 + ],
1072 + "fdr_significant": true
1073 + },
1074 + {
1075 + "layer": 25,
1076 + "task_acc_mean": 1.0,
1077 + "task_acc_seed_sd": 0.0,
1078 + "selectivity_mean": 0.675,
1079 + "selectivity_ci": [
1080 + 0.3416666666666667,
1081 + 0.8791666666666667
1082 + ],
1083 + "fdr_significant": true
1084 + },
1085 + {
1086 + "layer": 26,
1087 + "task_acc_mean": 1.0,
1088 + "task_acc_seed_sd": 0.0,
1089 + "selectivity_mean": 0.5958333333333333,
1090 + "selectivity_ci": [
1091 + 0.3416666666666666,
1092 + 0.85
1093 + ],
1094 + "fdr_significant": true
1095 + },
1096 + {
1097 + "layer": 27,
1098 + "task_acc_mean": 1.0,
1099 + "task_acc_seed_sd": 0.0,
1100 + "selectivity_mean": 0.6291666666666667,
1101 + "selectivity_ci": [
1102 + 0.3875,
1103 + 0.7999999999999999
1104 + ],
1105 + "fdr_significant": true
1106 + }
1107 + ],
1108 + "B": [
1109 + {
1110 + "layer": 0,
1111 + "task_acc_mean": 1.0,
1112 + "task_acc_seed_sd": 0.0,
1113 + "selectivity_mean": 0.5666666666666667,
1114 + "selectivity_ci": [
1115 + 0.3,
1116 + 0.8333333333333333
1117 + ],
1118 + "fdr_significant": true
1119 + },
1120 + {
1121 + "layer": 1,
1122 + "task_acc_mean": 1.0,
1123 + "task_acc_seed_sd": 0.0,
1124 + "selectivity_mean": 0.39583333333333337,
1125 + "selectivity_ci": [
1126 + 0.10833333333333332,
1127 + 0.7250000000000001
1128 + ],
1129 + "fdr_significant": true
1130 + },
1131 + {
1132 + "layer": 2,
1133 + "task_acc_mean": 1.0,
1134 + "task_acc_seed_sd": 0.0,
1135 + "selectivity_mean": 0.4125,
1136 + "selectivity_ci": [
1137 + 0.1,
1138 + 0.775
1139 + ],
1140 + "fdr_significant": true
1141 + },
1142 + {
1143 + "layer": 3,
1144 + "task_acc_mean": 1.0,
1145 + "task_acc_seed_sd": 0.0,
1146 + "selectivity_mean": 0.4958333333333333,
1147 + "selectivity_ci": [
1148 + 0.2041666666666667,
1149 + 0.7749999999999999
1150 + ],
1151 + "fdr_significant": true
1152 + },
1153 + {
1154 + "layer": 4,
1155 + "task_acc_mean": 1.0,
1156 + "task_acc_seed_sd": 0.0,
1157 + "selectivity_mean": 0.4208333333333334,
1158 + "selectivity_ci": [
1159 + 0.10416666666666667,
1160 + 0.7583333333333334
1161 + ],
1162 + "fdr_significant": true
1163 + },
1164 + {
1165 + "layer": 5,
1166 + "task_acc_mean": 1.0,
1167 + "task_acc_seed_sd": 0.0,
1168 + "selectivity_mean": 0.4208333333333333,
1169 + "selectivity_ci": [
1170 + 0.14999999999999997,
1171 + 0.7208333333333333
1172 + ],
1173 + "fdr_significant": true
1174 + },
1175 + {
1176 + "layer": 6,
1177 + "task_acc_mean": 1.0,
1178 + "task_acc_seed_sd": 0.0,
1179 + "selectivity_mean": 0.475,
1180 + "selectivity_ci": [
1181 + 0.2333333333333333,
1182 + 0.7375
1183 + ],
1184 + "fdr_significant": true
1185 + },
1186 + {
1187 + "layer": 7,
1188 + "task_acc_mean": 1.0,
1189 + "task_acc_seed_sd": 0.0,
1190 + "selectivity_mean": 0.2666666666666667,
1191 + "selectivity_ci": [
1192 + 0.08333333333333334,
1193 + 0.5166666666666668
1194 + ],
1195 + "fdr_significant": true
1196 + },
1197 + {
1198 + "layer": 8,
1199 + "task_acc_mean": 1.0,
1200 + "task_acc_seed_sd": 0.0,
1201 + "selectivity_mean": 0.2666666666666667,
1202 + "selectivity_ci": [
1203 + 0.07916666666666668,
1204 + 0.5333333333333333
1205 + ],
1206 + "fdr_significant": true
1207 + },
1208 + {
1209 + "layer": 9,
1210 + "task_acc_mean": 1.0,
1211 + "task_acc_seed_sd": 0.0,
1212 + "selectivity_mean": 0.5666666666666667,
1213 + "selectivity_ci": [
1214 + 0.2666666666666666,
1215 + 0.8666666666666666
1216 + ],
1217 + "fdr_significant": true
1218 + },
1219 + {
1220 + "layer": 10,
1221 + "task_acc_mean": 1.0,
1222 + "task_acc_seed_sd": 0.0,
1223 + "selectivity_mean": 0.4333333333333333,
1224 + "selectivity_ci": [
1225 + 0.19583333333333333,
1226 + 0.6708333333333333
1227 + ],
1228 + "fdr_significant": true
1229 + },
1230 + {
1231 + "layer": 11,
1232 + "task_acc_mean": 1.0,
1233 + "task_acc_seed_sd": 0.0,
1234 + "selectivity_mean": 0.3916666666666667,
1235 + "selectivity_ci": [
1236 + 0.1125,
1237 + 0.675
1238 + ],
1239 + "fdr_significant": true
1240 + },
1241 + {
1242 + "layer": 12,
1243 + "task_acc_mean": 1.0,
1244 + "task_acc_seed_sd": 0.0,
1245 + "selectivity_mean": 0.5416666666666666,
1246 + "selectivity_ci": [
1247 + 0.3208333333333334,
1248 + 0.7625
1249 + ],
1250 + "fdr_significant": true
1251 + },
1252 + {
1253 + "layer": 13,
1254 + "task_acc_mean": 1.0,
1255 + "task_acc_seed_sd": 0.0,
1256 + "selectivity_mean": 0.4166666666666667,
1257 + "selectivity_ci": [
1258 + 0.09166666666666667,
1259 + 0.7416666666666667
1260 + ],
1261 + "fdr_significant": true
1262 + },
1263 + {
1264 + "layer": 14,
1265 + "task_acc_mean": 1.0,
1266 + "task_acc_seed_sd": 0.0,
1267 + "selectivity_mean": 0.3749999999999999,
1268 + "selectivity_ci": [
1269 + 0.14166666666666664,
1270 + 0.6791666666666666
1271 + ],
1272 + "fdr_significant": true
1273 + },
1274 + {
1275 + "layer": 15,
1276 + "task_acc_mean": 1.0,
1277 + "task_acc_seed_sd": 0.0,
1278 + "selectivity_mean": 0.2291666666666666,
1279 + "selectivity_ci": [
1280 + 0.07500000000000002,
1281 + 0.4749999999999999
1282 + ],
1283 + "fdr_significant": true
1284 + },
1285 + {
1286 + "layer": 16,
1287 + "task_acc_mean": 1.0,
1288 + "task_acc_seed_sd": 0.0,
1289 + "selectivity_mean": 0.4041666666666667,
1290 + "selectivity_ci": [
1291 + 0.14583333333333334,
1292 + 0.6625
1293 + ],
1294 + "fdr_significant": true
1295 + },
1296 + {
1297 + "layer": 17,
1298 + "task_acc_mean": 1.0,
1299 + "task_acc_seed_sd": 0.0,
1300 + "selectivity_mean": 0.3125,
1301 + "selectivity_ci": [
1302 + 0.12916666666666668,
1303 + 0.6125
1304 + ],
1305 + "fdr_significant": true
1306 + },
1307 + {
1308 + "layer": 18,
1309 + "task_acc_mean": 1.0,
1310 + "task_acc_seed_sd": 0.0,
1311 + "selectivity_mean": 0.5375,
1312 + "selectivity_ci": [
1313 + 0.2083333333333334,
1314 + 0.8625
1315 + ],
1316 + "fdr_significant": true
1317 + },
1318 + {
1319 + "layer": 19,
1320 + "task_acc_mean": 1.0,
1321 + "task_acc_seed_sd": 0.0,
1322 + "selectivity_mean": 0.4833333333333333,
1323 + "selectivity_ci": [
1324 + 0.15833333333333335,
1325 + 0.8208333333333334
1326 + ],
1327 + "fdr_significant": true
1328 + },
1329 + {
1330 + "layer": 20,
1331 + "task_acc_mean": 1.0,
1332 + "task_acc_seed_sd": 0.0,
1333 + "selectivity_mean": 0.5166666666666667,
1334 + "selectivity_ci": [
1335 + 0.2541666666666667,
1336 + 0.7791666666666667
1337 + ],
1338 + "fdr_significant": true
1339 + },
1340 + {
1341 + "layer": 21,
1342 + "task_acc_mean": 1.0,
1343 + "task_acc_seed_sd": 0.0,
1344 + "selectivity_mean": 0.6875,
1345 + "selectivity_ci": [
1346 + 0.39583333333333337,
1347 + 0.8958333333333334
1348 + ],
1349 + "fdr_significant": true
1350 + },
1351 + {
1352 + "layer": 22,
1353 + "task_acc_mean": 1.0,
1354 + "task_acc_seed_sd": 0.0,
1355 + "selectivity_mean": 0.5666666666666667,
1356 + "selectivity_ci": [
1357 + 0.2875000000000001,
1358 + 0.8458333333333332
1359 + ],
1360 + "fdr_significant": true
1361 + },
1362 + {
1363 + "layer": 23,
1364 + "task_acc_mean": 1.0,
1365 + "task_acc_seed_sd": 0.0,
1366 + "selectivity_mean": 0.6416666666666667,
1367 + "selectivity_ci": [
1368 + 0.3166666666666667,
1369 + 0.8583333333333334
1370 + ],
1371 + "fdr_significant": true
1372 + },
1373 + {
1374 + "layer": 24,
1375 + "task_acc_mean": 1.0,
1376 + "task_acc_seed_sd": 0.0,
1377 + "selectivity_mean": 0.5458333333333334,
1378 + "selectivity_ci": [
1379 + 0.2833333333333334,
1380 + 0.8083333333333333
1381 + ],
1382 + "fdr_significant": true
1383 + },
1384 + {
1385 + "layer": 25,
1386 + "task_acc_mean": 1.0,
1387 + "task_acc_seed_sd": 0.0,
1388 + "selectivity_mean": 0.6875000000000001,
1389 + "selectivity_ci": [
1390 + 0.4625000000000001,
1391 + 0.8666666666666666
1392 + ],
1393 + "fdr_significant": true
1394 + },
1395 + {
1396 + "layer": 26,
1397 + "task_acc_mean": 1.0,
1398 + "task_acc_seed_sd": 0.0,
1399 + "selectivity_mean": 0.6499999999999999,
1400 + "selectivity_ci": [
1401 + 0.3125000000000001,
1402 + 0.8875
1403 + ],
1404 + "fdr_significant": true
1405 + },
1406 + {
1407 + "layer": 27,
1408 + "task_acc_mean": 1.0,
1409 + "task_acc_seed_sd": 0.0,
1410 + "selectivity_mean": 0.5208333333333334,
1411 + "selectivity_ci": [
1412 + 0.2041666666666667,
1413 + 0.8375
1414 + ],
1415 + "fdr_significant": true
1416 + }
1417 + ]
1418 + },
1419 + "twin_null_per_layer_A": [
1420 + {
1421 + "layer": 0,
1422 + "selectivity_mean": 0.575,
1423 + "fdr_significant": true
1424 + },
1425 + {
1426 + "layer": 1,
1427 + "selectivity_mean": 0.7,
1428 + "fdr_significant": true
1429 + },
1430 + {
1431 + "layer": 2,
1432 + "selectivity_mean": 0.4333333333333334,
1433 + "fdr_significant": true
1434 + },
1435 + {
1436 + "layer": 3,
1437 + "selectivity_mean": 0.4375,
1438 + "fdr_significant": true
1439 + },
1440 + {
1441 + "layer": 4,
1442 + "selectivity_mean": 0.4791666666666667,
1443 + "fdr_significant": true
1444 + },
1445 + {
1446 + "layer": 5,
1447 + "selectivity_mean": 0.2958333333333333,
1448 + "fdr_significant": true
1449 + },
1450 + {
1451 + "layer": 6,
1452 + "selectivity_mean": 0.7291666666666667,
1453 + "fdr_significant": true
1454 + },
1455 + {
1456 + "layer": 7,
1457 + "selectivity_mean": 0.6833333333333333,
1458 + "fdr_significant": true
1459 + },
1460 + {
1461 + "layer": 8,
1462 + "selectivity_mean": 0.5874999999999999,
1463 + "fdr_significant": true
1464 + },
1465 + {
1466 + "layer": 9,
1467 + "selectivity_mean": 0.8791666666666667,
1468 + "fdr_significant": true
1469 + },
1470 + {
1471 + "layer": 10,
1472 + "selectivity_mean": 0.6291666666666667,
1473 + "fdr_significant": true
1474 + },
1475 + {
1476 + "layer": 11,
1477 + "selectivity_mean": 0.4666666666666667,
1478 + "fdr_significant": true
1479 + },
1480 + {
1481 + "layer": 12,
1482 + "selectivity_mean": 0.5708333333333333,
1483 + "fdr_significant": true
1484 + },
1485 + {
1486 + "layer": 13,
1487 + "selectivity_mean": 0.5666666666666667,
1488 + "fdr_significant": true
1489 + },
1490 + {
1491 + "layer": 14,
1492 + "selectivity_mean": 0.7166666666666666,
1493 + "fdr_significant": true
1494 + },
1495 + {
1496 + "layer": 15,
1497 + "selectivity_mean": 0.36250000000000004,
1498 + "fdr_significant": true
1499 + },
1500 + {
1501 + "layer": 16,
1502 + "selectivity_mean": 0.6458333333333333,
1503 + "fdr_significant": true
1504 + },
1505 + {
1506 + "layer": 17,
1507 + "selectivity_mean": 0.4666666666666666,
1508 + "fdr_significant": true
1509 + },
1510 + {
1511 + "layer": 18,
1512 + "selectivity_mean": 0.7875,
1513 + "fdr_significant": true
1514 + },
1515 + {
1516 + "layer": 19,
1517 + "selectivity_mean": 0.6,
1518 + "fdr_significant": true
1519 + },
1520 + {
1521 + "layer": 20,
1522 + "selectivity_mean": 0.4291666666666667,
1523 + "fdr_significant": true
1524 + },
1525 + {
1526 + "layer": 21,
1527 + "selectivity_mean": 0.6375,
1528 + "fdr_significant": true
1529 + },
1530 + {
1531 + "layer": 22,
1532 + "selectivity_mean": 0.4125,
1533 + "fdr_significant": true
1534 + },
1535 + {
1536 + "layer": 23,
1537 + "selectivity_mean": 0.6083333333333334,
1538 + "fdr_significant": true
1539 + },
1540 + {
1541 + "layer": 24,
1542 + "selectivity_mean": 0.5708333333333333,
1543 + "fdr_significant": true
1544 + },
1545 + {
1546 + "layer": 25,
1547 + "selectivity_mean": 0.8833333333333332,
1548 + "fdr_significant": true
1549 + },
1550 + {
1551 + "layer": 26,
1552 + "selectivity_mean": 0.4708333333333333,
1553 + "fdr_significant": true
1554 + },
1555 + {
1556 + "layer": 27,
1557 + "selectivity_mean": 0.7333333333333334,
1558 + "fdr_significant": true
1559 + }
1560 + ]
1561 + },
1562 + "arith": {
1563 + "summary": {
1564 + "mean_seed_sd": 0.0,
1565 + "mean_dataset_shift": 0.0,
1566 + "layers_shift_gt_seed_sd": 0,
1567 + "n_layers": 28,
1568 + "max_task_acc_A": 1.0,
1569 + "max_task_acc_B": 1.0,
1570 + "twin_max_selectivity": 0.5583333333333333,
1571 + "replication_topk_A": 1.0,
1572 + "replication_topk_B": 1.0
1573 + },
1574 + "per_layer": {
1575 + "A": [
1576 + {
1577 + "layer": 0,
1578 + "task_acc_mean": 1.0,
1579 + "task_acc_seed_sd": 0.0,
1580 + "selectivity_mean": 0.7250000000000001,
1581 + "selectivity_ci": [
1582 + 0.4208333333333334,
1583 + 0.9291666666666668
1584 + ],
1585 + "fdr_significant": true
1586 + },
1587 + {
1588 + "layer": 1,
1589 + "task_acc_mean": 1.0,
1590 + "task_acc_seed_sd": 0.0,
1591 + "selectivity_mean": 0.5375,
1592 + "selectivity_ci": [
1593 + 0.22916666666666669,
1594 + 0.8458333333333332
1595 + ],
1596 + "fdr_significant": true
1597 + },
1598 + {
1599 + "layer": 2,
1600 + "task_acc_mean": 1.0,
1601 + "task_acc_seed_sd": 0.0,
1602 + "selectivity_mean": 0.5541666666666667,
1603 + "selectivity_ci": [
1604 + 0.2375,
1605 + 0.8625
1606 + ],
1607 + "fdr_significant": true
1608 + },
1609 + {
1610 + "layer": 3,
1611 + "task_acc_mean": 1.0,
1612 + "task_acc_seed_sd": 0.0,
1613 + "selectivity_mean": 0.35833333333333334,
1614 + "selectivity_ci": [
1615 + 0.06666666666666668,
1616 + 0.6583333333333334
1617 + ],
1618 + "fdr_significant": true
1619 + },
1620 + {
1621 + "layer": 4,
1622 + "task_acc_mean": 1.0,
1623 + "task_acc_seed_sd": 0.0,
1624 + "selectivity_mean": 0.2583333333333333,
1625 + "selectivity_ci": [
1626 + 0.09166666666666665,
1627 + 0.5166666666666666
1628 + ],
1629 + "fdr_significant": true
1630 + },
1631 + {
1632 + "layer": 5,
1633 + "task_acc_mean": 1.0,
1634 + "task_acc_seed_sd": 0.0,
1635 + "selectivity_mean": 0.3708333333333333,
1636 + "selectivity_ci": [
1637 + 0.1625,
1638 + 0.5874999999999998
1639 + ],
1640 + "fdr_significant": true
1641 + },
1642 + {
1643 + "layer": 6,
1644 + "task_acc_mean": 1.0,
1645 + "task_acc_seed_sd": 0.0,
1646 + "selectivity_mean": 0.2958333333333333,
1647 + "selectivity_ci": [
1648 + 0.1708333333333333,
1649 + 0.5083333333333332
1650 + ],
1651 + "fdr_significant": true
1652 + },
1653 + {
1654 + "layer": 7,
1655 + "task_acc_mean": 1.0,
1656 + "task_acc_seed_sd": 0.0,
1657 + "selectivity_mean": 0.2833333333333333,
1658 + "selectivity_ci": [
1659 + 0.12916666666666668,
1660 + 0.5291666666666667
1661 + ],
1662 + "fdr_significant": true
1663 + },
1664 + {
1665 + "layer": 8,
1666 + "task_acc_mean": 1.0,
1667 + "task_acc_seed_sd": 0.0,
1668 + "selectivity_mean": 0.3458333333333333,
1669 + "selectivity_ci": [
1670 + 0.14583333333333331,
1671 + 0.5916666666666666
1672 + ],
1673 + "fdr_significant": true
1674 + },
1675 + {
1676 + "layer": 9,
1677 + "task_acc_mean": 1.0,
1678 + "task_acc_seed_sd": 0.0,
1679 + "selectivity_mean": 0.3791666666666667,
1680 + "selectivity_ci": [
1681 + 0.17083333333333336,
1682 + 0.6125
1683 + ],
1684 + "fdr_significant": true
1685 + },
1686 + {
1687 + "layer": 10,
1688 + "task_acc_mean": 1.0,
1689 + "task_acc_seed_sd": 0.0,
1690 + "selectivity_mean": 0.2875,
1691 + "selectivity_ci": [
1692 + 0.12083333333333332,
1693 + 0.5375000000000001
1694 + ],
1695 + "fdr_significant": true
1696 + },
1697 + {
1698 + "layer": 11,
1699 + "task_acc_mean": 1.0,
1700 + "task_acc_seed_sd": 0.0,
1701 + "selectivity_mean": 0.4416666666666667,
1702 + "selectivity_ci": [
1703 + 0.19583333333333333,
1704 + 0.6958333333333334
1705 + ],
1706 + "fdr_significant": true
1707 + },
1708 + {
1709 + "layer": 12,
1710 + "task_acc_mean": 1.0,
1711 + "task_acc_seed_sd": 0.0,
1712 + "selectivity_mean": 0.3875,
1713 + "selectivity_ci": [
1714 + 0.22916666666666669,
1715 + 0.5791666666666667
1716 + ],
1717 + "fdr_significant": true
1718 + },
1719 + {
1720 + "layer": 13,
1721 + "task_acc_mean": 1.0,
1722 + "task_acc_seed_sd": 0.0,
1723 + "selectivity_mean": 0.3291666666666667,
1724 + "selectivity_ci": [
1725 + 0.16666666666666669,
1726 + 0.55
1727 + ],
1728 + "fdr_significant": true
1729 + },
1730 + {
1731 + "layer": 14,
1732 + "task_acc_mean": 1.0,
1733 + "task_acc_seed_sd": 0.0,
1734 + "selectivity_mean": 0.5458333333333334,
1735 + "selectivity_ci": [
1736 + 0.375,
1737 + 0.7
1738 + ],
1739 + "fdr_significant": true
1740 + },
1741 + {
1742 + "layer": 15,
1743 + "task_acc_mean": 1.0,
1744 + "task_acc_seed_sd": 0.0,
1745 + "selectivity_mean": 0.5708333333333332,
1746 + "selectivity_ci": [
1747 + 0.2708333333333333,
1748 + 0.8625
1749 + ],
1750 + "fdr_significant": true
1751 + },
1752 + {
1753 + "layer": 16,
1754 + "task_acc_mean": 1.0,
1755 + "task_acc_seed_sd": 0.0,
1756 + "selectivity_mean": 0.3791666666666667,
1757 + "selectivity_ci": [
1758 + 0.1375,
1759 + 0.6208333333333333
1760 + ],
1761 + "fdr_significant": true
1762 + },
1763 + {
1764 + "layer": 17,
1765 + "task_acc_mean": 1.0,
1766 + "task_acc_seed_sd": 0.0,
1767 + "selectivity_mean": 0.4958333333333334,
1768 + "selectivity_ci": [
1769 + 0.29166666666666663,
1770 + 0.7
1771 + ],
1772 + "fdr_significant": true
1773 + },
1774 + {
1775 + "layer": 18,
1776 + "task_acc_mean": 1.0,
1777 + "task_acc_seed_sd": 0.0,
1778 + "selectivity_mean": 0.375,
1779 + "selectivity_ci": [
1780 + 0.16666666666666669,
1781 + 0.6041666666666666
1782 + ],
1783 + "fdr_significant": true
1784 + },
1785 + {
1786 + "layer": 19,
1787 + "task_acc_mean": 1.0,
1788 + "task_acc_seed_sd": 0.0,
1789 + "selectivity_mean": 0.5541666666666666,
1790 + "selectivity_ci": [
1791 + 0.29166666666666663,
1792 + 0.8166666666666667
1793 + ],
1794 + "fdr_significant": true
1795 + },
1796 + {
1797 + "layer": 20,
1798 + "task_acc_mean": 1.0,
1799 + "task_acc_seed_sd": 0.0,
1800 + "selectivity_mean": 0.38333333333333336,
1801 + "selectivity_ci": [
1802 + 0.1375,
1803 + 0.6625
1804 + ],
1805 + "fdr_significant": true
1806 + },
1807 + {
1808 + "layer": 21,
1809 + "task_acc_mean": 1.0,
1810 + "task_acc_seed_sd": 0.0,
1811 + "selectivity_mean": 0.5291666666666666,
1812 + "selectivity_ci": [
1813 + 0.2583333333333333,
1814 + 0.8
1815 + ],
1816 + "fdr_significant": true
1817 + },
1818 + {
1819 + "layer": 22,
1820 + "task_acc_mean": 1.0,
1821 + "task_acc_seed_sd": 0.0,
1822 + "selectivity_mean": 0.5,
1823 + "selectivity_ci": [
1824 + 0.2333333333333334,
1825 + 0.7666666666666668
1826 + ],
1827 + "fdr_significant": true
1828 + },
1829 + {
1830 + "layer": 23,
1831 + "task_acc_mean": 1.0,
1832 + "task_acc_seed_sd": 0.0,
1833 + "selectivity_mean": 0.3666666666666667,
1834 + "selectivity_ci": [
1835 + 0.16666666666666669,
1836 + 0.5666666666666668
1837 + ],
1838 + "fdr_significant": true
1839 + },
1840 + {
1841 + "layer": 24,
1842 + "task_acc_mean": 1.0,
1843 + "task_acc_seed_sd": 0.0,
1844 + "selectivity_mean": 0.37916666666666665,
1845 + "selectivity_ci": [
1846 + 0.19166666666666668,
1847 + 0.5874999999999999
1848 + ],
1849 + "fdr_significant": true
1850 + },
1851 + {
1852 + "layer": 25,
1853 + "task_acc_mean": 1.0,
1854 + "task_acc_seed_sd": 0.0,
1855 + "selectivity_mean": 0.5083333333333334,
1856 + "selectivity_ci": [
1857 + 0.2541666666666667,
1858 + 0.7625000000000001
1859 + ],
1860 + "fdr_significant": true
1861 + },
1862 + {
1863 + "layer": 26,
1864 + "task_acc_mean": 1.0,
1865 + "task_acc_seed_sd": 0.0,
1866 + "selectivity_mean": 0.5333333333333334,
1867 + "selectivity_ci": [
1868 + 0.3125,
1869 + 0.7541666666666667
1870 + ],
1871 + "fdr_significant": true
1872 + },
1873 + {
1874 + "layer": 27,
1875 + "task_acc_mean": 1.0,
1876 + "task_acc_seed_sd": 0.0,
1877 + "selectivity_mean": 0.4708333333333333,
1878 + "selectivity_ci": [
1879 + 0.20833333333333331,
1880 + 0.7333333333333334
1881 + ],
1882 + "fdr_significant": true
1883 + }
1884 + ],
1885 + "B": [
1886 + {
1887 + "layer": 0,
1888 + "task_acc_mean": 1.0,
1889 + "task_acc_seed_sd": 0.0,
1890 + "selectivity_mean": 0.5083333333333333,
1891 + "selectivity_ci": [
1892 + 0.19999999999999998,
1893 + 0.8125
1894 + ],
1895 + "fdr_significant": true
1896 + },
1897 + {
1898 + "layer": 1,
1899 + "task_acc_mean": 1.0,
1900 + "task_acc_seed_sd": 0.0,
1901 + "selectivity_mean": 0.3916666666666667,
1902 + "selectivity_ci": [
1903 + 0.04583333333333332,
1904 + 0.7375
1905 + ],
1906 + "fdr_significant": true
1907 + },
1908 + {
1909 + "layer": 2,
1910 + "task_acc_mean": 1.0,
1911 + "task_acc_seed_sd": 0.0,
1912 + "selectivity_mean": 0.6625000000000001,
1913 + "selectivity_ci": [
1914 + 0.38333333333333336,
1915 + 0.8708333333333333
1916 + ],
1917 + "fdr_significant": true
1918 + },
1919 + {
1920 + "layer": 3,
1921 + "task_acc_mean": 1.0,
1922 + "task_acc_seed_sd": 0.0,
1923 + "selectivity_mean": 0.7125000000000001,
1924 + "selectivity_ci": [
1925 + 0.4208333333333334,
1926 + 0.9
1927 + ],
1928 + "fdr_significant": true
1929 + },
1930 + {
1931 + "layer": 4,
1932 + "task_acc_mean": 1.0,
1933 + "task_acc_seed_sd": 0.0,
1934 + "selectivity_mean": 0.5416666666666667,
1935 + "selectivity_ci": [
1936 + 0.19583333333333336,
1937 + 0.8875
1938 + ],
1939 + "fdr_significant": true
1940 + },
1941 + {
1942 + "layer": 5,
1943 + "task_acc_mean": 1.0,
1944 + "task_acc_seed_sd": 0.0,
1945 + "selectivity_mean": 0.4,
1946 + "selectivity_ci": [
1947 + 0.029166666666666674,
1948 + 0.7708333333333333
1949 + ],
1950 + "fdr_significant": true
1951 + },
1952 + {
1953 + "layer": 6,
1954 + "task_acc_mean": 1.0,
1955 + "task_acc_seed_sd": 0.0,
1956 + "selectivity_mean": 0.2375,
1957 + "selectivity_ci": [
1958 + 0.033333333333333326,
1959 + 0.5708333333333334
1960 + ],
1961 + "fdr_significant": true
1962 + },
1963 + {
1964 + "layer": 7,
1965 + "task_acc_mean": 1.0,
1966 + "task_acc_seed_sd": 0.0,
1967 + "selectivity_mean": 0.4166666666666667,
1968 + "selectivity_ci": [
1969 + 0.11250000000000002,
1970 + 0.7291666666666667
1971 + ],
1972 + "fdr_significant": true
1973 + },
1974 + {
1975 + "layer": 8,
1976 + "task_acc_mean": 1.0,
1977 + "task_acc_seed_sd": 0.0,
1978 + "selectivity_mean": 0.2208333333333333,
1979 + "selectivity_ci": [
1980 + 0.049999999999999975,
1981 + 0.5291666666666667
1982 + ],
1983 + "fdr_significant": true
1984 + },
1985 + {
1986 + "layer": 9,
1987 + "task_acc_mean": 1.0,
1988 + "task_acc_seed_sd": 0.0,
1989 + "selectivity_mean": 0.5708333333333334,
1990 + "selectivity_ci": [
1991 + 0.2166666666666667,
1992 + 0.925
1993 + ],
1994 + "fdr_significant": true
1995 + },
1996 + {
1997 + "layer": 10,
1998 + "task_acc_mean": 1.0,
1999 + "task_acc_seed_sd": 0.0,
2000 + "selectivity_mean": 0.6083333333333333,
2001 + "selectivity_ci": [
2002 + 0.2958333333333333,
2003 + 0.9083333333333332
2004 + ],
2005 + "fdr_significant": true
2006 + },
2007 + {
2008 + "layer": 11,
2009 + "task_acc_mean": 1.0,
2010 + "task_acc_seed_sd": 0.0,
2011 + "selectivity_mean": 0.41666666666666663,
2012 + "selectivity_ci": [
2013 + 0.12499999999999997,
2014 + 0.7291666666666666
2015 + ],
2016 + "fdr_significant": true
2017 + },
2018 + {
2019 + "layer": 12,
2020 + "task_acc_mean": 1.0,
2021 + "task_acc_seed_sd": 0.0,
2022 + "selectivity_mean": 0.4458333333333333,
2023 + "selectivity_ci": [
2024 + 0.15416666666666665,
2025 + 0.7499999999999999
2026 + ],
2027 + "fdr_significant": true
2028 + },
2029 + {
2030 + "layer": 13,
2031 + "task_acc_mean": 1.0,
2032 + "task_acc_seed_sd": 0.0,
2033 + "selectivity_mean": 0.2916666666666667,
2034 + "selectivity_ci": [
2035 + 0.12916666666666668,
2036 + 0.5666666666666667
2037 + ],
2038 + "fdr_significant": true
2039 + },
2040 + {
2041 + "layer": 14,
2042 + "task_acc_mean": 1.0,
2043 + "task_acc_seed_sd": 0.0,
2044 + "selectivity_mean": 0.4,
2045 + "selectivity_ci": [
2046 + 0.07916666666666668,
2047 + 0.7208333333333333
2048 + ],
2049 + "fdr_significant": true
2050 + },
2051 + {
2052 + "layer": 15,
2053 + "task_acc_mean": 1.0,
2054 + "task_acc_seed_sd": 0.0,
2055 + "selectivity_mean": 0.2583333333333333,
2056 + "selectivity_ci": [
2057 + 0.09166666666666667,
2058 + 0.5791666666666667
2059 + ],
2060 + "fdr_significant": true
2061 + },
2062 + {
2063 + "layer": 16,
2064 + "task_acc_mean": 1.0,
2065 + "task_acc_seed_sd": 0.0,
2066 + "selectivity_mean": 0.05833333333333335,
2067 + "selectivity_ci": [
2068 + 0.02083333333333335,
2069 + 0.09166666666666667
2070 + ],
2071 + "fdr_significant": true
2072 + },
2073 + {
2074 + "layer": 17,
2075 + "task_acc_mean": 1.0,
2076 + "task_acc_seed_sd": 0.0,
2077 + "selectivity_mean": 0.3958333333333333,
2078 + "selectivity_ci": [
2079 + 0.054166666666666675,
2080 + 0.7375
2081 + ],
2082 + "fdr_significant": true
2083 + },
2084 + {
2085 + "layer": 18,
2086 + "task_acc_mean": 1.0,
2087 + "task_acc_seed_sd": 0.0,
2088 + "selectivity_mean": 0.4,
2089 + "selectivity_ci": [
2090 + 0.07500000000000002,
2091 + 0.7291666666666667
2092 + ],
2093 + "fdr_significant": true
2094 + },
2095 + {
2096 + "layer": 19,
2097 + "task_acc_mean": 1.0,
2098 + "task_acc_seed_sd": 0.0,
2099 + "selectivity_mean": 0.2541666666666666,
2100 + "selectivity_ci": [
2101 + 0.05,
2102 + 0.5875
2103 + ],
2104 + "fdr_significant": true
2105 + },
2106 + {
2107 + "layer": 20,
2108 + "task_acc_mean": 1.0,
2109 + "task_acc_seed_sd": 0.0,
2110 + "selectivity_mean": 0.3791666666666667,
2111 + "selectivity_ci": [
2112 + 0.08333333333333333,
2113 + 0.6833333333333333
2114 + ],
2115 + "fdr_significant": true
2116 + },
2117 + {
2118 + "layer": 21,
2119 + "task_acc_mean": 1.0,
2120 + "task_acc_seed_sd": 0.0,
2121 + "selectivity_mean": 0.3833333333333333,
2122 + "selectivity_ci": [
2123 + 0.12916666666666665,
2124 + 0.6375
2125 + ],
2126 + "fdr_significant": true
2127 + },
2128 + {
2129 + "layer": 22,
2130 + "task_acc_mean": 1.0,
2131 + "task_acc_seed_sd": 0.0,
2132 + "selectivity_mean": 0.25,
2133 + "selectivity_ci": [
2134 + 0.05833333333333333,
2135 + 0.55
2136 + ],
2137 + "fdr_significant": true
2138 + },
2139 + {
2140 + "layer": 23,
2141 + "task_acc_mean": 1.0,
2142 + "task_acc_seed_sd": 0.0,
2143 + "selectivity_mean": 0.3708333333333333,
2144 + "selectivity_ci": [
2145 + 0.12916666666666662,
2146 + 0.6166666666666666
2147 + ],
2148 + "fdr_significant": true
2149 + },
2150 + {
2151 + "layer": 24,
2152 + "task_acc_mean": 1.0,
2153 + "task_acc_seed_sd": 0.0,
2154 + "selectivity_mean": 0.4083333333333333,
2155 + "selectivity_ci": [
2156 + 0.09166666666666663,
2157 + 0.7250000000000001
2158 + ],
2159 + "fdr_significant": true
2160 + },
2161 + {
2162 + "layer": 25,
2163 + "task_acc_mean": 1.0,
2164 + "task_acc_seed_sd": 0.0,
2165 + "selectivity_mean": 0.5208333333333334,
2166 + "selectivity_ci": [
2167 + 0.2041666666666667,
2168 + 0.8333333333333334
2169 + ],
2170 + "fdr_significant": true
2171 + },
2172 + {
2173 + "layer": 26,
2174 + "task_acc_mean": 1.0,
2175 + "task_acc_seed_sd": 0.0,
2176 + "selectivity_mean": 0.3625,
2177 + "selectivity_ci": [
2178 + 0.06666666666666665,
2179 + 0.6791666666666666
2180 + ],
2181 + "fdr_significant": true
2182 + },
2183 + {
2184 + "layer": 27,
2185 + "task_acc_mean": 1.0,
2186 + "task_acc_seed_sd": 0.0,
2187 + "selectivity_mean": 0.5166666666666667,
2188 + "selectivity_ci": [
2189 + 0.22083333333333335,
2190 + 0.8125
2191 + ],
2192 + "fdr_significant": true
2193 + }
2194 + ]
2195 + },
2196 + "twin_null_per_layer_A": [
2197 + {
2198 + "layer": 0,
2199 + "selectivity_mean": 0.3125,
2200 + "fdr_significant": true
2201 + },
2202 + {
2203 + "layer": 1,
2204 + "selectivity_mean": 0.5583333333333333,
2205 + "fdr_significant": true
2206 + },
2207 + {
2208 + "layer": 2,
2209 + "selectivity_mean": 0.4875,
2210 + "fdr_significant": true
2211 + },
2212 + {
2213 + "layer": 3,
2214 + "selectivity_mean": 0.5375,
2215 + "fdr_significant": true
2216 + },
2217 + {
2218 + "layer": 4,
2219 + "selectivity_mean": 0.3291666666666667,
2220 + "fdr_significant": true
2221 + },
2222 + {
2223 + "layer": 5,
2224 + "selectivity_mean": 0.5125,
2225 + "fdr_significant": true
2226 + },
2227 + {
2228 + "layer": 6,
2229 + "selectivity_mean": 0.3458333333333333,
2230 + "fdr_significant": true
2231 + },
2232 + {
2233 + "layer": 7,
2234 + "selectivity_mean": 0.3791666666666667,
2235 + "fdr_significant": true
2236 + },
2237 + {
2238 + "layer": 8,
2239 + "selectivity_mean": 0.3166666666666667,
2240 + "fdr_significant": true
2241 + },
2242 + {
2243 + "layer": 9,
2244 + "selectivity_mean": 0.3875,
2245 + "fdr_significant": true
2246 + },
2247 + {
2248 + "layer": 10,
2249 + "selectivity_mean": 0.3291666666666667,
2250 + "fdr_significant": true
2251 + },
2252 + {
2253 + "layer": 11,
2254 + "selectivity_mean": 0.5041666666666667,
2255 + "fdr_significant": true
2256 + },
2257 + {
2258 + "layer": 12,
2259 + "selectivity_mean": 0.19166666666666665,
2260 + "fdr_significant": true
2261 + },
2262 + {
2263 + "layer": 13,
2264 + "selectivity_mean": 0.18750000000000003,
2265 + "fdr_significant": true
2266 + },
2267 + {
2268 + "layer": 14,
2269 + "selectivity_mean": 0.22916666666666666,
2270 + "fdr_significant": true
2271 + },
2272 + {
2273 + "layer": 15,
2274 + "selectivity_mean": 0.2833333333333334,
2275 + "fdr_significant": true
2276 + },
2277 + {
2278 + "layer": 16,
2279 + "selectivity_mean": 0.4833333333333334,
2280 + "fdr_significant": true
2281 + },
2282 + {
2283 + "layer": 17,
2284 + "selectivity_mean": 0.5,
2285 + "fdr_significant": true
2286 + },
2287 + {
2288 + "layer": 18,
2289 + "selectivity_mean": 0.5083333333333334,
2290 + "fdr_significant": true
2291 + },
2292 + {
2293 + "layer": 19,
2294 + "selectivity_mean": 0.3666666666666667,
2295 + "fdr_significant": true
2296 + },
2297 + {
2298 + "layer": 20,
2299 + "selectivity_mean": 0.5333333333333334,
2300 + "fdr_significant": true
2301 + },
2302 + {
2303 + "layer": 21,
2304 + "selectivity_mean": 0.5208333333333334,
2305 + "fdr_significant": true
2306 + },
2307 + {
2308 + "layer": 22,
2309 + "selectivity_mean": 0.525,
2310 + "fdr_significant": true
2311 + },
2312 + {
2313 + "layer": 23,
2314 + "selectivity_mean": 0.3375,
2315 + "fdr_significant": true
2316 + },
2317 + {
2318 + "layer": 24,
2319 + "selectivity_mean": 0.09583333333333333,
2320 + "fdr_significant": true
2321 + },
2322 + {
2323 + "layer": 25,
2324 + "selectivity_mean": 0.25,
2325 + "fdr_significant": true
2326 + },
2327 + {
2328 + "layer": 26,
2329 + "selectivity_mean": 0.35,
2330 + "fdr_significant": true
2331 + },
2332 + {
2333 + "layer": 27,
2334 + "selectivity_mean": 0.2583333333333333,
2335 + "fdr_significant": true
2336 + }
2337 + ]
2338 + }
2339 + },
2340 + "source_results": "results/expA_probe_reliability/20260812T062605Z/results.json"
2341 +}
added atlas/qwen3-0.6b-4bit/probes/v1/mapcard.json +72 −0
@@ -0,0 +1,72 @@
1 +{
2 + "map_id": "atlas/qwen3-0.6b-4bit/probes/v1",
3 + "map_type": "probes",
4 + "model_id": "mlx-community/Qwen3-0.6B-4bit",
5 + "model_hash": "392e8d466d56100ada00eb82031fb854297fc9e389b7d303eba3af114e87bce2",
6 + "quantization": "q4 (mlx)",
7 + "commit": "3935e7933294b9c1293cd31b887126402fc53115",
8 + "config": "results/expA_probe_reliability/20260812T062605Z/results.json",
9 + "created": "2026-08-12",
10 + "hardware_manifest": {
11 + "author": "Simon-Pierre Boucher",
12 + "contact": "contact@spboucher.ai",
13 + "website": "https://modelmap.io",
14 + "chip": {
15 + "brand": "Apple M5 Max",
16 + "cores_total": 18,
17 + "cores_performance": 6,
18 + "cores_efficiency": 12
19 + },
20 + "memory": {
21 + "unified_gb": 48.0,
22 + "pagesize": 16384
23 + },
24 + "os": {
25 + "system": "Darwin",
26 + "version": "27.0",
27 + "arch": "arm64"
28 + },
29 + "software": {
30 + "python": "3.14.4",
31 + "numpy": "2.5.2",
32 + "mlx": "0.32.0",
33 + "torch": "2.13.0",
34 + "safetensors": "0.8.0"
35 + }
36 + },
37 + "confidence_level": 1,
38 + "regenerate_command": ".venv/bin/python benchmarks/promptsets/make_promptsets.py && .venv/bin/python experiments/micro/expA_probe_reliability/implementation/benchmark.py && .venv/bin/python experiments/micro/expA_probe_reliability/implementation/make_mapcard.py",
39 + "seeds": [
40 + 0,
41 + 1,
42 + 2,
43 + 3,
44 + 4
45 + ],
46 + "prompt_sets": [
47 + "arith_A.jsonl#2fd80600d8a1b4ad",
48 + "arith_B.jsonl#04bb0eceb4b9262e",
49 + "code_prose_A.jsonl#dfbfb13dade0fd0b",
50 + "code_prose_B.jsonl#e9c3f78d8754b0ad",
51 + "lang_id_A.jsonl#43bd7ed12d2d9f95",
52 + "lang_id_B.jsonl#833435dae9a61ae0"
53 + ],
54 + "controls": [
55 + "shuffled-label (every probe)",
56 + "random-init architecture twin",
57 + "BH-FDR q=0.05 across layer scans"
58 + ],
59 + "methods_in_agreement": [],
60 + "interventions": [],
61 + "replication_rate": 1.0,
62 + "per_dataset_agreement": null,
63 + "ablation_schemes": [],
64 + "featurizer_class": "natural-basis (mean-pooled residual)",
65 + "intervention_protocol": "none (observational map — Level 1 by design)",
66 + "negative_result": true,
67 + "notes": "NEGATIVE RESULT: the random-init architecture twin reaches task accuracy 1.00 at every layer for every property — the probe map is indistinguishable from the architecture+tokenizer null on these template promptsets. This map is published as evidence that probe maps on lexically separable classes are uninformative about trained structure. See analysis.md.",
68 + "author": "Simon-Pierre Boucher",
69 + "contact": "contact@spboucher.ai",
70 + "website": "https://modelmap.io",
71 + "schema_version": "0.1"
72 +}
added atlas/qwen3-0.6b-4bit/probes/v1/provenance.json +93 −0
@@ -0,0 +1,93 @@
1 +{
2 + "author": "Simon-Pierre Boucher",
3 + "contact": "contact@spboucher.ai",
4 + "website": "https://modelmap.io",
5 + "model_id": "mlx-community/Qwen3-0.6B-4bit",
6 + "map_type": "probes",
7 + "version": "v1",
8 + "commit": "3935e7933294b9c1293cd31b887126402fc53115",
9 + "model_hash": "392e8d466d56100ada00eb82031fb854297fc9e389b7d303eba3af114e87bce2",
10 + "config": {
11 + "model": "mlx-community/Qwen3-0.6B-4bit",
12 + "seeds": [
13 + 0,
14 + 1,
15 + 2,
16 + 3,
17 + 4
18 + ],
19 + "top_k": 5,
20 + "fdr_q": 0.05,
21 + "promptsets": {
22 + "version": "v1",
23 + "seed": 12345,
24 + "n_per_class": 120,
25 + "author": "Simon-Pierre Boucher",
26 + "contact": "contact@spboucher.ai",
27 + "website": "https://modelmap.io",
28 + "limitation": "template-generated v1; natural-corpus v2 registered",
29 + "files": {
30 + "arith_A.jsonl": {
31 + "sha256": "2fd80600d8a1b4ad89fed0cdeba2d5d6c33e795f7000553c08addc361305e074",
32 + "n": 240
33 + },
34 + "arith_B.jsonl": {
35 + "sha256": "04bb0eceb4b9262e090cd45b11d77487e438d14859a809d98377ac1807e3a502",
36 + "n": 240
37 + },
38 + "code_prose_A.jsonl": {
39 + "sha256": "dfbfb13dade0fd0be1202b0a745cfd03dfa796f2de4e22c6eca116abab71415c",
40 + "n": 240
41 + },
42 + "code_prose_B.jsonl": {
43 + "sha256": "e9c3f78d8754b0ad5c8921ed361fd4a4a8e824a3c116d2cb3f372417793fd4d7",
44 + "n": 240
45 + },
46 + "lang_id_A.jsonl": {
47 + "sha256": "43bd7ed12d2d9f95d643556a1a05a75623f32201b6f47fdc7d580b5f0cdce30a",
48 + "n": 240
49 + },
50 + "lang_id_B.jsonl": {
51 + "sha256": "833435dae9a61ae04cf354484693c821f6a88dfceaf19865b9da2947c9d71b52",
52 + "n": 240
53 + }
54 + }
55 + }
56 + },
57 + "seed": [
58 + 0,
59 + 1,
60 + 2,
61 + 3,
62 + 4
63 + ],
64 + "hardware_manifest": {
65 + "author": "Simon-Pierre Boucher",
66 + "contact": "contact@spboucher.ai",
67 + "website": "https://modelmap.io",
68 + "chip": {
69 + "brand": "Apple M5 Max",
70 + "cores_total": 18,
71 + "cores_performance": 6,
72 + "cores_efficiency": 12
73 + },
74 + "memory": {
75 + "unified_gb": 48.0,
76 + "pagesize": 16384
77 + },
78 + "os": {
79 + "system": "Darwin",
80 + "version": "27.0",
81 + "arch": "arm64"
82 + },
83 + "software": {
84 + "python": "3.14.4",
85 + "numpy": "2.5.2",
86 + "mlx": "0.32.0",
87 + "torch": "2.13.0",
88 + "safetensors": "0.8.0"
89 + }
90 + },
91 + "created": "2026-08-12",
92 + "source_results": "results/expA_probe_reliability/20260812T062605Z/results.json"
93 +}
added benchmarks/promptsets/arith_A.jsonl +240 −0
@@ -0,0 +1,240 @@
1 +{"text": "The lead engineer finally preferred the older method.", "label": "plain"}
2 +{"text": "A local historian quietly returned before the first frost.", "label": "plain"}
3 +{"text": "The apprentice later reviewed the plans before the meeting.", "label": "plain"}
4 +{"text": "The lead engineer eventually returned before the first frost.", "label": "plain"}
5 +{"text": "A local historian reluctantly questioned the original estimate.", "label": "plain"}
6 +{"text": "Calculate 38 + 42 and report the result.", "label": "arith"}
7 +{"text": "Calculate 48 + 91 and report the result.", "label": "arith"}
8 +{"text": "Calculate 59 + 21 and report the result.", "label": "arith"}
9 +{"text": "Calculate 90 + 18 and report the result.", "label": "arith"}
10 +{"text": "Calculate 65 + 85 and report the result.", "label": "arith"}
11 +{"text": "An early visitor quietly preferred the older method.", "label": "plain"}
12 +{"text": "The lead engineer quietly preferred the older method.", "label": "plain"}
13 +{"text": "Calculate 52 + 89 and report the result.", "label": "arith"}
14 +{"text": "A local historian eventually kept detailed notes for years.", "label": "plain"}
15 +{"text": "Calculate 60 + 64 and report the result.", "label": "arith"}
16 +{"text": "Calculate 83 + 11 and report the result.", "label": "arith"}
17 +{"text": "Her assistant reluctantly kept detailed notes for years.", "label": "plain"}
18 +{"text": "Calculate 29 + 53 and report the result.", "label": "arith"}
19 +{"text": "Calculate 75 + 31 and report the result.", "label": "arith"}
20 +{"text": "The committee finally questioned the original estimate.", "label": "plain"}
21 +{"text": "The apprentice reluctantly questioned the original estimate.", "label": "plain"}
22 +{"text": "The night watchman quietly reviewed the plans before the meeting.", "label": "plain"}
23 +{"text": "The apprentice later kept detailed notes for years.", "label": "plain"}
24 +{"text": "The lead engineer reluctantly described the process in a letter.", "label": "plain"}
25 +{"text": "Calculate 67 + 33 and report the result.", "label": "arith"}
26 +{"text": "The night watchman finally described the process in a letter.", "label": "plain"}
27 +{"text": "Calculate 53 + 35 and report the result.", "label": "arith"}
28 +{"text": "The town council finally returned before the first frost.", "label": "plain"}
29 +{"text": "A local historian later described the process in a letter.", "label": "plain"}
30 +{"text": "Calculate 30 + 34 and report the result.", "label": "arith"}
31 +{"text": "A local historian eventually reviewed the plans before the meeting.", "label": "plain"}
32 +{"text": "Her assistant eventually reviewed the plans before the meeting.", "label": "plain"}
33 +{"text": "The apprentice eventually questioned the original estimate.", "label": "plain"}
34 +{"text": "Calculate 62 + 81 and report the result.", "label": "arith"}
35 +{"text": "Calculate 43 + 12 and report the result.", "label": "arith"}
36 +{"text": "Calculate 57 + 38 and report the result.", "label": "arith"}
37 +{"text": "Calculate 67 + 92 and report the result.", "label": "arith"}
38 +{"text": "The apprentice reluctantly preferred the older method.", "label": "plain"}
39 +{"text": "Calculate 87 + 66 and report the result.", "label": "arith"}
40 +{"text": "Calculate 25 + 86 and report the result.", "label": "arith"}
41 +{"text": "Calculate 13 + 76 and report the result.", "label": "arith"}
42 +{"text": "Her assistant quietly questioned the original estimate.", "label": "plain"}
43 +{"text": "An early visitor later reviewed the plans before the meeting.", "label": "plain"}
44 +{"text": "Calculate 88 + 54 and report the result.", "label": "arith"}
45 +{"text": "Calculate 22 + 34 and report the result.", "label": "arith"}
46 +{"text": "The committee later questioned the original estimate.", "label": "plain"}
47 +{"text": "Calculate 72 + 39 and report the result.", "label": "arith"}
48 +{"text": "The town council reluctantly reviewed the plans before the meeting.", "label": "plain"}
49 +{"text": "Calculate 47 + 30 and report the result.", "label": "arith"}
50 +{"text": "Calculate 26 + 76 and report the result.", "label": "arith"}
51 +{"text": "Calculate 18 + 67 and report the result.", "label": "arith"}
52 +{"text": "The apprentice reluctantly returned before the first frost.", "label": "plain"}
53 +{"text": "Her assistant quietly preferred the older method.", "label": "plain"}
54 +{"text": "Calculate 39 + 40 and report the result.", "label": "arith"}
55 +{"text": "A local historian reluctantly preferred the older method.", "label": "plain"}
56 +{"text": "The committee eventually questioned the original estimate.", "label": "plain"}
57 +{"text": "Calculate 38 + 49 and report the result.", "label": "arith"}
58 +{"text": "Calculate 86 + 89 and report the result.", "label": "arith"}
59 +{"text": "Calculate 22 + 97 and report the result.", "label": "arith"}
60 +{"text": "The night watchman finally reviewed the plans before the meeting.", "label": "plain"}
61 +{"text": "Calculate 48 + 30 and report the result.", "label": "arith"}
62 +{"text": "The night watchman quietly preferred the older method.", "label": "plain"}
63 +{"text": "Calculate 35 + 85 and report the result.", "label": "arith"}
64 +{"text": "Calculate 16 + 40 and report the result.", "label": "arith"}
65 +{"text": "The committee reluctantly kept detailed notes for years.", "label": "plain"}
66 +{"text": "Calculate 78 + 63 and report the result.", "label": "arith"}
67 +{"text": "An early visitor reluctantly described the process in a letter.", "label": "plain"}
68 +{"text": "Calculate 35 + 14 and report the result.", "label": "arith"}
69 +{"text": "Calculate 50 + 82 and report the result.", "label": "arith"}
70 +{"text": "Her assistant quietly reviewed the plans before the meeting.", "label": "plain"}
71 +{"text": "Calculate 37 + 22 and report the result.", "label": "arith"}
72 +{"text": "The committee eventually kept detailed notes for years.", "label": "plain"}
73 +{"text": "Calculate 71 + 15 and report the result.", "label": "arith"}
74 +{"text": "The town council later questioned the original estimate.", "label": "plain"}
75 +{"text": "The committee quietly kept detailed notes for years.", "label": "plain"}
76 +{"text": "The town council finally questioned the original estimate.", "label": "plain"}
77 +{"text": "Calculate 24 + 75 and report the result.", "label": "arith"}
78 +{"text": "Calculate 21 + 31 and report the result.", "label": "arith"}
79 +{"text": "Her assistant finally reviewed the plans before the meeting.", "label": "plain"}
80 +{"text": "Calculate 18 + 44 and report the result.", "label": "arith"}
81 +{"text": "Calculate 67 + 25 and report the result.", "label": "arith"}
82 +{"text": "Calculate 65 + 34 and report the result.", "label": "arith"}
83 +{"text": "The town council later reviewed the plans before the meeting.", "label": "plain"}
84 +{"text": "The committee finally reviewed the plans before the meeting.", "label": "plain"}
85 +{"text": "Calculate 85 + 50 and report the result.", "label": "arith"}
86 +{"text": "The night watchman eventually preferred the older method.", "label": "plain"}
87 +{"text": "Calculate 29 + 60 and report the result.", "label": "arith"}
88 +{"text": "Calculate 25 + 47 and report the result.", "label": "arith"}
89 +{"text": "Calculate 33 + 86 and report the result.", "label": "arith"}
90 +{"text": "The lead engineer reluctantly reviewed the plans before the meeting.", "label": "plain"}
91 +{"text": "Calculate 65 + 61 and report the result.", "label": "arith"}
92 +{"text": "Calculate 44 + 17 and report the result.", "label": "arith"}
93 +{"text": "Calculate 31 + 54 and report the result.", "label": "arith"}
94 +{"text": "Her assistant eventually described the process in a letter.", "label": "plain"}
95 +{"text": "Calculate 46 + 96 and report the result.", "label": "arith"}
96 +{"text": "Calculate 24 + 30 and report the result.", "label": "arith"}
97 +{"text": "Calculate 75 + 27 and report the result.", "label": "arith"}
98 +{"text": "Calculate 58 + 68 and report the result.", "label": "arith"}
99 +{"text": "Her assistant quietly returned before the first frost.", "label": "plain"}
100 +{"text": "Calculate 15 + 72 and report the result.", "label": "arith"}
101 +{"text": "Calculate 50 + 36 and report the result.", "label": "arith"}
102 +{"text": "Calculate 64 + 12 and report the result.", "label": "arith"}
103 +{"text": "The committee later described the process in a letter.", "label": "plain"}
104 +{"text": "The apprentice finally questioned the original estimate.", "label": "plain"}
105 +{"text": "Calculate 55 + 71 and report the result.", "label": "arith"}
106 +{"text": "Her assistant reluctantly preferred the older method.", "label": "plain"}
107 +{"text": "The committee quietly preferred the older method.", "label": "plain"}
108 +{"text": "The night watchman later reviewed the plans before the meeting.", "label": "plain"}
109 +{"text": "Calculate 12 + 44 and report the result.", "label": "arith"}
110 +{"text": "Calculate 57 + 96 and report the result.", "label": "arith"}
111 +{"text": "Calculate 42 + 45 and report the result.", "label": "arith"}
112 +{"text": "An early visitor quietly questioned the original estimate.", "label": "plain"}
113 +{"text": "Calculate 54 + 35 and report the result.", "label": "arith"}
114 +{"text": "Calculate 45 + 16 and report the result.", "label": "arith"}
115 +{"text": "The town council quietly described the process in a letter.", "label": "plain"}
116 +{"text": "Calculate 35 + 96 and report the result.", "label": "arith"}
117 +{"text": "Calculate 12 + 72 and report the result.", "label": "arith"}
118 +{"text": "A local historian eventually questioned the original estimate.", "label": "plain"}
119 +{"text": "The night watchman later kept detailed notes for years.", "label": "plain"}
120 +{"text": "Calculate 54 + 38 and report the result.", "label": "arith"}
121 +{"text": "The lead engineer quietly reviewed the plans before the meeting.", "label": "plain"}
122 +{"text": "Calculate 64 + 82 and report the result.", "label": "arith"}
123 +{"text": "The town council eventually kept detailed notes for years.", "label": "plain"}
124 +{"text": "The town council finally preferred the older method.", "label": "plain"}
125 +{"text": "Calculate 16 + 64 and report the result.", "label": "arith"}
126 +{"text": "A local historian later questioned the original estimate.", "label": "plain"}
127 +{"text": "The lead engineer later preferred the older method.", "label": "plain"}
128 +{"text": "The night watchman later questioned the original estimate.", "label": "plain"}
129 +{"text": "Calculate 65 + 18 and report the result.", "label": "arith"}
130 +{"text": "The committee reluctantly reviewed the plans before the meeting.", "label": "plain"}
131 +{"text": "An early visitor eventually reviewed the plans before the meeting.", "label": "plain"}
132 +{"text": "Calculate 59 + 55 and report the result.", "label": "arith"}
133 +{"text": "The lead engineer finally kept detailed notes for years.", "label": "plain"}
134 +{"text": "Calculate 26 + 16 and report the result.", "label": "arith"}
135 +{"text": "A local historian finally questioned the original estimate.", "label": "plain"}
136 +{"text": "Calculate 77 + 27 and report the result.", "label": "arith"}
137 +{"text": "An early visitor eventually kept detailed notes for years.", "label": "plain"}
138 +{"text": "Calculate 76 + 23 and report the result.", "label": "arith"}
139 +{"text": "An early visitor finally preferred the older method.", "label": "plain"}
140 +{"text": "The committee reluctantly returned before the first frost.", "label": "plain"}
141 +{"text": "The night watchman eventually returned before the first frost.", "label": "plain"}
142 +{"text": "The lead engineer quietly returned before the first frost.", "label": "plain"}
143 +{"text": "Calculate 50 + 44 and report the result.", "label": "arith"}
144 +{"text": "The town council eventually reviewed the plans before the meeting.", "label": "plain"}
145 +{"text": "Calculate 55 + 41 and report the result.", "label": "arith"}
146 +{"text": "Calculate 62 + 44 and report the result.", "label": "arith"}
147 +{"text": "Calculate 20 + 28 and report the result.", "label": "arith"}
148 +{"text": "The lead engineer later reviewed the plans before the meeting.", "label": "plain"}
149 +{"text": "Calculate 49 + 69 and report the result.", "label": "arith"}
150 +{"text": "Calculate 91 + 73 and report the result.", "label": "arith"}
151 +{"text": "The committee eventually described the process in a letter.", "label": "plain"}
152 +{"text": "Calculate 14 + 24 and report the result.", "label": "arith"}
153 +{"text": "The night watchman quietly kept detailed notes for years.", "label": "plain"}
154 +{"text": "The apprentice quietly returned before the first frost.", "label": "plain"}
155 +{"text": "Her assistant reluctantly questioned the original estimate.", "label": "plain"}
156 +{"text": "The apprentice quietly preferred the older method.", "label": "plain"}
157 +{"text": "The committee finally kept detailed notes for years.", "label": "plain"}
158 +{"text": "The lead engineer later described the process in a letter.", "label": "plain"}
159 +{"text": "Calculate 20 + 84 and report the result.", "label": "arith"}
160 +{"text": "The town council later described the process in a letter.", "label": "plain"}
161 +{"text": "The night watchman reluctantly questioned the original estimate.", "label": "plain"}
162 +{"text": "Calculate 67 + 43 and report the result.", "label": "arith"}
163 +{"text": "A local historian later kept detailed notes for years.", "label": "plain"}
164 +{"text": "Calculate 54 + 45 and report the result.", "label": "arith"}
165 +{"text": "Calculate 57 + 61 and report the result.", "label": "arith"}
166 +{"text": "Calculate 62 + 45 and report the result.", "label": "arith"}
167 +{"text": "The lead engineer eventually questioned the original estimate.", "label": "plain"}
168 +{"text": "The apprentice eventually kept detailed notes for years.", "label": "plain"}
169 +{"text": "The committee later reviewed the plans before the meeting.", "label": "plain"}
170 +{"text": "Calculate 34 + 78 and report the result.", "label": "arith"}
171 +{"text": "Calculate 72 + 56 and report the result.", "label": "arith"}
172 +{"text": "Calculate 65 + 44 and report the result.", "label": "arith"}
173 +{"text": "A local historian finally described the process in a letter.", "label": "plain"}
174 +{"text": "The apprentice later preferred the older method.", "label": "plain"}
175 +{"text": "Calculate 87 + 48 and report the result.", "label": "arith"}
176 +{"text": "Calculate 46 + 85 and report the result.", "label": "arith"}
177 +{"text": "The apprentice finally described the process in a letter.", "label": "plain"}
178 +{"text": "A local historian reluctantly kept detailed notes for years.", "label": "plain"}
179 +{"text": "A local historian eventually described the process in a letter.", "label": "plain"}
180 +{"text": "Calculate 14 + 49 and report the result.", "label": "arith"}
181 +{"text": "The night watchman later described the process in a letter.", "label": "plain"}
182 +{"text": "Her assistant later questioned the original estimate.", "label": "plain"}
183 +{"text": "The apprentice quietly reviewed the plans before the meeting.", "label": "plain"}
184 +{"text": "Calculate 35 + 27 and report the result.", "label": "arith"}
185 +{"text": "An early visitor eventually described the process in a letter.", "label": "plain"}
186 +{"text": "Calculate 12 + 58 and report the result.", "label": "arith"}
187 +{"text": "The night watchman eventually described the process in a letter.", "label": "plain"}
188 +{"text": "Calculate 44 + 65 and report the result.", "label": "arith"}
189 +{"text": "Calculate 71 + 48 and report the result.", "label": "arith"}
190 +{"text": "Calculate 77 + 84 and report the result.", "label": "arith"}
191 +{"text": "Calculate 20 + 58 and report the result.", "label": "arith"}
192 +{"text": "Calculate 75 + 49 and report the result.", "label": "arith"}
193 +{"text": "Calculate 51 + 30 and report the result.", "label": "arith"}
194 +{"text": "Calculate 28 + 73 and report the result.", "label": "arith"}
195 +{"text": "The night watchman reluctantly preferred the older method.", "label": "plain"}
196 +{"text": "The lead engineer later questioned the original estimate.", "label": "plain"}
197 +{"text": "An early visitor later returned before the first frost.", "label": "plain"}
198 +{"text": "Calculate 56 + 78 and report the result.", "label": "arith"}
199 +{"text": "An early visitor eventually returned before the first frost.", "label": "plain"}
200 +{"text": "The apprentice later described the process in a letter.", "label": "plain"}
201 +{"text": "Calculate 89 + 84 and report the result.", "label": "arith"}
202 +{"text": "Calculate 76 + 90 and report the result.", "label": "arith"}
203 +{"text": "Calculate 18 + 21 and report the result.", "label": "arith"}
204 +{"text": "Her assistant later reviewed the plans before the meeting.", "label": "plain"}
205 +{"text": "A local historian finally preferred the older method.", "label": "plain"}
206 +{"text": "An early visitor reluctantly reviewed the plans before the meeting.", "label": "plain"}
207 +{"text": "Calculate 21 + 29 and report the result.", "label": "arith"}
208 +{"text": "The night watchman later returned before the first frost.", "label": "plain"}
209 +{"text": "The committee finally described the process in a letter.", "label": "plain"}
210 +{"text": "The night watchman quietly returned before the first frost.", "label": "plain"}
211 +{"text": "Calculate 79 + 67 and report the result.", "label": "arith"}
212 +{"text": "The apprentice finally reviewed the plans before the meeting.", "label": "plain"}
213 +{"text": "Calculate 76 + 64 and report the result.", "label": "arith"}
214 +{"text": "Calculate 28 + 17 and report the result.", "label": "arith"}
215 +{"text": "The lead engineer finally questioned the original estimate.", "label": "plain"}
216 +{"text": "Calculate 97 + 29 and report the result.", "label": "arith"}
217 +{"text": "Calculate 35 + 79 and report the result.", "label": "arith"}
218 +{"text": "An early visitor finally described the process in a letter.", "label": "plain"}
219 +{"text": "Calculate 66 + 86 and report the result.", "label": "arith"}
220 +{"text": "The committee reluctantly preferred the older method.", "label": "plain"}
221 +{"text": "An early visitor later preferred the older method.", "label": "plain"}
222 +{"text": "Calculate 25 + 56 and report the result.", "label": "arith"}
223 +{"text": "Calculate 44 + 75 and report the result.", "label": "arith"}
224 +{"text": "Her assistant quietly described the process in a letter.", "label": "plain"}
225 +{"text": "The night watchman reluctantly returned before the first frost.", "label": "plain"}
226 +{"text": "Calculate 96 + 51 and report the result.", "label": "arith"}
227 +{"text": "The town council later kept detailed notes for years.", "label": "plain"}
228 +{"text": "The apprentice finally returned before the first frost.", "label": "plain"}
229 +{"text": "A local historian eventually preferred the older method.", "label": "plain"}
230 +{"text": "Calculate 65 + 37 and report the result.", "label": "arith"}
231 +{"text": "Calculate 43 + 51 and report the result.", "label": "arith"}
232 +{"text": "Calculate 55 + 19 and report the result.", "label": "arith"}
233 +{"text": "The lead engineer quietly described the process in a letter.", "label": "plain"}
234 +{"text": "Calculate 95 + 16 and report the result.", "label": "arith"}
235 +{"text": "The night watchman eventually reviewed the plans before the meeting.", "label": "plain"}
236 +{"text": "Calculate 35 + 55 and report the result.", "label": "arith"}
237 +{"text": "A local historian finally reviewed the plans before the meeting.", "label": "plain"}
238 +{"text": "The lead engineer finally returned before the first frost.", "label": "plain"}
239 +{"text": "The town council reluctantly preferred the older method.", "label": "plain"}
240 +{"text": "Calculate 77 + 30 and report the result.", "label": "arith"}
added benchmarks/promptsets/arith_B.jsonl +240 −0
@@ -0,0 +1,240 @@
1 +{"text": "If the crate holds 23 jars and 38 more arrive, how many jars are there in total?", "label": "arith"}
2 +{"text": "According to the vast report, the archive changed hands twice last century.", "label": "plain"}
3 +{"text": "If the crate holds 17 jars and 84 more arrive, how many jars are there in total?", "label": "arith"}
4 +{"text": "According to the ancient report, the harbor changed hands twice last century.", "label": "plain"}
5 +{"text": "According to the quiet report, the market attracted visitors from the region.", "label": "plain"}
6 +{"text": "According to the vast report, the observatory remained open despite the storm.", "label": "plain"}
7 +{"text": "According to the ancient report, the observatory changed hands twice last century.", "label": "plain"}
8 +{"text": "According to the restored report, the observatory changed hands twice last century.", "label": "plain"}
9 +{"text": "If the crate holds 29 jars and 72 more arrive, how many jars are there in total?", "label": "arith"}
10 +{"text": "If the crate holds 32 jars and 34 more arrive, how many jars are there in total?", "label": "arith"}
11 +{"text": "According to the famous report, the lighthouse attracted visitors from the region.", "label": "plain"}
12 +{"text": "If the crate holds 16 jars and 48 more arrive, how many jars are there in total?", "label": "arith"}
13 +{"text": "If the crate holds 27 jars and 66 more arrive, how many jars are there in total?", "label": "arith"}
14 +{"text": "If the crate holds 43 jars and 64 more arrive, how many jars are there in total?", "label": "arith"}
15 +{"text": "If the crate holds 14 jars and 56 more arrive, how many jars are there in total?", "label": "arith"}
16 +{"text": "According to the crowded report, the archive attracted visitors from the region.", "label": "plain"}
17 +{"text": "According to the restored report, the vineyard changed hands twice last century.", "label": "plain"}
18 +{"text": "If the crate holds 89 jars and 36 more arrive, how many jars are there in total?", "label": "arith"}
19 +{"text": "If the crate holds 70 jars and 51 more arrive, how many jars are there in total?", "label": "arith"}
20 +{"text": "According to the abandoned report, the observatory changed hands twice last century.", "label": "plain"}
21 +{"text": "If the crate holds 12 jars and 65 more arrive, how many jars are there in total?", "label": "arith"}
22 +{"text": "If the crate holds 31 jars and 28 more arrive, how many jars are there in total?", "label": "arith"}
23 +{"text": "According to the quiet report, the observatory required constant maintenance.", "label": "plain"}
24 +{"text": "According to the quiet report, the observatory was documented in the survey.", "label": "plain"}
25 +{"text": "According to the ancient report, the market required constant maintenance.", "label": "plain"}
26 +{"text": "If the crate holds 66 jars and 66 more arrive, how many jars are there in total?", "label": "arith"}
27 +{"text": "If the crate holds 64 jars and 53 more arrive, how many jars are there in total?", "label": "arith"}
28 +{"text": "According to the ancient report, the archive remained open despite the storm.", "label": "plain"}
29 +{"text": "According to the modest report, the archive changed hands twice last century.", "label": "plain"}
30 +{"text": "According to the vast report, the orchard was documented in the survey.", "label": "plain"}
31 +{"text": "If the crate holds 29 jars and 14 more arrive, how many jars are there in total?", "label": "arith"}
32 +{"text": "According to the ancient report, the harbor was documented in the survey.", "label": "plain"}
33 +{"text": "If the crate holds 21 jars and 36 more arrive, how many jars are there in total?", "label": "arith"}
34 +{"text": "If the crate holds 37 jars and 84 more arrive, how many jars are there in total?", "label": "arith"}
35 +{"text": "If the crate holds 71 jars and 35 more arrive, how many jars are there in total?", "label": "arith"}
36 +{"text": "According to the quiet report, the workshop changed hands twice last century.", "label": "plain"}
37 +{"text": "If the crate holds 86 jars and 36 more arrive, how many jars are there in total?", "label": "arith"}
38 +{"text": "According to the quiet report, the orchard was documented in the survey.", "label": "plain"}
39 +{"text": "According to the quiet report, the vineyard changed hands twice last century.", "label": "plain"}
40 +{"text": "If the crate holds 30 jars and 83 more arrive, how many jars are there in total?", "label": "arith"}
41 +{"text": "According to the abandoned report, the lighthouse changed hands twice last century.", "label": "plain"}
42 +{"text": "According to the restored report, the harbor attracted visitors from the region.", "label": "plain"}
43 +{"text": "If the crate holds 32 jars and 29 more arrive, how many jars are there in total?", "label": "arith"}
44 +{"text": "If the crate holds 12 jars and 12 more arrive, how many jars are there in total?", "label": "arith"}
45 +{"text": "If the crate holds 80 jars and 18 more arrive, how many jars are there in total?", "label": "arith"}
46 +{"text": "If the crate holds 46 jars and 39 more arrive, how many jars are there in total?", "label": "arith"}
47 +{"text": "If the crate holds 31 jars and 36 more arrive, how many jars are there in total?", "label": "arith"}
48 +{"text": "According to the modest report, the orchard changed hands twice last century.", "label": "plain"}
49 +{"text": "According to the quiet report, the archive required constant maintenance.", "label": "plain"}
50 +{"text": "According to the ancient report, the archive changed hands twice last century.", "label": "plain"}
51 +{"text": "According to the vast report, the market changed hands twice last century.", "label": "plain"}
52 +{"text": "According to the restored report, the library required constant maintenance.", "label": "plain"}
53 +{"text": "According to the vast report, the library attracted visitors from the region.", "label": "plain"}
54 +{"text": "According to the modest report, the workshop changed hands twice last century.", "label": "plain"}
55 +{"text": "If the crate holds 28 jars and 84 more arrive, how many jars are there in total?", "label": "arith"}
56 +{"text": "According to the restored report, the workshop required constant maintenance.", "label": "plain"}
57 +{"text": "If the crate holds 30 jars and 69 more arrive, how many jars are there in total?", "label": "arith"}
58 +{"text": "According to the abandoned report, the vineyard remained open despite the storm.", "label": "plain"}
59 +{"text": "If the crate holds 78 jars and 26 more arrive, how many jars are there in total?", "label": "arith"}
60 +{"text": "According to the vast report, the observatory changed hands twice last century.", "label": "plain"}
61 +{"text": "If the crate holds 72 jars and 47 more arrive, how many jars are there in total?", "label": "arith"}
62 +{"text": "According to the abandoned report, the observatory stood at the edge of town.", "label": "plain"}
63 +{"text": "If the crate holds 34 jars and 43 more arrive, how many jars are there in total?", "label": "arith"}
64 +{"text": "If the crate holds 21 jars and 13 more arrive, how many jars are there in total?", "label": "arith"}
65 +{"text": "According to the famous report, the lighthouse remained open despite the storm.", "label": "plain"}
66 +{"text": "According to the abandoned report, the glacier stood at the edge of town.", "label": "plain"}
67 +{"text": "If the crate holds 35 jars and 81 more arrive, how many jars are there in total?", "label": "arith"}
68 +{"text": "If the crate holds 66 jars and 27 more arrive, how many jars are there in total?", "label": "arith"}
69 +{"text": "If the crate holds 54 jars and 58 more arrive, how many jars are there in total?", "label": "arith"}
70 +{"text": "According to the modest report, the lighthouse required constant maintenance.", "label": "plain"}
71 +{"text": "If the crate holds 52 jars and 41 more arrive, how many jars are there in total?", "label": "arith"}
72 +{"text": "According to the abandoned report, the glacier required constant maintenance.", "label": "plain"}
73 +{"text": "According to the modest report, the observatory attracted visitors from the region.", "label": "plain"}
74 +{"text": "If the crate holds 83 jars and 89 more arrive, how many jars are there in total?", "label": "arith"}
75 +{"text": "If the crate holds 52 jars and 27 more arrive, how many jars are there in total?", "label": "arith"}
76 +{"text": "According to the famous report, the harbor changed hands twice last century.", "label": "plain"}
77 +{"text": "According to the restored report, the orchard attracted visitors from the region.", "label": "plain"}
78 +{"text": "According to the modest report, the library stood at the edge of town.", "label": "plain"}
79 +{"text": "If the crate holds 31 jars and 66 more arrive, how many jars are there in total?", "label": "arith"}
80 +{"text": "According to the modest report, the market required constant maintenance.", "label": "plain"}
81 +{"text": "According to the vast report, the archive was documented in the survey.", "label": "plain"}
82 +{"text": "If the crate holds 84 jars and 33 more arrive, how many jars are there in total?", "label": "arith"}
83 +{"text": "According to the abandoned report, the workshop attracted visitors from the region.", "label": "plain"}
84 +{"text": "If the crate holds 49 jars and 71 more arrive, how many jars are there in total?", "label": "arith"}
85 +{"text": "According to the vast report, the observatory stood at the edge of town.", "label": "plain"}
86 +{"text": "According to the ancient report, the harbor required constant maintenance.", "label": "plain"}
87 +{"text": "According to the crowded report, the orchard attracted visitors from the region.", "label": "plain"}
88 +{"text": "If the crate holds 36 jars and 84 more arrive, how many jars are there in total?", "label": "arith"}
89 +{"text": "If the crate holds 17 jars and 52 more arrive, how many jars are there in total?", "label": "arith"}
90 +{"text": "If the crate holds 52 jars and 23 more arrive, how many jars are there in total?", "label": "arith"}
91 +{"text": "If the crate holds 42 jars and 17 more arrive, how many jars are there in total?", "label": "arith"}
92 +{"text": "According to the ancient report, the workshop changed hands twice last century.", "label": "plain"}
93 +{"text": "If the crate holds 49 jars and 12 more arrive, how many jars are there in total?", "label": "arith"}
94 +{"text": "If the crate holds 13 jars and 32 more arrive, how many jars are there in total?", "label": "arith"}
95 +{"text": "According to the modest report, the glacier attracted visitors from the region.", "label": "plain"}
96 +{"text": "According to the crowded report, the orchard stood at the edge of town.", "label": "plain"}
97 +{"text": "According to the modest report, the orchard required constant maintenance.", "label": "plain"}
98 +{"text": "If the crate holds 18 jars and 75 more arrive, how many jars are there in total?", "label": "arith"}
99 +{"text": "If the crate holds 32 jars and 54 more arrive, how many jars are there in total?", "label": "arith"}
100 +{"text": "According to the crowded report, the lighthouse stood at the edge of town.", "label": "plain"}
101 +{"text": "According to the ancient report, the observatory remained open despite the storm.", "label": "plain"}
102 +{"text": "If the crate holds 52 jars and 66 more arrive, how many jars are there in total?", "label": "arith"}
103 +{"text": "According to the vast report, the library stood at the edge of town.", "label": "plain"}
104 +{"text": "If the crate holds 34 jars and 88 more arrive, how many jars are there in total?", "label": "arith"}
105 +{"text": "If the crate holds 74 jars and 72 more arrive, how many jars are there in total?", "label": "arith"}
106 +{"text": "If the crate holds 87 jars and 53 more arrive, how many jars are there in total?", "label": "arith"}
107 +{"text": "If the crate holds 88 jars and 30 more arrive, how many jars are there in total?", "label": "arith"}
108 +{"text": "If the crate holds 87 jars and 33 more arrive, how many jars are there in total?", "label": "arith"}
109 +{"text": "If the crate holds 65 jars and 16 more arrive, how many jars are there in total?", "label": "arith"}
110 +{"text": "If the crate holds 72 jars and 49 more arrive, how many jars are there in total?", "label": "arith"}
111 +{"text": "If the crate holds 84 jars and 85 more arrive, how many jars are there in total?", "label": "arith"}
112 +{"text": "According to the abandoned report, the harbor was documented in the survey.", "label": "plain"}
113 +{"text": "According to the modest report, the harbor remained open despite the storm.", "label": "plain"}
114 +{"text": "According to the ancient report, the archive required constant maintenance.", "label": "plain"}
115 +{"text": "According to the famous report, the harbor was documented in the survey.", "label": "plain"}
116 +{"text": "According to the crowded report, the vineyard stood at the edge of town.", "label": "plain"}
117 +{"text": "If the crate holds 84 jars and 58 more arrive, how many jars are there in total?", "label": "arith"}
118 +{"text": "According to the famous report, the workshop was documented in the survey.", "label": "plain"}
119 +{"text": "According to the abandoned report, the archive required constant maintenance.", "label": "plain"}
120 +{"text": "According to the abandoned report, the archive attracted visitors from the region.", "label": "plain"}
121 +{"text": "According to the quiet report, the harbor stood at the edge of town.", "label": "plain"}
122 +{"text": "According to the vast report, the market was documented in the survey.", "label": "plain"}
123 +{"text": "According to the modest report, the observatory changed hands twice last century.", "label": "plain"}
124 +{"text": "According to the famous report, the vineyard stood at the edge of town.", "label": "plain"}
125 +{"text": "According to the quiet report, the vineyard remained open despite the storm.", "label": "plain"}
126 +{"text": "If the crate holds 34 jars and 12 more arrive, how many jars are there in total?", "label": "arith"}
127 +{"text": "According to the ancient report, the market stood at the edge of town.", "label": "plain"}
128 +{"text": "If the crate holds 73 jars and 16 more arrive, how many jars are there in total?", "label": "arith"}
129 +{"text": "According to the crowded report, the observatory remained open despite the storm.", "label": "plain"}
130 +{"text": "If the crate holds 25 jars and 18 more arrive, how many jars are there in total?", "label": "arith"}
131 +{"text": "If the crate holds 61 jars and 73 more arrive, how many jars are there in total?", "label": "arith"}
132 +{"text": "According to the vast report, the lighthouse attracted visitors from the region.", "label": "plain"}
133 +{"text": "According to the quiet report, the lighthouse remained open despite the storm.", "label": "plain"}
134 +{"text": "If the crate holds 26 jars and 83 more arrive, how many jars are there in total?", "label": "arith"}
135 +{"text": "If the crate holds 78 jars and 55 more arrive, how many jars are there in total?", "label": "arith"}
136 +{"text": "According to the abandoned report, the orchard was documented in the survey.", "label": "plain"}
137 +{"text": "If the crate holds 89 jars and 23 more arrive, how many jars are there in total?", "label": "arith"}
138 +{"text": "If the crate holds 27 jars and 54 more arrive, how many jars are there in total?", "label": "arith"}
139 +{"text": "If the crate holds 25 jars and 81 more arrive, how many jars are there in total?", "label": "arith"}
140 +{"text": "According to the modest report, the glacier remained open despite the storm.", "label": "plain"}
141 +{"text": "According to the restored report, the lighthouse remained open despite the storm.", "label": "plain"}
142 +{"text": "If the crate holds 40 jars and 76 more arrive, how many jars are there in total?", "label": "arith"}
143 +{"text": "According to the vast report, the harbor required constant maintenance.", "label": "plain"}
144 +{"text": "According to the abandoned report, the harbor remained open despite the storm.", "label": "plain"}
145 +{"text": "If the crate holds 23 jars and 61 more arrive, how many jars are there in total?", "label": "arith"}
146 +{"text": "According to the famous report, the market changed hands twice last century.", "label": "plain"}
147 +{"text": "If the crate holds 47 jars and 83 more arrive, how many jars are there in total?", "label": "arith"}
148 +{"text": "If the crate holds 76 jars and 31 more arrive, how many jars are there in total?", "label": "arith"}
149 +{"text": "According to the restored report, the observatory remained open despite the storm.", "label": "plain"}
150 +{"text": "According to the vast report, the orchard changed hands twice last century.", "label": "plain"}
151 +{"text": "If the crate holds 58 jars and 78 more arrive, how many jars are there in total?", "label": "arith"}
152 +{"text": "According to the ancient report, the glacier remained open despite the storm.", "label": "plain"}
153 +{"text": "According to the abandoned report, the market required constant maintenance.", "label": "plain"}
154 +{"text": "If the crate holds 62 jars and 49 more arrive, how many jars are there in total?", "label": "arith"}
155 +{"text": "If the crate holds 46 jars and 61 more arrive, how many jars are there in total?", "label": "arith"}
156 +{"text": "According to the restored report, the workshop attracted visitors from the region.", "label": "plain"}
157 +{"text": "If the crate holds 48 jars and 15 more arrive, how many jars are there in total?", "label": "arith"}
158 +{"text": "If the crate holds 56 jars and 30 more arrive, how many jars are there in total?", "label": "arith"}
159 +{"text": "According to the famous report, the lighthouse required constant maintenance.", "label": "plain"}
160 +{"text": "If the crate holds 66 jars and 46 more arrive, how many jars are there in total?", "label": "arith"}
161 +{"text": "According to the ancient report, the glacier attracted visitors from the region.", "label": "plain"}
162 +{"text": "If the crate holds 39 jars and 36 more arrive, how many jars are there in total?", "label": "arith"}
163 +{"text": "According to the modest report, the orchard was documented in the survey.", "label": "plain"}
164 +{"text": "If the crate holds 23 jars and 74 more arrive, how many jars are there in total?", "label": "arith"}
165 +{"text": "According to the famous report, the market attracted visitors from the region.", "label": "plain"}
166 +{"text": "If the crate holds 19 jars and 30 more arrive, how many jars are there in total?", "label": "arith"}
167 +{"text": "If the crate holds 87 jars and 83 more arrive, how many jars are there in total?", "label": "arith"}
168 +{"text": "According to the famous report, the workshop attracted visitors from the region.", "label": "plain"}
169 +{"text": "If the crate holds 47 jars and 59 more arrive, how many jars are there in total?", "label": "arith"}
170 +{"text": "According to the modest report, the lighthouse stood at the edge of town.", "label": "plain"}
171 +{"text": "If the crate holds 54 jars and 41 more arrive, how many jars are there in total?", "label": "arith"}
172 +{"text": "If the crate holds 12 jars and 83 more arrive, how many jars are there in total?", "label": "arith"}
173 +{"text": "According to the famous report, the glacier remained open despite the storm.", "label": "plain"}
174 +{"text": "According to the famous report, the observatory remained open despite the storm.", "label": "plain"}
175 +{"text": "According to the quiet report, the orchard attracted visitors from the region.", "label": "plain"}
176 +{"text": "According to the crowded report, the library changed hands twice last century.", "label": "plain"}
177 +{"text": "If the crate holds 46 jars and 84 more arrive, how many jars are there in total?", "label": "arith"}
178 +{"text": "If the crate holds 45 jars and 34 more arrive, how many jars are there in total?", "label": "arith"}
179 +{"text": "According to the crowded report, the harbor stood at the edge of town.", "label": "plain"}
180 +{"text": "If the crate holds 78 jars and 30 more arrive, how many jars are there in total?", "label": "arith"}
181 +{"text": "According to the vast report, the market required constant maintenance.", "label": "plain"}
182 +{"text": "If the crate holds 30 jars and 12 more arrive, how many jars are there in total?", "label": "arith"}
183 +{"text": "If the crate holds 86 jars and 43 more arrive, how many jars are there in total?", "label": "arith"}
184 +{"text": "According to the crowded report, the observatory was documented in the survey.", "label": "plain"}
185 +{"text": "According to the restored report, the market stood at the edge of town.", "label": "plain"}
186 +{"text": "According to the crowded report, the observatory changed hands twice last century.", "label": "plain"}
187 +{"text": "If the crate holds 58 jars and 19 more arrive, how many jars are there in total?", "label": "arith"}
188 +{"text": "According to the crowded report, the lighthouse remained open despite the storm.", "label": "plain"}
189 +{"text": "According to the abandoned report, the workshop was documented in the survey.", "label": "plain"}
190 +{"text": "According to the quiet report, the lighthouse changed hands twice last century.", "label": "plain"}
191 +{"text": "If the crate holds 41 jars and 60 more arrive, how many jars are there in total?", "label": "arith"}
192 +{"text": "According to the abandoned report, the glacier changed hands twice last century.", "label": "plain"}
193 +{"text": "According to the quiet report, the market changed hands twice last century.", "label": "plain"}
194 +{"text": "If the crate holds 82 jars and 53 more arrive, how many jars are there in total?", "label": "arith"}
195 +{"text": "According to the famous report, the orchard stood at the edge of town.", "label": "plain"}
196 +{"text": "If the crate holds 46 jars and 76 more arrive, how many jars are there in total?", "label": "arith"}
197 +{"text": "According to the quiet report, the market required constant maintenance.", "label": "plain"}
198 +{"text": "If the crate holds 76 jars and 61 more arrive, how many jars are there in total?", "label": "arith"}
199 +{"text": "If the crate holds 78 jars and 15 more arrive, how many jars are there in total?", "label": "arith"}
200 +{"text": "If the crate holds 50 jars and 25 more arrive, how many jars are there in total?", "label": "arith"}
201 +{"text": "According to the quiet report, the library attracted visitors from the region.", "label": "plain"}
202 +{"text": "If the crate holds 18 jars and 67 more arrive, how many jars are there in total?", "label": "arith"}
203 +{"text": "If the crate holds 50 jars and 76 more arrive, how many jars are there in total?", "label": "arith"}
204 +{"text": "According to the restored report, the library stood at the edge of town.", "label": "plain"}
205 +{"text": "According to the ancient report, the harbor attracted visitors from the region.", "label": "plain"}
206 +{"text": "According to the famous report, the orchard changed hands twice last century.", "label": "plain"}
207 +{"text": "According to the abandoned report, the orchard attracted visitors from the region.", "label": "plain"}
208 +{"text": "If the crate holds 68 jars and 50 more arrive, how many jars are there in total?", "label": "arith"}
209 +{"text": "According to the crowded report, the workshop was documented in the survey.", "label": "plain"}
210 +{"text": "According to the abandoned report, the observatory remained open despite the storm.", "label": "plain"}
211 +{"text": "According to the crowded report, the harbor required constant maintenance.", "label": "plain"}
212 +{"text": "If the crate holds 52 jars and 38 more arrive, how many jars are there in total?", "label": "arith"}
213 +{"text": "If the crate holds 50 jars and 38 more arrive, how many jars are there in total?", "label": "arith"}
214 +{"text": "According to the modest report, the market attracted visitors from the region.", "label": "plain"}
215 +{"text": "According to the quiet report, the lighthouse required constant maintenance.", "label": "plain"}
216 +{"text": "If the crate holds 47 jars and 13 more arrive, how many jars are there in total?", "label": "arith"}
217 +{"text": "According to the crowded report, the market attracted visitors from the region.", "label": "plain"}
218 +{"text": "If the crate holds 21 jars and 84 more arrive, how many jars are there in total?", "label": "arith"}
219 +{"text": "If the crate holds 39 jars and 20 more arrive, how many jars are there in total?", "label": "arith"}
220 +{"text": "If the crate holds 80 jars and 85 more arrive, how many jars are there in total?", "label": "arith"}
221 +{"text": "According to the crowded report, the archive remained open despite the storm.", "label": "plain"}
222 +{"text": "If the crate holds 64 jars and 71 more arrive, how many jars are there in total?", "label": "arith"}
223 +{"text": "If the crate holds 65 jars and 71 more arrive, how many jars are there in total?", "label": "arith"}
224 +{"text": "If the crate holds 53 jars and 22 more arrive, how many jars are there in total?", "label": "arith"}
225 +{"text": "If the crate holds 16 jars and 30 more arrive, how many jars are there in total?", "label": "arith"}
226 +{"text": "According to the restored report, the lighthouse stood at the edge of town.", "label": "plain"}
227 +{"text": "If the crate holds 70 jars and 69 more arrive, how many jars are there in total?", "label": "arith"}
228 +{"text": "If the crate holds 62 jars and 33 more arrive, how many jars are there in total?", "label": "arith"}
229 +{"text": "If the crate holds 76 jars and 35 more arrive, how many jars are there in total?", "label": "arith"}
230 +{"text": "If the crate holds 25 jars and 51 more arrive, how many jars are there in total?", "label": "arith"}
231 +{"text": "According to the ancient report, the observatory attracted visitors from the region.", "label": "plain"}
232 +{"text": "If the crate holds 60 jars and 69 more arrive, how many jars are there in total?", "label": "arith"}
233 +{"text": "According to the quiet report, the glacier remained open despite the storm.", "label": "plain"}
234 +{"text": "If the crate holds 24 jars and 33 more arrive, how many jars are there in total?", "label": "arith"}
235 +{"text": "If the crate holds 25 jars and 35 more arrive, how many jars are there in total?", "label": "arith"}
236 +{"text": "According to the modest report, the workshop was documented in the survey.", "label": "plain"}
237 +{"text": "If the crate holds 59 jars and 86 more arrive, how many jars are there in total?", "label": "arith"}
238 +{"text": "If the crate holds 36 jars and 75 more arrive, how many jars are there in total?", "label": "arith"}
239 +{"text": "If the crate holds 44 jars and 78 more arrive, how many jars are there in total?", "label": "arith"}
240 +{"text": "If the crate holds 27 jars and 71 more arrive, how many jars are there in total?", "label": "arith"}
added benchmarks/promptsets/code_prose_A.jsonl +240 −0
@@ -0,0 +1,240 @@
1 +{"text": "Her assistant eventually returned before the first frost.", "label": "prose"}
2 +{"text": "def clip(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
3 +{"text": "The town council eventually returned before the first frost.", "label": "prose"}
4 +{"text": "def merge(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
5 +{"text": "The town council reluctantly questioned the original estimate.", "label": "prose"}
6 +{"text": "def scale(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
7 +{"text": "The apprentice reluctantly preferred the older method.", "label": "prose"}
8 +{"text": "def clip(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
9 +{"text": "def trim(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
10 +{"text": "def clip(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
11 +{"text": "def rank(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
12 +{"text": "def total(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
13 +{"text": "The committee reluctantly questioned the original estimate.", "label": "prose"}
14 +{"text": "The committee reluctantly reviewed the plans before the meeting.", "label": "prose"}
15 +{"text": "def clip(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
16 +{"text": "def scale(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
17 +{"text": "def score(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
18 +{"text": "def fold(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
19 +{"text": "def trim(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
20 +{"text": "Her assistant reluctantly reviewed the plans before the meeting.", "label": "prose"}
21 +{"text": "def trim(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
22 +{"text": "def total(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
23 +{"text": "def fold(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
24 +{"text": "The apprentice quietly preferred the older method.", "label": "prose"}
25 +{"text": "def scale(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
26 +{"text": "def total(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
27 +{"text": "The committee eventually questioned the original estimate.", "label": "prose"}
28 +{"text": "A local historian quietly kept detailed notes for years.", "label": "prose"}
29 +{"text": "def merge(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
30 +{"text": "def rank(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
31 +{"text": "def merge(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
32 +{"text": "def clip(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
33 +{"text": "def score(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
34 +{"text": "The apprentice reluctantly reviewed the plans before the meeting.", "label": "prose"}
35 +{"text": "The apprentice eventually described the process in a letter.", "label": "prose"}
36 +{"text": "Her assistant quietly kept detailed notes for years.", "label": "prose"}
37 +{"text": "def clip(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
38 +{"text": "def scale(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
39 +{"text": "def rank(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
40 +{"text": "def trim(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
41 +{"text": "A local historian reluctantly reviewed the plans before the meeting.", "label": "prose"}
42 +{"text": "The town council quietly preferred the older method.", "label": "prose"}
43 +{"text": "The lead engineer eventually preferred the older method.", "label": "prose"}
44 +{"text": "The apprentice eventually preferred the older method.", "label": "prose"}
45 +{"text": "Her assistant reluctantly returned before the first frost.", "label": "prose"}
46 +{"text": "def fold(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
47 +{"text": "def score(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
48 +{"text": "An early visitor reluctantly returned before the first frost.", "label": "prose"}
49 +{"text": "The night watchman quietly described the process in a letter.", "label": "prose"}
50 +{"text": "def merge(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
51 +{"text": "def rank(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
52 +{"text": "The lead engineer quietly preferred the older method.", "label": "prose"}
53 +{"text": "Her assistant quietly returned before the first frost.", "label": "prose"}
54 +{"text": "An early visitor eventually kept detailed notes for years.", "label": "prose"}
55 +{"text": "Her assistant eventually kept detailed notes for years.", "label": "prose"}
56 +{"text": "Her assistant quietly reviewed the plans before the meeting.", "label": "prose"}
57 +{"text": "An early visitor quietly questioned the original estimate.", "label": "prose"}
58 +{"text": "def fold(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
59 +{"text": "The committee quietly questioned the original estimate.", "label": "prose"}
60 +{"text": "def total(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
61 +{"text": "def merge(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
62 +{"text": "The night watchman reluctantly reviewed the plans before the meeting.", "label": "prose"}
63 +{"text": "def clip(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
64 +{"text": "def merge(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
65 +{"text": "def rank(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
66 +{"text": "def total(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
67 +{"text": "def score(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
68 +{"text": "def score(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
69 +{"text": "def scale(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
70 +{"text": "def merge(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
71 +{"text": "An early visitor eventually preferred the older method.", "label": "prose"}
72 +{"text": "The night watchman quietly kept detailed notes for years.", "label": "prose"}
73 +{"text": "The apprentice reluctantly kept detailed notes for years.", "label": "prose"}
74 +{"text": "The night watchman eventually questioned the original estimate.", "label": "prose"}
75 +{"text": "def rank(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
76 +{"text": "The apprentice quietly described the process in a letter.", "label": "prose"}
77 +{"text": "Her assistant quietly described the process in a letter.", "label": "prose"}
78 +{"text": "The committee reluctantly kept detailed notes for years.", "label": "prose"}
79 +{"text": "def trim(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
80 +{"text": "An early visitor eventually described the process in a letter.", "label": "prose"}
81 +{"text": "The night watchman quietly returned before the first frost.", "label": "prose"}
82 +{"text": "The apprentice reluctantly questioned the original estimate.", "label": "prose"}
83 +{"text": "def rank(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
84 +{"text": "def fold(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
85 +{"text": "def trim(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
86 +{"text": "The committee quietly reviewed the plans before the meeting.", "label": "prose"}
87 +{"text": "A local historian eventually reviewed the plans before the meeting.", "label": "prose"}
88 +{"text": "def total(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
89 +{"text": "def score(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
90 +{"text": "The night watchman reluctantly kept detailed notes for years.", "label": "prose"}
91 +{"text": "def rank(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
92 +{"text": "def scale(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
93 +{"text": "A local historian eventually kept detailed notes for years.", "label": "prose"}
94 +{"text": "The town council quietly kept detailed notes for years.", "label": "prose"}
95 +{"text": "The town council eventually questioned the original estimate.", "label": "prose"}
96 +{"text": "def trim(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
97 +{"text": "def scale(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
98 +{"text": "The committee reluctantly returned before the first frost.", "label": "prose"}
99 +{"text": "def score(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
100 +{"text": "def fold(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
101 +{"text": "The apprentice quietly kept detailed notes for years.", "label": "prose"}
102 +{"text": "A local historian quietly reviewed the plans before the meeting.", "label": "prose"}
103 +{"text": "A local historian quietly described the process in a letter.", "label": "prose"}
104 +{"text": "A local historian quietly returned before the first frost.", "label": "prose"}
105 +{"text": "def trim(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
106 +{"text": "The town council quietly returned before the first frost.", "label": "prose"}
107 +{"text": "def merge(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
108 +{"text": "def scale(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
109 +{"text": "Her assistant eventually described the process in a letter.", "label": "prose"}
110 +{"text": "The town council reluctantly returned before the first frost.", "label": "prose"}
111 +{"text": "The town council eventually described the process in a letter.", "label": "prose"}
112 +{"text": "An early visitor reluctantly described the process in a letter.", "label": "prose"}
113 +{"text": "The night watchman quietly reviewed the plans before the meeting.", "label": "prose"}
114 +{"text": "A local historian eventually questioned the original estimate.", "label": "prose"}
115 +{"text": "def fold(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
116 +{"text": "def scale(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
117 +{"text": "def clip(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
118 +{"text": "def rank(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
119 +{"text": "The town council eventually preferred the older method.", "label": "prose"}
120 +{"text": "def clip(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
121 +{"text": "def rank(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
122 +{"text": "The apprentice reluctantly returned before the first frost.", "label": "prose"}
123 +{"text": "def fold(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
124 +{"text": "The town council reluctantly reviewed the plans before the meeting.", "label": "prose"}
125 +{"text": "The town council reluctantly preferred the older method.", "label": "prose"}
126 +{"text": "The committee eventually returned before the first frost.", "label": "prose"}
127 +{"text": "def trim(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
128 +{"text": "def total(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
129 +{"text": "def scale(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
130 +{"text": "def score(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
131 +{"text": "The committee quietly preferred the older method.", "label": "prose"}
132 +{"text": "def score(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
133 +{"text": "def rank(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
134 +{"text": "def merge(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
135 +{"text": "def clip(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
136 +{"text": "An early visitor quietly preferred the older method.", "label": "prose"}
137 +{"text": "The town council quietly reviewed the plans before the meeting.", "label": "prose"}
138 +{"text": "def rank(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
139 +{"text": "def fold(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
140 +{"text": "A local historian reluctantly described the process in a letter.", "label": "prose"}
141 +{"text": "The lead engineer reluctantly described the process in a letter.", "label": "prose"}
142 +{"text": "def score(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
143 +{"text": "The lead engineer quietly returned before the first frost.", "label": "prose"}
144 +{"text": "The town council eventually kept detailed notes for years.", "label": "prose"}
145 +{"text": "A local historian quietly questioned the original estimate.", "label": "prose"}
146 +{"text": "A local historian reluctantly questioned the original estimate.", "label": "prose"}
147 +{"text": "def fold(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
148 +{"text": "def total(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
149 +{"text": "The night watchman reluctantly described the process in a letter.", "label": "prose"}
150 +{"text": "The night watchman eventually described the process in a letter.", "label": "prose"}
151 +{"text": "A local historian eventually preferred the older method.", "label": "prose"}
152 +{"text": "def clip(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
153 +{"text": "The night watchman eventually returned before the first frost.", "label": "prose"}
154 +{"text": "def rank(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
155 +{"text": "def total(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
156 +{"text": "An early visitor reluctantly questioned the original estimate.", "label": "prose"}
157 +{"text": "The town council quietly questioned the original estimate.", "label": "prose"}
158 +{"text": "def total(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
159 +{"text": "def score(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
160 +{"text": "The lead engineer reluctantly returned before the first frost.", "label": "prose"}
161 +{"text": "The apprentice quietly questioned the original estimate.", "label": "prose"}
162 +{"text": "def rank(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
163 +{"text": "def scale(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
164 +{"text": "def rank(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
165 +{"text": "def merge(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
166 +{"text": "The lead engineer eventually questioned the original estimate.", "label": "prose"}
167 +{"text": "def total(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
168 +{"text": "def score(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
169 +{"text": "The lead engineer reluctantly kept detailed notes for years.", "label": "prose"}
170 +{"text": "def score(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
171 +{"text": "A local historian reluctantly returned before the first frost.", "label": "prose"}
172 +{"text": "def clip(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
173 +{"text": "The committee reluctantly preferred the older method.", "label": "prose"}
174 +{"text": "An early visitor eventually reviewed the plans before the meeting.", "label": "prose"}
175 +{"text": "An early visitor eventually questioned the original estimate.", "label": "prose"}
176 +{"text": "def merge(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
177 +{"text": "def score(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
178 +{"text": "An early visitor quietly kept detailed notes for years.", "label": "prose"}
179 +{"text": "def merge(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
180 +{"text": "Her assistant quietly questioned the original estimate.", "label": "prose"}
181 +{"text": "def scale(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
182 +{"text": "def trim(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
183 +{"text": "An early visitor reluctantly kept detailed notes for years.", "label": "prose"}
184 +{"text": "def trim(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
185 +{"text": "def fold(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
186 +{"text": "The night watchman eventually preferred the older method.", "label": "prose"}
187 +{"text": "The night watchman reluctantly preferred the older method.", "label": "prose"}
188 +{"text": "def score(xs):\n return [x - 3 for x in xs if x > 4]", "label": "code"}
189 +{"text": "An early visitor quietly returned before the first frost.", "label": "prose"}
190 +{"text": "An early visitor eventually returned before the first frost.", "label": "prose"}
191 +{"text": "An early visitor reluctantly preferred the older method.", "label": "prose"}
192 +{"text": "def merge(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
193 +{"text": "def total(xs):\n return [x + 1 for x in xs if x > 2]", "label": "code"}
194 +{"text": "def total(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
195 +{"text": "The apprentice eventually kept detailed notes for years.", "label": "prose"}
196 +{"text": "The apprentice quietly reviewed the plans before the meeting.", "label": "prose"}
197 +{"text": "The night watchman quietly questioned the original estimate.", "label": "prose"}
198 +{"text": "An early visitor quietly described the process in a letter.", "label": "prose"}
199 +{"text": "def scale(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
200 +{"text": "A local historian reluctantly kept detailed notes for years.", "label": "prose"}
201 +{"text": "The lead engineer quietly described the process in a letter.", "label": "prose"}
202 +{"text": "def fold(xs):\n return [x - 7 for x in xs if x > 8]", "label": "code"}
203 +{"text": "The lead engineer eventually returned before the first frost.", "label": "prose"}
204 +{"text": "The lead engineer reluctantly reviewed the plans before the meeting.", "label": "prose"}
205 +{"text": "The committee eventually kept detailed notes for years.", "label": "prose"}
206 +{"text": "The town council reluctantly described the process in a letter.", "label": "prose"}
207 +{"text": "def clip(xs):\n return [x + 2 for x in xs if x > 3]", "label": "code"}
208 +{"text": "The night watchman quietly preferred the older method.", "label": "prose"}
209 +{"text": "def clip(xs):\n return [x * 3 for x in xs if x > 4]", "label": "code"}
210 +{"text": "A local historian eventually returned before the first frost.", "label": "prose"}
211 +{"text": "def fold(xs):\n return [x * 7 for x in xs if x > 8]", "label": "code"}
212 +{"text": "The town council quietly described the process in a letter.", "label": "prose"}
213 +{"text": "An early visitor reluctantly reviewed the plans before the meeting.", "label": "prose"}
214 +{"text": "The lead engineer quietly questioned the original estimate.", "label": "prose"}
215 +{"text": "def trim(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
216 +{"text": "Her assistant eventually reviewed the plans before the meeting.", "label": "prose"}
217 +{"text": "def merge(xs):\n return [x - 1 for x in xs if x > 2]", "label": "code"}
218 +{"text": "The committee eventually reviewed the plans before the meeting.", "label": "prose"}
219 +{"text": "def trim(xs):\n return [x + 7 for x in xs if x > 8]", "label": "code"}
220 +{"text": "def scale(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
221 +{"text": "def trim(xs):\n return [x * 2 for x in xs if x > 3]", "label": "code"}
222 +{"text": "def total(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
223 +{"text": "The apprentice eventually questioned the original estimate.", "label": "prose"}
224 +{"text": "Her assistant quietly preferred the older method.", "label": "prose"}
225 +{"text": "def scale(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
226 +{"text": "def merge(xs):\n return [x - 5 for x in xs if x > 6]", "label": "code"}
227 +{"text": "def fold(xs):\n return [x + 3 for x in xs if x > 4]", "label": "code"}
228 +{"text": "def trim(xs):\n return [x * 5 for x in xs if x > 6]", "label": "code"}
229 +{"text": "The committee reluctantly described the process in a letter.", "label": "prose"}
230 +{"text": "The lead engineer eventually kept detailed notes for years.", "label": "prose"}
231 +{"text": "def total(xs):\n return [x - 2 for x in xs if x > 3]", "label": "code"}
232 +{"text": "def clip(xs):\n return [x + 5 for x in xs if x > 6]", "label": "code"}
233 +{"text": "Her assistant eventually preferred the older method.", "label": "prose"}
234 +{"text": "def fold(xs):\n return [x * 1 for x in xs if x > 2]", "label": "code"}
235 +{"text": "The committee eventually described the process in a letter.", "label": "prose"}
236 +{"text": "An early visitor quietly reviewed the plans before the meeting.", "label": "prose"}
237 +{"text": "Her assistant reluctantly described the process in a letter.", "label": "prose"}
238 +{"text": "The night watchman eventually kept detailed notes for years.", "label": "prose"}
239 +{"text": "The lead engineer reluctantly questioned the original estimate.", "label": "prose"}
240 +{"text": "The town council reluctantly kept detailed notes for years.", "label": "prose"}
added benchmarks/promptsets/code_prose_B.jsonl +240 −0
@@ -0,0 +1,240 @@
1 +{"text": "const out = cache.find(x => x.size > 0).length;", "label": "code"}
2 +{"text": "const out = edges.filter(x => x.size > 1).length;", "label": "code"}
3 +{"text": "An early visitor, though abandoned, questioned the original estimate.", "label": "prose"}
4 +{"text": "const out = cache.some(x => x.size > 9).length;", "label": "code"}
5 +{"text": "const out = rows.find(x => x.size > 9).length;", "label": "code"}
6 +{"text": "A local historian, though restored, kept detailed notes for years.", "label": "prose"}
7 +{"text": "const out = cache.map(x => x.size > 9).length;", "label": "code"}
8 +{"text": "A local historian, though abandoned, kept detailed notes for years.", "label": "prose"}
9 +{"text": "The lead engineer, though crowded, questioned the original estimate.", "label": "prose"}
10 +{"text": "const out = queue.filter(x => x.size > 1).length;", "label": "code"}
11 +{"text": "The town council, though abandoned, questioned the original estimate.", "label": "prose"}
12 +{"text": "An early visitor, though ancient, kept detailed notes for years.", "label": "prose"}
13 +{"text": "The lead engineer, though restored, kept detailed notes for years.", "label": "prose"}
14 +{"text": "const out = rows.find(x => x.size > 1).length;", "label": "code"}
15 +{"text": "Her assistant, though quiet, questioned the original estimate.", "label": "prose"}
16 +{"text": "const out = keys.find(x => x.size > 1).length;", "label": "code"}
17 +{"text": "const out = rows.filter(x => x.size > 1).length;", "label": "code"}
18 +{"text": "The apprentice, though restored, reviewed the plans before the meeting.", "label": "prose"}
19 +{"text": "const out = queue.some(x => x.size > 9).length;", "label": "code"}
20 +{"text": "The apprentice, though abandoned, kept detailed notes for years.", "label": "prose"}
21 +{"text": "The committee, though restored, kept detailed notes for years.", "label": "prose"}
22 +{"text": "const out = cache.map(x => x.size > 1).length;", "label": "code"}
23 +{"text": "A local historian, though restored, reviewed the plans before the meeting.", "label": "prose"}
24 +{"text": "The lead engineer, though restored, reviewed the plans before the meeting.", "label": "prose"}
25 +{"text": "const out = rows.some(x => x.size > 9).length;", "label": "code"}
26 +{"text": "The committee, though quiet, reviewed the plans before the meeting.", "label": "prose"}
27 +{"text": "A local historian, though ancient, reviewed the plans before the meeting.", "label": "prose"}
28 +{"text": "const out = bins.some(x => x.size > 9).length;", "label": "code"}
29 +{"text": "The committee, though quiet, questioned the original estimate.", "label": "prose"}
30 +{"text": "const out = queue.some(x => x.size > 4).length;", "label": "code"}
31 +{"text": "The town council, though crowded, kept detailed notes for years.", "label": "prose"}
32 +{"text": "A local historian, though crowded, reviewed the plans before the meeting.", "label": "prose"}
33 +{"text": "const out = keys.some(x => x.size > 1).length;", "label": "code"}
34 +{"text": "const out = cache.some(x => x.size > 4).length;", "label": "code"}
35 +{"text": "const out = keys.some(x => x.size > 0).length;", "label": "code"}
36 +{"text": "Her assistant, though ancient, kept detailed notes for years.", "label": "prose"}
37 +{"text": "The night watchman, though crowded, kept detailed notes for years.", "label": "prose"}
38 +{"text": "The apprentice, though ancient, kept detailed notes for years.", "label": "prose"}
39 +{"text": "const out = items.find(x => x.size > 0).length;", "label": "code"}
40 +{"text": "const out = queue.filter(x => x.size > 0).length;", "label": "code"}
41 +{"text": "The town council, though crowded, questioned the original estimate.", "label": "prose"}
42 +{"text": "Her assistant, though crowded, kept detailed notes for years.", "label": "prose"}
43 +{"text": "The night watchman, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
44 +{"text": "const out = queue.find(x => x.size > 9).length;", "label": "code"}
45 +{"text": "The committee, though ancient, kept detailed notes for years.", "label": "prose"}
46 +{"text": "The night watchman, though restored, reviewed the plans before the meeting.", "label": "prose"}
47 +{"text": "const out = nodes.find(x => x.size > 1).length;", "label": "code"}
48 +{"text": "const out = edges.filter(x => x.size > 4).length;", "label": "code"}
49 +{"text": "The town council, though crowded, reviewed the plans before the meeting.", "label": "prose"}
50 +{"text": "const out = items.filter(x => x.size > 1).length;", "label": "code"}
51 +{"text": "const out = keys.filter(x => x.size > 4).length;", "label": "code"}
52 +{"text": "const out = queue.find(x => x.size > 1).length;", "label": "code"}
53 +{"text": "const out = edges.some(x => x.size > 0).length;", "label": "code"}
54 +{"text": "An early visitor, though restored, questioned the original estimate.", "label": "prose"}
55 +{"text": "The committee, though abandoned, questioned the original estimate.", "label": "prose"}
56 +{"text": "const out = nodes.find(x => x.size > 4).length;", "label": "code"}
57 +{"text": "Her assistant, though restored, questioned the original estimate.", "label": "prose"}
58 +{"text": "const out = bins.filter(x => x.size > 9).length;", "label": "code"}
59 +{"text": "const out = nodes.filter(x => x.size > 0).length;", "label": "code"}
60 +{"text": "A local historian, though ancient, questioned the original estimate.", "label": "prose"}
61 +{"text": "An early visitor, though crowded, questioned the original estimate.", "label": "prose"}
62 +{"text": "const out = nodes.some(x => x.size > 0).length;", "label": "code"}
63 +{"text": "const out = rows.some(x => x.size > 1).length;", "label": "code"}
64 +{"text": "const out = bins.find(x => x.size > 9).length;", "label": "code"}
65 +{"text": "const out = cache.map(x => x.size > 0).length;", "label": "code"}
66 +{"text": "The night watchman, though ancient, questioned the original estimate.", "label": "prose"}
67 +{"text": "const out = rows.find(x => x.size > 0).length;", "label": "code"}
68 +{"text": "const out = items.map(x => x.size > 4).length;", "label": "code"}
69 +{"text": "Her assistant, though ancient, questioned the original estimate.", "label": "prose"}
70 +{"text": "An early visitor, though quiet, questioned the original estimate.", "label": "prose"}
71 +{"text": "const out = rows.filter(x => x.size > 4).length;", "label": "code"}
72 +{"text": "The committee, though crowded, questioned the original estimate.", "label": "prose"}
73 +{"text": "const out = nodes.map(x => x.size > 1).length;", "label": "code"}
74 +{"text": "The night watchman, though quiet, kept detailed notes for years.", "label": "prose"}
75 +{"text": "The apprentice, though crowded, reviewed the plans before the meeting.", "label": "prose"}
76 +{"text": "The apprentice, though crowded, kept detailed notes for years.", "label": "prose"}
77 +{"text": "The lead engineer, though ancient, reviewed the plans before the meeting.", "label": "prose"}
78 +{"text": "A local historian, though abandoned, questioned the original estimate.", "label": "prose"}
79 +{"text": "The lead engineer, though crowded, reviewed the plans before the meeting.", "label": "prose"}
80 +{"text": "const out = cache.find(x => x.size > 1).length;", "label": "code"}
81 +{"text": "const out = nodes.some(x => x.size > 1).length;", "label": "code"}
82 +{"text": "const out = rows.filter(x => x.size > 9).length;", "label": "code"}
83 +{"text": "const out = cache.filter(x => x.size > 9).length;", "label": "code"}
84 +{"text": "Her assistant, though quiet, reviewed the plans before the meeting.", "label": "prose"}
85 +{"text": "An early visitor, though restored, kept detailed notes for years.", "label": "prose"}
86 +{"text": "const out = cache.filter(x => x.size > 4).length;", "label": "code"}
87 +{"text": "const out = items.filter(x => x.size > 9).length;", "label": "code"}
88 +{"text": "Her assistant, though crowded, reviewed the plans before the meeting.", "label": "prose"}
89 +{"text": "The apprentice, though quiet, reviewed the plans before the meeting.", "label": "prose"}
90 +{"text": "const out = bins.some(x => x.size > 0).length;", "label": "code"}
91 +{"text": "The lead engineer, though quiet, reviewed the plans before the meeting.", "label": "prose"}
92 +{"text": "const out = nodes.find(x => x.size > 9).length;", "label": "code"}
93 +{"text": "The apprentice, though restored, questioned the original estimate.", "label": "prose"}
94 +{"text": "const out = rows.map(x => x.size > 0).length;", "label": "code"}
95 +{"text": "An early visitor, though quiet, reviewed the plans before the meeting.", "label": "prose"}
96 +{"text": "The night watchman, though abandoned, kept detailed notes for years.", "label": "prose"}
97 +{"text": "A local historian, though crowded, kept detailed notes for years.", "label": "prose"}
98 +{"text": "const out = items.filter(x => x.size > 0).length;", "label": "code"}
99 +{"text": "const out = rows.some(x => x.size > 0).length;", "label": "code"}
100 +{"text": "const out = rows.map(x => x.size > 1).length;", "label": "code"}
101 +{"text": "The apprentice, though ancient, reviewed the plans before the meeting.", "label": "prose"}
102 +{"text": "const out = keys.find(x => x.size > 4).length;", "label": "code"}
103 +{"text": "The town council, though abandoned, kept detailed notes for years.", "label": "prose"}
104 +{"text": "The apprentice, though quiet, kept detailed notes for years.", "label": "prose"}
105 +{"text": "const out = items.map(x => x.size > 0).length;", "label": "code"}
106 +{"text": "The committee, though ancient, reviewed the plans before the meeting.", "label": "prose"}
107 +{"text": "const out = queue.some(x => x.size > 1).length;", "label": "code"}
108 +{"text": "The night watchman, though abandoned, questioned the original estimate.", "label": "prose"}
109 +{"text": "const out = items.some(x => x.size > 0).length;", "label": "code"}
110 +{"text": "const out = edges.find(x => x.size > 0).length;", "label": "code"}
111 +{"text": "Her assistant, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
112 +{"text": "const out = cache.find(x => x.size > 4).length;", "label": "code"}
113 +{"text": "The night watchman, though crowded, questioned the original estimate.", "label": "prose"}
114 +{"text": "An early visitor, though abandoned, kept detailed notes for years.", "label": "prose"}
115 +{"text": "An early visitor, though crowded, kept detailed notes for years.", "label": "prose"}
116 +{"text": "const out = keys.map(x => x.size > 9).length;", "label": "code"}
117 +{"text": "The committee, though crowded, reviewed the plans before the meeting.", "label": "prose"}
118 +{"text": "The committee, though restored, questioned the original estimate.", "label": "prose"}
119 +{"text": "The committee, though crowded, kept detailed notes for years.", "label": "prose"}
120 +{"text": "Her assistant, though restored, reviewed the plans before the meeting.", "label": "prose"}
121 +{"text": "A local historian, though quiet, kept detailed notes for years.", "label": "prose"}
122 +{"text": "A local historian, though restored, questioned the original estimate.", "label": "prose"}
123 +{"text": "const out = items.some(x => x.size > 9).length;", "label": "code"}
124 +{"text": "The town council, though quiet, kept detailed notes for years.", "label": "prose"}
125 +{"text": "The committee, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
126 +{"text": "An early visitor, though crowded, reviewed the plans before the meeting.", "label": "prose"}
127 +{"text": "const out = queue.some(x => x.size > 0).length;", "label": "code"}
128 +{"text": "The night watchman, though quiet, questioned the original estimate.", "label": "prose"}
129 +{"text": "The night watchman, though ancient, reviewed the plans before the meeting.", "label": "prose"}
130 +{"text": "const out = items.some(x => x.size > 1).length;", "label": "code"}
131 +{"text": "const out = queue.filter(x => x.size > 9).length;", "label": "code"}
132 +{"text": "The lead engineer, though crowded, kept detailed notes for years.", "label": "prose"}
133 +{"text": "const out = edges.some(x => x.size > 9).length;", "label": "code"}
134 +{"text": "const out = rows.some(x => x.size > 4).length;", "label": "code"}
135 +{"text": "The lead engineer, though quiet, questioned the original estimate.", "label": "prose"}
136 +{"text": "const out = items.find(x => x.size > 1).length;", "label": "code"}
137 +{"text": "The apprentice, though abandoned, questioned the original estimate.", "label": "prose"}
138 +{"text": "const out = rows.map(x => x.size > 4).length;", "label": "code"}
139 +{"text": "Her assistant, though restored, kept detailed notes for years.", "label": "prose"}
140 +{"text": "const out = edges.map(x => x.size > 9).length;", "label": "code"}
141 +{"text": "const out = bins.filter(x => x.size > 4).length;", "label": "code"}
142 +{"text": "const out = nodes.map(x => x.size > 9).length;", "label": "code"}
143 +{"text": "The town council, though restored, questioned the original estimate.", "label": "prose"}
144 +{"text": "const out = cache.find(x => x.size > 9).length;", "label": "code"}
145 +{"text": "const out = queue.map(x => x.size > 1).length;", "label": "code"}
146 +{"text": "The apprentice, though ancient, questioned the original estimate.", "label": "prose"}
147 +{"text": "The committee, though ancient, questioned the original estimate.", "label": "prose"}
148 +{"text": "const out = bins.find(x => x.size > 0).length;", "label": "code"}
149 +{"text": "The town council, though ancient, questioned the original estimate.", "label": "prose"}
150 +{"text": "Her assistant, though ancient, reviewed the plans before the meeting.", "label": "prose"}
151 +{"text": "const out = nodes.some(x => x.size > 9).length;", "label": "code"}
152 +{"text": "The town council, though quiet, reviewed the plans before the meeting.", "label": "prose"}
153 +{"text": "A local historian, though quiet, reviewed the plans before the meeting.", "label": "prose"}
154 +{"text": "const out = bins.some(x => x.size > 4).length;", "label": "code"}
155 +{"text": "const out = queue.map(x => x.size > 9).length;", "label": "code"}
156 +{"text": "const out = rows.map(x => x.size > 9).length;", "label": "code"}
157 +{"text": "const out = queue.filter(x => x.size > 4).length;", "label": "code"}
158 +{"text": "const out = bins.map(x => x.size > 0).length;", "label": "code"}
159 +{"text": "The town council, though ancient, reviewed the plans before the meeting.", "label": "prose"}
160 +{"text": "const out = edges.filter(x => x.size > 9).length;", "label": "code"}
161 +{"text": "const out = keys.map(x => x.size > 0).length;", "label": "code"}
162 +{"text": "const out = queue.find(x => x.size > 0).length;", "label": "code"}
163 +{"text": "const out = nodes.some(x => x.size > 4).length;", "label": "code"}
164 +{"text": "const out = keys.filter(x => x.size > 0).length;", "label": "code"}
165 +{"text": "The lead engineer, though ancient, kept detailed notes for years.", "label": "prose"}
166 +{"text": "const out = cache.some(x => x.size > 1).length;", "label": "code"}
167 +{"text": "const out = edges.some(x => x.size > 4).length;", "label": "code"}
168 +{"text": "const out = rows.filter(x => x.size > 0).length;", "label": "code"}
169 +{"text": "The town council, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
170 +{"text": "The night watchman, though ancient, kept detailed notes for years.", "label": "prose"}
171 +{"text": "const out = bins.map(x => x.size > 1).length;", "label": "code"}
172 +{"text": "An early visitor, though quiet, kept detailed notes for years.", "label": "prose"}
173 +{"text": "const out = bins.some(x => x.size > 1).length;", "label": "code"}
174 +{"text": "The night watchman, though quiet, reviewed the plans before the meeting.", "label": "prose"}
175 +{"text": "const out = queue.find(x => x.size > 4).length;", "label": "code"}
176 +{"text": "The lead engineer, though restored, questioned the original estimate.", "label": "prose"}
177 +{"text": "const out = cache.map(x => x.size > 4).length;", "label": "code"}
178 +{"text": "const out = keys.find(x => x.size > 0).length;", "label": "code"}
179 +{"text": "The apprentice, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
180 +{"text": "The apprentice, though crowded, questioned the original estimate.", "label": "prose"}
181 +{"text": "const out = cache.filter(x => x.size > 1).length;", "label": "code"}
182 +{"text": "const out = keys.some(x => x.size > 4).length;", "label": "code"}
183 +{"text": "const out = keys.map(x => x.size > 4).length;", "label": "code"}
184 +{"text": "const out = bins.filter(x => x.size > 1).length;", "label": "code"}
185 +{"text": "const out = rows.find(x => x.size > 4).length;", "label": "code"}
186 +{"text": "The committee, though quiet, kept detailed notes for years.", "label": "prose"}
187 +{"text": "const out = items.find(x => x.size > 9).length;", "label": "code"}
188 +{"text": "A local historian, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
189 +{"text": "const out = keys.find(x => x.size > 9).length;", "label": "code"}
190 +{"text": "Her assistant, though abandoned, kept detailed notes for years.", "label": "prose"}
191 +{"text": "const out = bins.filter(x => x.size > 0).length;", "label": "code"}
192 +{"text": "A local historian, though ancient, kept detailed notes for years.", "label": "prose"}
193 +{"text": "const out = bins.map(x => x.size > 9).length;", "label": "code"}
194 +{"text": "const out = nodes.filter(x => x.size > 9).length;", "label": "code"}
195 +{"text": "The lead engineer, though quiet, kept detailed notes for years.", "label": "prose"}
196 +{"text": "const out = edges.find(x => x.size > 9).length;", "label": "code"}
197 +{"text": "The committee, though abandoned, kept detailed notes for years.", "label": "prose"}
198 +{"text": "The lead engineer, though abandoned, kept detailed notes for years.", "label": "prose"}
199 +{"text": "The town council, though quiet, questioned the original estimate.", "label": "prose"}
200 +{"text": "A local historian, though quiet, questioned the original estimate.", "label": "prose"}
201 +{"text": "const out = edges.map(x => x.size > 0).length;", "label": "code"}
202 +{"text": "const out = items.map(x => x.size > 9).length;", "label": "code"}
203 +{"text": "const out = nodes.map(x => x.size > 0).length;", "label": "code"}
204 +{"text": "const out = bins.find(x => x.size > 4).length;", "label": "code"}
205 +{"text": "The night watchman, though restored, kept detailed notes for years.", "label": "prose"}
206 +{"text": "The lead engineer, though abandoned, questioned the original estimate.", "label": "prose"}
207 +{"text": "const out = items.filter(x => x.size > 4).length;", "label": "code"}
208 +{"text": "An early visitor, though ancient, questioned the original estimate.", "label": "prose"}
209 +{"text": "const out = nodes.filter(x => x.size > 4).length;", "label": "code"}
210 +{"text": "const out = bins.find(x => x.size > 1).length;", "label": "code"}
211 +{"text": "The night watchman, though crowded, reviewed the plans before the meeting.", "label": "prose"}
212 +{"text": "const out = keys.map(x => x.size > 1).length;", "label": "code"}
213 +{"text": "const out = edges.map(x => x.size > 1).length;", "label": "code"}
214 +{"text": "An early visitor, though ancient, reviewed the plans before the meeting.", "label": "prose"}
215 +{"text": "const out = edges.map(x => x.size > 4).length;", "label": "code"}
216 +{"text": "Her assistant, though quiet, kept detailed notes for years.", "label": "prose"}
217 +{"text": "The lead engineer, though ancient, questioned the original estimate.", "label": "prose"}
218 +{"text": "Her assistant, though crowded, questioned the original estimate.", "label": "prose"}
219 +{"text": "The lead engineer, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
220 +{"text": "const out = items.some(x => x.size > 4).length;", "label": "code"}
221 +{"text": "const out = nodes.find(x => x.size > 0).length;", "label": "code"}
222 +{"text": "const out = cache.filter(x => x.size > 0).length;", "label": "code"}
223 +{"text": "An early visitor, though abandoned, reviewed the plans before the meeting.", "label": "prose"}
224 +{"text": "The apprentice, though restored, kept detailed notes for years.", "label": "prose"}
225 +{"text": "const out = items.map(x => x.size > 1).length;", "label": "code"}
226 +{"text": "const out = keys.filter(x => x.size > 9).length;", "label": "code"}
227 +{"text": "const out = edges.find(x => x.size > 1).length;", "label": "code"}
228 +{"text": "An early visitor, though restored, reviewed the plans before the meeting.", "label": "prose"}
229 +{"text": "The night watchman, though restored, questioned the original estimate.", "label": "prose"}
230 +{"text": "const out = cache.some(x => x.size > 0).length;", "label": "code"}
231 +{"text": "const out = edges.filter(x => x.size > 0).length;", "label": "code"}
232 +{"text": "const out = nodes.filter(x => x.size > 1).length;", "label": "code"}
233 +{"text": "Her assistant, though abandoned, questioned the original estimate.", "label": "prose"}
234 +{"text": "The town council, though restored, reviewed the plans before the meeting.", "label": "prose"}
235 +{"text": "The committee, though restored, reviewed the plans before the meeting.", "label": "prose"}
236 +{"text": "const out = items.find(x => x.size > 4).length;", "label": "code"}
237 +{"text": "The town council, though restored, kept detailed notes for years.", "label": "prose"}
238 +{"text": "A local historian, though crowded, questioned the original estimate.", "label": "prose"}
239 +{"text": "The town council, though ancient, kept detailed notes for years.", "label": "prose"}
240 +{"text": "The apprentice, though quiet, questioned the original estimate.", "label": "prose"}
added benchmarks/promptsets/lang_id_A.jsonl +240 −0
@@ -0,0 +1,240 @@
1 +{"text": "Le site bondé près de les archives a été documenté dans l'enquête.", "label": "fr"}
2 +{"text": "Le site ancien près de le port exigeait un entretien constant.", "label": "fr"}
3 +{"text": "The ancient site near the market remained open despite the storm.", "label": "en"}
4 +{"text": "The vast site near the archive attracted visitors from the region.", "label": "en"}
5 +{"text": "The quiet site near the harbor remained open despite the storm.", "label": "en"}
6 +{"text": "Le site restauré près de le phare a été documenté dans l'enquête.", "label": "fr"}
7 +{"text": "The abandoned site near the glacier remained open despite the storm.", "label": "en"}
8 +{"text": "Le site restauré près de le vignoble exigeait un entretien constant.", "label": "fr"}
9 +{"text": "Le site bondé près de le port a attiré des visiteurs de la région.", "label": "fr"}
10 +{"text": "Le site célèbre près de l'atelier est resté ouvert malgré la tempête.", "label": "fr"}
11 +{"text": "The vast site near the orchard stood at the edge of town.", "label": "en"}
12 +{"text": "The vast site near the observatory stood at the edge of town.", "label": "en"}
13 +{"text": "The famous site near the workshop stood at the edge of town.", "label": "en"}
14 +{"text": "Le site calme près de le marché a été documenté dans l'enquête.", "label": "fr"}
15 +{"text": "Le site modeste près de le verger a changé de mains deux fois au siècle dernier.", "label": "fr"}
16 +{"text": "Le site abandonné près de l'observatoire a changé de mains deux fois au siècle dernier.", "label": "fr"}
17 +{"text": "Le site ancien près de le marché a attiré des visiteurs de la région.", "label": "fr"}
18 +{"text": "The quiet site near the observatory changed hands twice last century.", "label": "en"}
19 +{"text": "Le site bondé près de l'observatoire est resté ouvert malgré la tempête.", "label": "fr"}
20 +{"text": "Le site modeste près de le marché se trouvait à la limite de la ville.", "label": "fr"}
21 +{"text": "The ancient site near the glacier attracted visitors from the region.", "label": "en"}
22 +{"text": "The abandoned site near the harbor required constant maintenance.", "label": "en"}
23 +{"text": "Le site célèbre près de le vignoble se trouvait à la limite de la ville.", "label": "fr"}
24 +{"text": "Le site abandonné près de les archives a été documenté dans l'enquête.", "label": "fr"}
25 +{"text": "The quiet site near the harbor required constant maintenance.", "label": "en"}
26 +{"text": "The quiet site near the vineyard required constant maintenance.", "label": "en"}
27 +{"text": "The famous site near the orchard required constant maintenance.", "label": "en"}
28 +{"text": "The quiet site near the vineyard attracted visitors from the region.", "label": "en"}
29 +{"text": "The crowded site near the archive required constant maintenance.", "label": "en"}
30 +{"text": "Le site calme près de la bibliothèque a changé de mains deux fois au siècle dernier.", "label": "fr"}
31 +{"text": "The quiet site near the archive required constant maintenance.", "label": "en"}
32 +{"text": "Le site célèbre près de l'observatoire a changé de mains deux fois au siècle dernier.", "label": "fr"}
33 +{"text": "The restored site near the library required constant maintenance.", "label": "en"}
34 +{"text": "Le site vaste près de le verger est resté ouvert malgré la tempête.", "label": "fr"}
35 +{"text": "Le site calme près de le vignoble a changé de mains deux fois au siècle dernier.", "label": "fr"}
36 +{"text": "Le site modeste près de le vignoble a changé de mains deux fois au siècle dernier.", "label": "fr"}
37 +{"text": "The restored site near the workshop attracted visitors from the region.", "label": "en"}
38 +{"text": "The crowded site near the vineyard required constant maintenance.", "label": "en"}
39 +{"text": "Le site vaste près de la bibliothèque a été documenté dans l'enquête.", "label": "fr"}
40 +{"text": "The famous site near the library remained open despite the storm.", "label": "en"}
41 +{"text": "Le site vaste près de le verger a changé de mains deux fois au siècle dernier.", "label": "fr"}
42 +{"text": "The crowded site near the observatory changed hands twice last century.", "label": "en"}
43 +{"text": "The famous site near the market attracted visitors from the region.", "label": "en"}
44 +{"text": "The famous site near the archive required constant maintenance.", "label": "en"}
45 +{"text": "The modest site near the orchard was documented in the survey.", "label": "en"}
46 +{"text": "Le site bondé près de l'observatoire se trouvait à la limite de la ville.", "label": "fr"}
47 +{"text": "The restored site near the orchard stood at the edge of town.", "label": "en"}
48 +{"text": "Le site vaste près de les archives exigeait un entretien constant.", "label": "fr"}
49 +{"text": "Le site vaste près de l'atelier se trouvait à la limite de la ville.", "label": "fr"}
50 +{"text": "Le site bondé près de le verger exigeait un entretien constant.", "label": "fr"}
51 +{"text": "The restored site near the library attracted visitors from the region.", "label": "en"}
52 +{"text": "The crowded site near the market stood at the edge of town.", "label": "en"}
53 +{"text": "Le site vaste près de l'atelier a changé de mains deux fois au siècle dernier.", "label": "fr"}
54 +{"text": "Le site abandonné près de le verger est resté ouvert malgré la tempête.", "label": "fr"}
55 +{"text": "The abandoned site near the observatory stood at the edge of town.", "label": "en"}
56 +{"text": "The ancient site near the vineyard remained open despite the storm.", "label": "en"}
57 +{"text": "Le site ancien près de le vignoble a changé de mains deux fois au siècle dernier.", "label": "fr"}
58 +{"text": "Le site abandonné près de l'atelier est resté ouvert malgré la tempête.", "label": "fr"}
59 +{"text": "Le site modeste près de l'observatoire a attiré des visiteurs de la région.", "label": "fr"}
60 +{"text": "The vast site near the market remained open despite the storm.", "label": "en"}
61 +{"text": "Le site vaste près de le glacier se trouvait à la limite de la ville.", "label": "fr"}
62 +{"text": "Le site calme près de le port se trouvait à la limite de la ville.", "label": "fr"}
63 +{"text": "The crowded site near the lighthouse stood at the edge of town.", "label": "en"}
64 +{"text": "The restored site near the glacier attracted visitors from the region.", "label": "en"}
65 +{"text": "The restored site near the workshop stood at the edge of town.", "label": "en"}
66 +{"text": "Le site ancien près de l'observatoire exigeait un entretien constant.", "label": "fr"}
67 +{"text": "The quiet site near the glacier remained open despite the storm.", "label": "en"}
68 +{"text": "Le site ancien près de la bibliothèque se trouvait à la limite de la ville.", "label": "fr"}
69 +{"text": "Le site ancien près de le glacier est resté ouvert malgré la tempête.", "label": "fr"}
70 +{"text": "Le site restauré près de la bibliothèque se trouvait à la limite de la ville.", "label": "fr"}
71 +{"text": "The crowded site near the workshop changed hands twice last century.", "label": "en"}
72 +{"text": "Le site restauré près de l'atelier a changé de mains deux fois au siècle dernier.", "label": "fr"}
73 +{"text": "The modest site near the archive changed hands twice last century.", "label": "en"}
74 +{"text": "The vast site near the lighthouse was documented in the survey.", "label": "en"}
75 +{"text": "Le site célèbre près de le port est resté ouvert malgré la tempête.", "label": "fr"}
76 +{"text": "Le site ancien près de l'atelier a changé de mains deux fois au siècle dernier.", "label": "fr"}
77 +{"text": "The modest site near the orchard attracted visitors from the region.", "label": "en"}
78 +{"text": "Le site ancien près de le verger a été documenté dans l'enquête.", "label": "fr"}
79 +{"text": "The restored site near the vineyard required constant maintenance.", "label": "en"}
80 +{"text": "The famous site near the library stood at the edge of town.", "label": "en"}
81 +{"text": "The modest site near the lighthouse changed hands twice last century.", "label": "en"}
82 +{"text": "Le site modeste près de l'atelier a été documenté dans l'enquête.", "label": "fr"}
83 +{"text": "Le site bondé près de l'atelier a changé de mains deux fois au siècle dernier.", "label": "fr"}
84 +{"text": "Le site vaste près de le vignoble a attiré des visiteurs de la région.", "label": "fr"}
85 +{"text": "Le site bondé près de le port a été documenté dans l'enquête.", "label": "fr"}
86 +{"text": "The ancient site near the library stood at the edge of town.", "label": "en"}
87 +{"text": "Le site bondé près de la bibliothèque exigeait un entretien constant.", "label": "fr"}
88 +{"text": "The vast site near the library remained open despite the storm.", "label": "en"}
89 +{"text": "The vast site near the glacier required constant maintenance.", "label": "en"}
90 +{"text": "The abandoned site near the workshop attracted visitors from the region.", "label": "en"}
91 +{"text": "Le site vaste près de le vignoble se trouvait à la limite de la ville.", "label": "fr"}
92 +{"text": "Le site abandonné près de le phare a attiré des visiteurs de la région.", "label": "fr"}
93 +{"text": "Le site abandonné près de le glacier exigeait un entretien constant.", "label": "fr"}
94 +{"text": "The vast site near the lighthouse stood at the edge of town.", "label": "en"}
95 +{"text": "Le site calme près de le vignoble se trouvait à la limite de la ville.", "label": "fr"}
96 +{"text": "Le site bondé près de le marché a changé de mains deux fois au siècle dernier.", "label": "fr"}
97 +{"text": "The quiet site near the workshop was documented in the survey.", "label": "en"}
98 +{"text": "Le site vaste près de le verger exigeait un entretien constant.", "label": "fr"}
99 +{"text": "Le site modeste près de le glacier exigeait un entretien constant.", "label": "fr"}
100 +{"text": "The vast site near the workshop required constant maintenance.", "label": "en"}
101 +{"text": "Le site célèbre près de le marché se trouvait à la limite de la ville.", "label": "fr"}
102 +{"text": "Le site ancien près de le verger est resté ouvert malgré la tempête.", "label": "fr"}
103 +{"text": "Le site ancien près de l'atelier exigeait un entretien constant.", "label": "fr"}
104 +{"text": "The abandoned site near the lighthouse changed hands twice last century.", "label": "en"}
105 +{"text": "Le site célèbre près de la bibliothèque se trouvait à la limite de la ville.", "label": "fr"}
106 +{"text": "The quiet site near the vineyard changed hands twice last century.", "label": "en"}
107 +{"text": "The restored site near the orchard was documented in the survey.", "label": "en"}
108 +{"text": "The modest site near the vineyard changed hands twice last century.", "label": "en"}
109 +{"text": "The famous site near the glacier required constant maintenance.", "label": "en"}
110 +{"text": "The famous site near the market stood at the edge of town.", "label": "en"}
111 +{"text": "The crowded site near the orchard stood at the edge of town.", "label": "en"}
112 +{"text": "The famous site near the glacier attracted visitors from the region.", "label": "en"}
113 +{"text": "The vast site near the observatory required constant maintenance.", "label": "en"}
114 +{"text": "The ancient site near the market required constant maintenance.", "label": "en"}
115 +{"text": "Le site bondé près de le phare a changé de mains deux fois au siècle dernier.", "label": "fr"}
116 +{"text": "Le site restauré près de l'observatoire se trouvait à la limite de la ville.", "label": "fr"}
117 +{"text": "Le site modeste près de la bibliothèque est resté ouvert malgré la tempête.", "label": "fr"}
118 +{"text": "The quiet site near the workshop changed hands twice last century.", "label": "en"}
119 +{"text": "Le site calme près de le marché exigeait un entretien constant.", "label": "fr"}
120 +{"text": "The modest site near the workshop remained open despite the storm.", "label": "en"}
121 +{"text": "The modest site near the archive stood at the edge of town.", "label": "en"}
122 +{"text": "Le site ancien près de les archives a attiré des visiteurs de la région.", "label": "fr"}
123 +{"text": "The abandoned site near the glacier required constant maintenance.", "label": "en"}
124 +{"text": "Le site célèbre près de les archives exigeait un entretien constant.", "label": "fr"}
125 +{"text": "Le site célèbre près de le port a été documenté dans l'enquête.", "label": "fr"}
126 +{"text": "Le site calme près de l'atelier se trouvait à la limite de la ville.", "label": "fr"}
127 +{"text": "The abandoned site near the library was documented in the survey.", "label": "en"}
128 +{"text": "The famous site near the harbor remained open despite the storm.", "label": "en"}
129 +{"text": "The crowded site near the market changed hands twice last century.", "label": "en"}
130 +{"text": "Le site calme près de le marché se trouvait à la limite de la ville.", "label": "fr"}
131 +{"text": "The restored site near the orchard attracted visitors from the region.", "label": "en"}
132 +{"text": "Le site calme près de l'observatoire exigeait un entretien constant.", "label": "fr"}
133 +{"text": "Le site vaste près de le marché exigeait un entretien constant.", "label": "fr"}
134 +{"text": "Le site calme près de le vignoble a été documenté dans l'enquête.", "label": "fr"}
135 +{"text": "Le site bondé près de le phare exigeait un entretien constant.", "label": "fr"}
136 +{"text": "Le site restauré près de le marché exigeait un entretien constant.", "label": "fr"}
137 +{"text": "Le site abandonné près de l'observatoire exigeait un entretien constant.", "label": "fr"}
138 +{"text": "Le site calme près de le glacier se trouvait à la limite de la ville.", "label": "fr"}
139 +{"text": "The modest site near the lighthouse was documented in the survey.", "label": "en"}
140 +{"text": "Le site vaste près de la bibliothèque se trouvait à la limite de la ville.", "label": "fr"}
141 +{"text": "Le site modeste près de la bibliothèque a été documenté dans l'enquête.", "label": "fr"}
142 +{"text": "The modest site near the glacier was documented in the survey.", "label": "en"}
143 +{"text": "Le site célèbre près de le vignoble a changé de mains deux fois au siècle dernier.", "label": "fr"}
144 +{"text": "Le site bondé près de le port est resté ouvert malgré la tempête.", "label": "fr"}
145 +{"text": "Le site calme près de le verger a été documenté dans l'enquête.", "label": "fr"}
146 +{"text": "Le site célèbre près de le marché a été documenté dans l'enquête.", "label": "fr"}
147 +{"text": "The restored site near the lighthouse attracted visitors from the region.", "label": "en"}
148 +{"text": "Le site modeste près de le verger est resté ouvert malgré la tempête.", "label": "fr"}
149 +{"text": "Le site restauré près de le verger exigeait un entretien constant.", "label": "fr"}
150 +{"text": "The restored site near the lighthouse required constant maintenance.", "label": "en"}
151 +{"text": "Le site vaste près de l'observatoire exigeait un entretien constant.", "label": "fr"}
152 +{"text": "The crowded site near the glacier remained open despite the storm.", "label": "en"}
153 +{"text": "The crowded site near the workshop stood at the edge of town.", "label": "en"}
154 +{"text": "The abandoned site near the market stood at the edge of town.", "label": "en"}
155 +{"text": "The modest site near the workshop was documented in the survey.", "label": "en"}
156 +{"text": "Le site abandonné près de le verger a changé de mains deux fois au siècle dernier.", "label": "fr"}
157 +{"text": "Le site modeste près de le port a été documenté dans l'enquête.", "label": "fr"}
158 +{"text": "The famous site near the harbor required constant maintenance.", "label": "en"}
159 +{"text": "The modest site near the lighthouse remained open despite the storm.", "label": "en"}
160 +{"text": "The famous site near the archive stood at the edge of town.", "label": "en"}
161 +{"text": "Le site abandonné près de l'atelier exigeait un entretien constant.", "label": "fr"}
162 +{"text": "The ancient site near the harbor required constant maintenance.", "label": "en"}
163 +{"text": "Le site vaste près de le phare a changé de mains deux fois au siècle dernier.", "label": "fr"}
164 +{"text": "Le site célèbre près de le phare est resté ouvert malgré la tempête.", "label": "fr"}
165 +{"text": "The vast site near the glacier was documented in the survey.", "label": "en"}
166 +{"text": "The crowded site near the harbor remained open despite the storm.", "label": "en"}
167 +{"text": "The vast site near the library attracted visitors from the region.", "label": "en"}
168 +{"text": "The famous site near the library changed hands twice last century.", "label": "en"}
169 +{"text": "Le site abandonné près de la bibliothèque exigeait un entretien constant.", "label": "fr"}
170 +{"text": "The restored site near the harbor was documented in the survey.", "label": "en"}
171 +{"text": "The vast site near the library changed hands twice last century.", "label": "en"}
172 +{"text": "The ancient site near the vineyard changed hands twice last century.", "label": "en"}
173 +{"text": "The quiet site near the workshop attracted visitors from the region.", "label": "en"}
174 +{"text": "Le site célèbre près de les archives a été documenté dans l'enquête.", "label": "fr"}
175 +{"text": "The famous site near the lighthouse required constant maintenance.", "label": "en"}
176 +{"text": "Le site abandonné près de le marché se trouvait à la limite de la ville.", "label": "fr"}
177 +{"text": "Le site célèbre près de l'observatoire se trouvait à la limite de la ville.", "label": "fr"}
178 +{"text": "The abandoned site near the glacier was documented in the survey.", "label": "en"}
179 +{"text": "The ancient site near the library remained open despite the storm.", "label": "en"}
180 +{"text": "The quiet site near the vineyard was documented in the survey.", "label": "en"}
181 +{"text": "The famous site near the lighthouse remained open despite the storm.", "label": "en"}
182 +{"text": "The crowded site near the harbor attracted visitors from the region.", "label": "en"}
183 +{"text": "The modest site near the vineyard was documented in the survey.", "label": "en"}
184 +{"text": "The famous site near the vineyard was documented in the survey.", "label": "en"}
185 +{"text": "Le site modeste près de le phare a été documenté dans l'enquête.", "label": "fr"}
186 +{"text": "Le site ancien près de le phare a été documenté dans l'enquête.", "label": "fr"}
187 +{"text": "Le site ancien près de les archives exigeait un entretien constant.", "label": "fr"}
188 +{"text": "Le site calme près de le verger exigeait un entretien constant.", "label": "fr"}
189 +{"text": "Le site célèbre près de la bibliothèque a été documenté dans l'enquête.", "label": "fr"}
190 +{"text": "Le site modeste près de l'atelier a changé de mains deux fois au siècle dernier.", "label": "fr"}
191 +{"text": "Le site modeste près de le glacier se trouvait à la limite de la ville.", "label": "fr"}
192 +{"text": "Le site modeste près de le phare a changé de mains deux fois au siècle dernier.", "label": "fr"}
193 +{"text": "Le site calme près de le marché a changé de mains deux fois au siècle dernier.", "label": "fr"}
194 +{"text": "The abandoned site near the observatory remained open despite the storm.", "label": "en"}
195 +{"text": "Le site bondé près de le phare se trouvait à la limite de la ville.", "label": "fr"}
196 +{"text": "Le site célèbre près de le phare a été documenté dans l'enquête.", "label": "fr"}
197 +{"text": "Le site ancien près de le phare se trouvait à la limite de la ville.", "label": "fr"}
198 +{"text": "The modest site near the workshop changed hands twice last century.", "label": "en"}
199 +{"text": "Le site bondé près de le vignoble est resté ouvert malgré la tempête.", "label": "fr"}
200 +{"text": "Le site bondé près de le glacier se trouvait à la limite de la ville.", "label": "fr"}
201 +{"text": "The restored site near the vineyard remained open despite the storm.", "label": "en"}
202 +{"text": "Le site vaste près de l'observatoire a attiré des visiteurs de la région.", "label": "fr"}
203 +{"text": "The restored site near the workshop required constant maintenance.", "label": "en"}
204 +{"text": "The ancient site near the workshop required constant maintenance.", "label": "en"}
205 +{"text": "Le site vaste près de la bibliothèque a attiré des visiteurs de la région.", "label": "fr"}
206 +{"text": "The modest site near the market attracted visitors from the region.", "label": "en"}
207 +{"text": "The restored site near the orchard changed hands twice last century.", "label": "en"}
208 +{"text": "The vast site near the archive changed hands twice last century.", "label": "en"}
209 +{"text": "Le site célèbre près de le marché exigeait un entretien constant.", "label": "fr"}
210 +{"text": "Le site bondé près de l'atelier exigeait un entretien constant.", "label": "fr"}
211 +{"text": "The crowded site near the library remained open despite the storm.", "label": "en"}
212 +{"text": "The restored site near the vineyard stood at the edge of town.", "label": "en"}
213 +{"text": "The abandoned site near the glacier changed hands twice last century.", "label": "en"}
214 +{"text": "The quiet site near the lighthouse remained open despite the storm.", "label": "en"}
215 +{"text": "The crowded site near the workshop was documented in the survey.", "label": "en"}
216 +{"text": "The famous site near the library required constant maintenance.", "label": "en"}
217 +{"text": "Le site restauré près de le marché se trouvait à la limite de la ville.", "label": "fr"}
218 +{"text": "Le site abandonné près de l'observatoire est resté ouvert malgré la tempête.", "label": "fr"}
219 +{"text": "Le site restauré près de la bibliothèque a attiré des visiteurs de la région.", "label": "fr"}
220 +{"text": "Le site abandonné près de le vignoble exigeait un entretien constant.", "label": "fr"}
221 +{"text": "The vast site near the observatory attracted visitors from the region.", "label": "en"}
222 +{"text": "The ancient site near the lighthouse required constant maintenance.", "label": "en"}
223 +{"text": "Le site abandonné près de l'atelier a changé de mains deux fois au siècle dernier.", "label": "fr"}
224 +{"text": "Le site restauré près de le verger a été documenté dans l'enquête.", "label": "fr"}
225 +{"text": "Le site abandonné près de l'observatoire se trouvait à la limite de la ville.", "label": "fr"}
226 +{"text": "The vast site near the workshop was documented in the survey.", "label": "en"}
227 +{"text": "Le site calme près de l'atelier exigeait un entretien constant.", "label": "fr"}
228 +{"text": "The famous site near the orchard was documented in the survey.", "label": "en"}
229 +{"text": "The modest site near the workshop required constant maintenance.", "label": "en"}
230 +{"text": "Le site vaste près de l'observatoire est resté ouvert malgré la tempête.", "label": "fr"}
231 +{"text": "The modest site near the library remained open despite the storm.", "label": "en"}
232 +{"text": "The vast site near the observatory changed hands twice last century.", "label": "en"}
233 +{"text": "Le site ancien près de les archives a été documenté dans l'enquête.", "label": "fr"}
234 +{"text": "Le site vaste près de le glacier a été documenté dans l'enquête.", "label": "fr"}
235 +{"text": "The vast site near the lighthouse remained open despite the storm.", "label": "en"}
236 +{"text": "The ancient site near the market attracted visitors from the region.", "label": "en"}
237 +{"text": "The restored site near the orchard required constant maintenance.", "label": "en"}
238 +{"text": "Le site bondé près de le verger a attiré des visiteurs de la région.", "label": "fr"}
239 +{"text": "The abandoned site near the orchard remained open despite the storm.", "label": "en"}
240 +{"text": "The ancient site near the vineyard required constant maintenance.", "label": "en"}
added benchmarks/promptsets/lang_id_B.jsonl +240 −0
@@ -0,0 +1,240 @@
1 +{"text": "Je me demandais si l'endroit près de le glacier est resté ouvert malgré la tempête ?", "label": "fr"}
2 +{"text": "Could you tell me whether the place near the workshop was documented in the survey?", "label": "en"}
3 +{"text": "Je me demandais si l'endroit près de l'observatoire a changé de mains deux fois au siècle dernier ?", "label": "fr"}
4 +{"text": "Je me demandais si l'endroit près de le port a été documenté dans l'enquête ?", "label": "fr"}
5 +{"text": "I was wondering whether the place near the glacier was documented in the survey?", "label": "en"}
6 +{"text": "Do you happen to know if the place near the glacier remained open despite the storm?", "label": "en"}
7 +{"text": "Je me demandais si l'endroit près de le phare a été documenté dans l'enquête ?", "label": "fr"}
8 +{"text": "Could you tell me whether the place near the workshop required constant maintenance?", "label": "en"}
9 +{"text": "Pourrais-tu me dire si l'endroit près de le phare exigeait un entretien constant ?", "label": "fr"}
10 +{"text": "Je me demandais si l'endroit près de le glacier se trouvait à la limite de la ville ?", "label": "fr"}
11 +{"text": "Do you happen to know if the place near the observatory required constant maintenance?", "label": "en"}
12 +{"text": "Pourrais-tu me dire si l'endroit près de l'observatoire se trouvait à la limite de la ville ?", "label": "fr"}
13 +{"text": "I was wondering whether the place near the observatory stood at the edge of town?", "label": "en"}
14 +{"text": "Pourrais-tu me dire si l'endroit près de la bibliothèque exigeait un entretien constant ?", "label": "fr"}
15 +{"text": "I was wondering whether the place near the archive stood at the edge of town?", "label": "en"}
16 +{"text": "Sais-tu par hasard si l'endroit près de l'observatoire est resté ouvert malgré la tempête ?", "label": "fr"}
17 +{"text": "Je me demandais si l'endroit près de le verger a été documenté dans l'enquête ?", "label": "fr"}
18 +{"text": "I was wondering whether the place near the workshop attracted visitors from the region?", "label": "en"}
19 +{"text": "I was wondering whether the place near the harbor stood at the edge of town?", "label": "en"}
20 +{"text": "I was wondering whether the place near the vineyard was documented in the survey?", "label": "en"}
21 +{"text": "Pourrais-tu me dire si l'endroit près de le verger a changé de mains deux fois au siècle dernier ?", "label": "fr"}
22 +{"text": "Pourrais-tu me dire si l'endroit près de les archives a changé de mains deux fois au siècle dernier ?", "label": "fr"}
23 +{"text": "Je me demandais si l'endroit près de le verger est resté ouvert malgré la tempête ?", "label": "fr"}
24 +{"text": "Do you happen to know if the place near the harbor changed hands twice last century?", "label": "en"}
25 +{"text": "Sais-tu par hasard si l'endroit près de le phare exigeait un entretien constant ?", "label": "fr"}
26 +{"text": "Could you tell me whether the place near the library was documented in the survey?", "label": "en"}
27 +{"text": "Pourrais-tu me dire si l'endroit près de le phare a changé de mains deux fois au siècle dernier ?", "label": "fr"}
28 +{"text": "Sais-tu par hasard si l'endroit près de les archives exigeait un entretien constant ?", "label": "fr"}
29 +{"text": "I was wondering whether the place near the harbor attracted visitors from the region?", "label": "en"}
30 +{"text": "Do you happen to know if the place near the vineyard attracted visitors from the region?", "label": "en"}
31 +{"text": "I was wondering whether the place near the lighthouse was documented in the survey?", "label": "en"}
32 +{"text": "I was wondering whether the place near the lighthouse attracted visitors from the region?", "label": "en"}
33 +{"text": "Could you tell me whether the place near the lighthouse attracted visitors from the region?", "label": "en"}
34 +{"text": "I was wondering whether the place near the harbor remained open despite the storm?", "label": "en"}
35 +{"text": "Sais-tu par hasard si l'endroit près de la bibliothèque est resté ouvert malgré la tempête ?", "label": "fr"}
36 +{"text": "Je me demandais si l'endroit près de la bibliothèque est resté ouvert malgré la tempête ?", "label": "fr"}
37 +{"text": "Could you tell me whether the place near the orchard required constant maintenance?", "label": "en"}
38 +{"text": "Je me demandais si l'endroit près de l'observatoire est resté ouvert malgré la tempête ?", "label": "fr"}
39 +{"text": "Could you tell me whether the place near the lighthouse remained open despite the storm?", "label": "en"}
40 +{"text": "Pourrais-tu me dire si l'endroit près de le marché a changé de mains deux fois au siècle dernier ?", "label": "fr"}
41 +{"text": "Pourrais-tu me dire si l'endroit près de l'observatoire a changé de mains deux fois au siècle dernier ?", "label": "fr"}
42 +{"text": "Pourrais-tu me dire si l'endroit près de les archives a attiré des visiteurs de la région ?", "label": "fr"}
43 +{"text": "Could you tell me whether the place near the observatory attracted visitors from the region?", "label": "en"}
44 +{"text": "Could you tell me whether the place near the harbor stood at the edge of town?", "label": "en"}
45 +{"text": "I was wondering whether the place near the harbor changed hands twice last century?", "label": "en"}
46 +{"text": "Do you happen to know if the place near the orchard attracted visitors from the region?", "label": "en"}
47 +{"text": "Do you happen to know if the place near the orchard changed hands twice last century?", "label": "en"}
48 +{"text": "Sais-tu par hasard si l'endroit près de la bibliothèque a attiré des visiteurs de la région ?", "label": "fr"}
49 +{"text": "Je me demandais si l'endroit près de le vignoble a attiré des visiteurs de la région ?", "label": "fr"}
50 +{"text": "Je me demandais si l'endroit près de le phare est resté ouvert malgré la tempête ?", "label": "fr"}
51 +{"text": "Pourrais-tu me dire si l'endroit près de le verger a été documenté dans l'enquête ?", "label": "fr"}
52 +{"text": "Sais-tu par hasard si l'endroit près de l'observatoire a changé de mains deux fois au siècle dernier ?", "label": "fr"}
53 +{"text": "Pourrais-tu me dire si l'endroit près de le phare est resté ouvert malgré la tempête ?", "label": "fr"}
54 +{"text": "Could you tell me whether the place near the harbor changed hands twice last century?", "label": "en"}
55 +{"text": "I was wondering whether the place near the workshop changed hands twice last century?", "label": "en"}
56 +{"text": "Sais-tu par hasard si l'endroit près de le marché a attiré des visiteurs de la région ?", "label": "fr"}
57 +{"text": "I was wondering whether the place near the vineyard remained open despite the storm?", "label": "en"}
58 +{"text": "Could you tell me whether the place near the glacier required constant maintenance?", "label": "en"}
59 +{"text": "Je me demandais si l'endroit près de l'atelier a changé de mains deux fois au siècle dernier ?", "label": "fr"}
60 +{"text": "Do you happen to know if the place near the harbor required constant maintenance?", "label": "en"}
61 +{"text": "Do you happen to know if the place near the archive changed hands twice last century?", "label": "en"}
62 +{"text": "Je me demandais si l'endroit près de l'observatoire exigeait un entretien constant ?", "label": "fr"}
63 +{"text": "Could you tell me whether the place near the orchard attracted visitors from the region?", "label": "en"}
64 +{"text": "Pourrais-tu me dire si l'endroit près de le verger se trouvait à la limite de la ville ?", "label": "fr"}
65 +{"text": "Sais-tu par hasard si l'endroit près de l'atelier a été documenté dans l'enquête ?", "label": "fr"}
66 +{"text": "Could you tell me whether the place near the glacier was documented in the survey?", "label": "en"}
67 +{"text": "Do you happen to know if the place near the glacier changed hands twice last century?", "label": "en"}
68 +{"text": "Je me demandais si l'endroit près de le marché a changé de mains deux fois au siècle dernier ?", "label": "fr"}
69 +{"text": "Do you happen to know if the place near the harbor stood at the edge of town?", "label": "en"}
70 +{"text": "Do you happen to know if the place near the lighthouse required constant maintenance?", "label": "en"}
71 +{"text": "Je me demandais si l'endroit près de le marché a attiré des visiteurs de la région ?", "label": "fr"}
72 +{"text": "Sais-tu par hasard si l'endroit près de les archives a attiré des visiteurs de la région ?", "label": "fr"}
73 +{"text": "I was wondering whether the place near the library was documented in the survey?", "label": "en"}
74 +{"text": "Could you tell me whether the place near the orchard changed hands twice last century?", "label": "en"}
75 +{"text": "I was wondering whether the place near the orchard changed hands twice last century?", "label": "en"}
76 +{"text": "Could you tell me whether the place near the observatory was documented in the survey?", "label": "en"}
77 +{"text": "Je me demandais si l'endroit près de les archives a attiré des visiteurs de la région ?", "label": "fr"}
78 +{"text": "I was wondering whether the place near the vineyard required constant maintenance?", "label": "en"}
79 +{"text": "Pourrais-tu me dire si l'endroit près de la bibliothèque se trouvait à la limite de la ville ?", "label": "fr"}
80 +{"text": "Je me demandais si l'endroit près de le vignoble est resté ouvert malgré la tempête ?", "label": "fr"}
81 +{"text": "I was wondering whether the place near the workshop was documented in the survey?", "label": "en"}
82 +{"text": "I was wondering whether the place near the market required constant maintenance?", "label": "en"}
83 +{"text": "Je me demandais si l'endroit près de le glacier a changé de mains deux fois au siècle dernier ?", "label": "fr"}
84 +{"text": "Could you tell me whether the place near the lighthouse stood at the edge of town?", "label": "en"}
85 +{"text": "Do you happen to know if the place near the workshop remained open despite the storm?", "label": "en"}
86 +{"text": "Sais-tu par hasard si l'endroit près de le verger exigeait un entretien constant ?", "label": "fr"}
87 +{"text": "Could you tell me whether the place near the market remained open despite the storm?", "label": "en"}
88 +{"text": "Pourrais-tu me dire si l'endroit près de le marché est resté ouvert malgré la tempête ?", "label": "fr"}
89 +{"text": "Pourrais-tu me dire si l'endroit près de le glacier a changé de mains deux fois au siècle dernier ?", "label": "fr"}
90 +{"text": "Je me demandais si l'endroit près de l'atelier exigeait un entretien constant ?", "label": "fr"}
91 +{"text": "Pourrais-tu me dire si l'endroit près de les archives a été documenté dans l'enquête ?", "label": "fr"}
92 +{"text": "Do you happen to know if the place near the observatory was documented in the survey?", "label": "en"}
93 +{"text": "Do you happen to know if the place near the vineyard stood at the edge of town?", "label": "en"}
94 +{"text": "Pourrais-tu me dire si l'endroit près de le port a été documenté dans l'enquête ?", "label": "fr"}
95 +{"text": "Sais-tu par hasard si l'endroit près de le phare a attiré des visiteurs de la région ?", "label": "fr"}
96 +{"text": "Je me demandais si l'endroit près de le phare exigeait un entretien constant ?", "label": "fr"}
97 +{"text": "I was wondering whether the place near the glacier stood at the edge of town?", "label": "en"}
98 +{"text": "Sais-tu par hasard si l'endroit près de la bibliothèque a changé de mains deux fois au siècle dernier ?", "label": "fr"}
99 +{"text": "Do you happen to know if the place near the workshop was documented in the survey?", "label": "en"}
100 +{"text": "I was wondering whether the place near the observatory was documented in the survey?", "label": "en"}
101 +{"text": "Pourrais-tu me dire si l'endroit près de la bibliothèque a changé de mains deux fois au siècle dernier ?", "label": "fr"}
102 +{"text": "Pourrais-tu me dire si l'endroit près de le port se trouvait à la limite de la ville ?", "label": "fr"}
103 +{"text": "Je me demandais si l'endroit près de le phare a changé de mains deux fois au siècle dernier ?", "label": "fr"}
104 +{"text": "Je me demandais si l'endroit près de le vignoble se trouvait à la limite de la ville ?", "label": "fr"}
105 +{"text": "Sais-tu par hasard si l'endroit près de l'atelier a attiré des visiteurs de la région ?", "label": "fr"}
106 +{"text": "Could you tell me whether the place near the workshop remained open despite the storm?", "label": "en"}
107 +{"text": "Je me demandais si l'endroit près de les archives a changé de mains deux fois au siècle dernier ?", "label": "fr"}
108 +{"text": "Pourrais-tu me dire si l'endroit près de l'observatoire a été documenté dans l'enquête ?", "label": "fr"}
109 +{"text": "I was wondering whether the place near the glacier attracted visitors from the region?", "label": "en"}
110 +{"text": "I was wondering whether the place near the orchard remained open despite the storm?", "label": "en"}
111 +{"text": "Je me demandais si l'endroit près de le marché est resté ouvert malgré la tempête ?", "label": "fr"}
112 +{"text": "Sais-tu par hasard si l'endroit près de les archives a été documenté dans l'enquête ?", "label": "fr"}
113 +{"text": "Pourrais-tu me dire si l'endroit près de la bibliothèque a été documenté dans l'enquête ?", "label": "fr"}
114 +{"text": "Je me demandais si l'endroit près de le port est resté ouvert malgré la tempête ?", "label": "fr"}
115 +{"text": "Pourrais-tu me dire si l'endroit près de l'observatoire a attiré des visiteurs de la région ?", "label": "fr"}
116 +{"text": "Pourrais-tu me dire si l'endroit près de le glacier exigeait un entretien constant ?", "label": "fr"}
117 +{"text": "Could you tell me whether the place near the observatory required constant maintenance?", "label": "en"}
118 +{"text": "Could you tell me whether the place near the archive attracted visitors from the region?", "label": "en"}
119 +{"text": "Pourrais-tu me dire si l'endroit près de le vignoble est resté ouvert malgré la tempête ?", "label": "fr"}
120 +{"text": "Do you happen to know if the place near the lighthouse attracted visitors from the region?", "label": "en"}
121 +{"text": "Sais-tu par hasard si l'endroit près de le port est resté ouvert malgré la tempête ?", "label": "fr"}
122 +{"text": "Je me demandais si l'endroit près de le port se trouvait à la limite de la ville ?", "label": "fr"}
123 +{"text": "Je me demandais si l'endroit près de les archives exigeait un entretien constant ?", "label": "fr"}
124 +{"text": "Could you tell me whether the place near the library remained open despite the storm?", "label": "en"}
125 +{"text": "Je me demandais si l'endroit près de la bibliothèque a changé de mains deux fois au siècle dernier ?", "label": "fr"}
126 +{"text": "Sais-tu par hasard si l'endroit près de le port exigeait un entretien constant ?", "label": "fr"}
127 +{"text": "Do you happen to know if the place near the lighthouse remained open despite the storm?", "label": "en"}
128 +{"text": "Je me demandais si l'endroit près de l'atelier se trouvait à la limite de la ville ?", "label": "fr"}
129 +{"text": "I was wondering whether the place near the archive remained open despite the storm?", "label": "en"}
130 +{"text": "Sais-tu par hasard si l'endroit près de le verger est resté ouvert malgré la tempête ?", "label": "fr"}
131 +{"text": "I was wondering whether the place near the glacier remained open despite the storm?", "label": "en"}
132 +{"text": "Je me demandais si l'endroit près de le port exigeait un entretien constant ?", "label": "fr"}
133 +{"text": "I was wondering whether the place near the observatory remained open despite the storm?", "label": "en"}
134 +{"text": "Do you happen to know if the place near the vineyard was documented in the survey?", "label": "en"}
135 +{"text": "Could you tell me whether the place near the library changed hands twice last century?", "label": "en"}
136 +{"text": "Sais-tu par hasard si l'endroit près de le phare a été documenté dans l'enquête ?", "label": "fr"}
137 +{"text": "Could you tell me whether the place near the glacier attracted visitors from the region?", "label": "en"}
138 +{"text": "I was wondering whether the place near the lighthouse required constant maintenance?", "label": "en"}
139 +{"text": "Sais-tu par hasard si l'endroit près de le marché exigeait un entretien constant ?", "label": "fr"}
140 +{"text": "Pourrais-tu me dire si l'endroit près de le vignoble se trouvait à la limite de la ville ?", "label": "fr"}
141 +{"text": "I was wondering whether the place near the market stood at the edge of town?", "label": "en"}
142 +{"text": "Sais-tu par hasard si l'endroit près de le vignoble a attiré des visiteurs de la région ?", "label": "fr"}
143 +{"text": "Do you happen to know if the place near the library changed hands twice last century?", "label": "en"}
144 +{"text": "Sais-tu par hasard si l'endroit près de le phare a changé de mains deux fois au siècle dernier ?", "label": "fr"}
145 +{"text": "Sais-tu par hasard si l'endroit près de l'observatoire a attiré des visiteurs de la région ?", "label": "fr"}
146 +{"text": "I was wondering whether the place near the market attracted visitors from the region?", "label": "en"}
147 +{"text": "Je me demandais si l'endroit près de le glacier a attiré des visiteurs de la région ?", "label": "fr"}
148 +{"text": "Pourrais-tu me dire si l'endroit près de le glacier a été documenté dans l'enquête ?", "label": "fr"}
149 +{"text": "Je me demandais si l'endroit près de le verger a attiré des visiteurs de la région ?", "label": "fr"}
150 +{"text": "I was wondering whether the place near the harbor required constant maintenance?", "label": "en"}
151 +{"text": "Could you tell me whether the place near the market was documented in the survey?", "label": "en"}
152 +{"text": "Could you tell me whether the place near the market changed hands twice last century?", "label": "en"}
153 +{"text": "Pourrais-tu me dire si l'endroit près de l'observatoire exigeait un entretien constant ?", "label": "fr"}
154 +{"text": "Je me demandais si l'endroit près de les archives a été documenté dans l'enquête ?", "label": "fr"}
155 +{"text": "Do you happen to know if the place near the workshop stood at the edge of town?", "label": "en"}
156 +{"text": "Could you tell me whether the place near the orchard was documented in the survey?", "label": "en"}
157 +{"text": "Could you tell me whether the place near the lighthouse changed hands twice last century?", "label": "en"}
158 +{"text": "I was wondering whether the place near the orchard attracted visitors from the region?", "label": "en"}
159 +{"text": "I was wondering whether the place near the workshop remained open despite the storm?", "label": "en"}
160 +{"text": "Could you tell me whether the place near the lighthouse required constant maintenance?", "label": "en"}
161 +{"text": "Sais-tu par hasard si l'endroit près de le marché est resté ouvert malgré la tempête ?", "label": "fr"}
162 +{"text": "Could you tell me whether the place near the archive required constant maintenance?", "label": "en"}
163 +{"text": "Pourrais-tu me dire si l'endroit près de le glacier se trouvait à la limite de la ville ?", "label": "fr"}
164 +{"text": "Do you happen to know if the place near the library stood at the edge of town?", "label": "en"}
165 +{"text": "Do you happen to know if the place near the orchard was documented in the survey?", "label": "en"}
166 +{"text": "I was wondering whether the place near the vineyard changed hands twice last century?", "label": "en"}
167 +{"text": "Do you happen to know if the place near the lighthouse stood at the edge of town?", "label": "en"}
168 +{"text": "Pourrais-tu me dire si l'endroit près de les archives se trouvait à la limite de la ville ?", "label": "fr"}
169 +{"text": "Sais-tu par hasard si l'endroit près de le glacier a changé de mains deux fois au siècle dernier ?", "label": "fr"}
170 +{"text": "Sais-tu par hasard si l'endroit près de le vignoble est resté ouvert malgré la tempête ?", "label": "fr"}
171 +{"text": "Do you happen to know if the place near the archive remained open despite the storm?", "label": "en"}
172 +{"text": "Je me demandais si l'endroit près de le marché exigeait un entretien constant ?", "label": "fr"}
173 +{"text": "Do you happen to know if the place near the orchard required constant maintenance?", "label": "en"}
174 +{"text": "Je me demandais si l'endroit près de le glacier exigeait un entretien constant ?", "label": "fr"}
175 +{"text": "Sais-tu par hasard si l'endroit près de le vignoble se trouvait à la limite de la ville ?", "label": "fr"}
176 +{"text": "Could you tell me whether the place near the observatory changed hands twice last century?", "label": "en"}
177 +{"text": "Pourrais-tu me dire si l'endroit près de l'atelier se trouvait à la limite de la ville ?", "label": "fr"}
178 +{"text": "Could you tell me whether the place near the orchard remained open despite the storm?", "label": "en"}
179 +{"text": "Sais-tu par hasard si l'endroit près de la bibliothèque a été documenté dans l'enquête ?", "label": "fr"}
180 +{"text": "I was wondering whether the place near the orchard was documented in the survey?", "label": "en"}
181 +{"text": "Sais-tu par hasard si l'endroit près de le verger se trouvait à la limite de la ville ?", "label": "fr"}
182 +{"text": "Pourrais-tu me dire si l'endroit près de le marché exigeait un entretien constant ?", "label": "fr"}
183 +{"text": "I was wondering whether the place near the lighthouse stood at the edge of town?", "label": "en"}
184 +{"text": "Sais-tu par hasard si l'endroit près de l'observatoire se trouvait à la limite de la ville ?", "label": "fr"}
185 +{"text": "Do you happen to know if the place near the library attracted visitors from the region?", "label": "en"}
186 +{"text": "Je me demandais si l'endroit près de les archives se trouvait à la limite de la ville ?", "label": "fr"}
187 +{"text": "Could you tell me whether the place near the observatory stood at the edge of town?", "label": "en"}
188 +{"text": "Je me demandais si l'endroit près de la bibliothèque a attiré des visiteurs de la région ?", "label": "fr"}
189 +{"text": "I was wondering whether the place near the market remained open despite the storm?", "label": "en"}
190 +{"text": "Do you happen to know if the place near the glacier attracted visitors from the region?", "label": "en"}
191 +{"text": "Sais-tu par hasard si l'endroit près de le glacier a été documenté dans l'enquête ?", "label": "fr"}
192 +{"text": "Could you tell me whether the place near the harbor required constant maintenance?", "label": "en"}
193 +{"text": "Could you tell me whether the place near the market stood at the edge of town?", "label": "en"}
194 +{"text": "Sais-tu par hasard si l'endroit près de l'observatoire a été documenté dans l'enquête ?", "label": "fr"}
195 +{"text": "Je me demandais si l'endroit près de le port a attiré des visiteurs de la région ?", "label": "fr"}
196 +{"text": "Sais-tu par hasard si l'endroit près de le glacier se trouvait à la limite de la ville ?", "label": "fr"}
197 +{"text": "Do you happen to know if the place near the archive attracted visitors from the region?", "label": "en"}
198 +{"text": "Could you tell me whether the place near the archive was documented in the survey?", "label": "en"}
199 +{"text": "Do you happen to know if the place near the glacier stood at the edge of town?", "label": "en"}
200 +{"text": "I was wondering whether the place near the vineyard attracted visitors from the region?", "label": "en"}
201 +{"text": "Sais-tu par hasard si l'endroit près de la bibliothèque exigeait un entretien constant ?", "label": "fr"}
202 +{"text": "Could you tell me whether the place near the vineyard changed hands twice last century?", "label": "en"}
203 +{"text": "Pourrais-tu me dire si l'endroit près de le phare a été documenté dans l'enquête ?", "label": "fr"}
204 +{"text": "Sais-tu par hasard si l'endroit près de le glacier est resté ouvert malgré la tempête ?", "label": "fr"}
205 +{"text": "Sais-tu par hasard si l'endroit près de le phare se trouvait à la limite de la ville ?", "label": "fr"}
206 +{"text": "Do you happen to know if the place near the market remained open despite the storm?", "label": "en"}
207 +{"text": "Je me demandais si l'endroit près de le phare a attiré des visiteurs de la région ?", "label": "fr"}
208 +{"text": "Could you tell me whether the place near the harbor was documented in the survey?", "label": "en"}
209 +{"text": "Do you happen to know if the place near the observatory attracted visitors from the region?", "label": "en"}
210 +{"text": "Do you happen to know if the place near the workshop changed hands twice last century?", "label": "en"}
211 +{"text": "Could you tell me whether the place near the archive remained open despite the storm?", "label": "en"}
212 +{"text": "Je me demandais si l'endroit près de le marché a été documenté dans l'enquête ?", "label": "fr"}
213 +{"text": "Do you happen to know if the place near the workshop attracted visitors from the region?", "label": "en"}
214 +{"text": "I was wondering whether the place near the vineyard stood at the edge of town?", "label": "en"}
215 +{"text": "I was wondering whether the place near the observatory required constant maintenance?", "label": "en"}
216 +{"text": "Could you tell me whether the place near the workshop stood at the edge of town?", "label": "en"}
217 +{"text": "Pourrais-tu me dire si l'endroit près de le marché se trouvait à la limite de la ville ?", "label": "fr"}
218 +{"text": "Could you tell me whether the place near the glacier remained open despite the storm?", "label": "en"}
219 +{"text": "Pourrais-tu me dire si l'endroit près de le marché a été documenté dans l'enquête ?", "label": "fr"}
220 +{"text": "Do you happen to know if the place near the market stood at the edge of town?", "label": "en"}
221 +{"text": "Je me demandais si l'endroit près de le glacier a été documenté dans l'enquête ?", "label": "fr"}
222 +{"text": "I was wondering whether the place near the observatory changed hands twice last century?", "label": "en"}
223 +{"text": "I was wondering whether the place near the harbor was documented in the survey?", "label": "en"}
224 +{"text": "Do you happen to know if the place near the vineyard changed hands twice last century?", "label": "en"}
225 +{"text": "Pourrais-tu me dire si l'endroit près de l'atelier est resté ouvert malgré la tempête ?", "label": "fr"}
226 +{"text": "Sais-tu par hasard si l'endroit près de le vignoble exigeait un entretien constant ?", "label": "fr"}
227 +{"text": "I was wondering whether the place near the library changed hands twice last century?", "label": "en"}
228 +{"text": "Pourrais-tu me dire si l'endroit près de le port a attiré des visiteurs de la région ?", "label": "fr"}
229 +{"text": "I was wondering whether the place near the library attracted visitors from the region?", "label": "en"}
230 +{"text": "Je me demandais si l'endroit près de l'atelier est resté ouvert malgré la tempête ?", "label": "fr"}
231 +{"text": "Sais-tu par hasard si l'endroit près de l'atelier a changé de mains deux fois au siècle dernier ?", "label": "fr"}
232 +{"text": "Je me demandais si l'endroit près de les archives est resté ouvert malgré la tempête ?", "label": "fr"}
233 +{"text": "Pourrais-tu me dire si l'endroit près de le marché a attiré des visiteurs de la région ?", "label": "fr"}
234 +{"text": "Do you happen to know if the place near the lighthouse was documented in the survey?", "label": "en"}
235 +{"text": "I was wondering whether the place near the library stood at the edge of town?", "label": "en"}
236 +{"text": "Sais-tu par hasard si l'endroit près de le marché a été documenté dans l'enquête ?", "label": "fr"}
237 +{"text": "Sais-tu par hasard si l'endroit près de le port a attiré des visiteurs de la région ?", "label": "fr"}
238 +{"text": "Pourrais-tu me dire si l'endroit près de l'observatoire est resté ouvert malgré la tempête ?", "label": "fr"}
239 +{"text": "Could you tell me whether the place near the glacier changed hands twice last century?", "label": "en"}
240 +{"text": "I was wondering whether the place near the library required constant maintenance?", "label": "en"}
added benchmarks/promptsets/make_promptsets.py +154 −0
@@ -0,0 +1,154 @@
1 +#!/usr/bin/env python3
2 +# =============================================================================
3 +# Project : modelmap
4 +# File : benchmarks/promptsets/make_promptsets.py
5 +# Purpose : Generate versioned, checksummed probing corpora (v1, template)
6 +# Author : Simon-Pierre Boucher
7 +# Contact : contact@spboucher.ai
8 +# Website : https://modelmap.io
9 +# Created : 2026-08-12
10 +# Modified : 2026-08-12
11 +# Platform : macOS / Apple Silicon (arm64)
12 +# License : All rights reserved (research code)
13 +# =============================================================================
14 +"""Probing corpora v1 (charter §0.3: versioned artifacts with checksums).
15 +
16 +Three binary properties, TWO deliberately different template families per
17 +property (set A / set B) so that dataset sensitivity is measurable:
18 + lang_id : French vs English sentences
19 + code_prose : code snippets vs English prose
20 + arith : arithmetic-context vs non-arithmetic sentences
21 +
22 +Deterministic given SEED. v1 limitation, declared: template-generated text,
23 +not natural corpora — lexical diversity is bounded; natural-corpus v2 is a
24 +registered follow-up. Output: <name>.jsonl + manifest.json with sha256.
25 +"""
26 +
27 +from __future__ import annotations
28 +
29 +import hashlib
30 +import itertools
31 +import json
32 +import random
33 +from pathlib import Path
34 +
35 +SEED = 12345
36 +N_PER_CLASS = 120
37 +VERSION = "v1"
38 +OUT = Path(__file__).resolve().parent
39 +
40 +TOPICS_EN = ["the harbor", "the library", "the orchard", "the workshop", "the observatory",
41 + "the market", "the glacier", "the archive", "the vineyard", "the lighthouse"]
42 +TOPICS_FR = ["le port", "la bibliothèque", "le verger", "l'atelier", "l'observatoire",
43 + "le marché", "le glacier", "les archives", "le vignoble", "le phare"]
44 +ADJ_EN = ["quiet", "ancient", "crowded", "restored", "abandoned", "famous", "modest", "vast"]
45 +ADJ_FR = ["calme", "ancien", "bondé", "restauré", "abandonné", "célèbre", "modeste", "vaste"]
46 +VERB_EN = ["remained open despite the storm", "attracted visitors from the region",
47 + "was documented in the survey", "changed hands twice last century",
48 + "stood at the edge of town", "required constant maintenance"]
49 +VERB_FR = ["est resté ouvert malgré la tempête", "a attiré des visiteurs de la région",
50 + "a été documenté dans l'enquête", "a changé de mains deux fois au siècle dernier",
51 + "se trouvait à la limite de la ville", "exigeait un entretien constant"]
52 +Q_EN = ["Could you tell me whether", "Do you happen to know if", "I was wondering whether"]
53 +Q_FR = ["Pourrais-tu me dire si", "Sais-tu par hasard si", "Je me demandais si"]
54 +PY_FUNCS = ["total", "scale", "merge", "clip", "score", "rank", "fold", "trim"]
55 +JS_VARS = ["items", "nodes", "queue", "cache", "rows", "edges", "bins", "keys"]
56 +PROSE_SUBJ = ["The committee", "A local historian", "The lead engineer", "Her assistant",
57 + "The night watchman", "An early visitor", "The town council", "The apprentice"]
58 +PROSE_TAIL = ["reviewed the plans before the meeting.", "kept detailed notes for years.",
59 + "questioned the original estimate.", "preferred the older method.",
60 + "described the process in a letter.", "returned before the first frost."]
61 +
62 +
63 +def gen(rng: random.Random):
64 + sets: dict[str, list[dict]] = {}
65 +
66 + # -------- lang_id: set A = descriptive statements, set B = questions
67 + a, b = [], []
68 + for t, adj, v in itertools.product(TOPICS_EN, ADJ_EN, VERB_EN):
69 + a.append({"text": f"The {adj} site near {t} {v}.", "label": "en"})
70 + for t, adj, v in itertools.product(TOPICS_FR, ADJ_FR, VERB_FR):
71 + a.append({"text": f"Le site {adj} près de {t} {v}.", "label": "fr"})
72 + for q, t, v in itertools.product(Q_EN, TOPICS_EN, VERB_EN):
73 + b.append({"text": f"{q} the place near {t} {v}?", "label": "en"})
74 + for q, t, v in itertools.product(Q_FR, TOPICS_FR, VERB_FR):
75 + b.append({"text": f"{q} l'endroit près de {t} {v} ?", "label": "fr"})
76 + sets["lang_id_A"], sets["lang_id_B"] = a, b
77 +
78 + # -------- code_prose: set A = python vs prose, set B = js vs prose
79 + a, b = [], []
80 + for f, op, k in itertools.product(PY_FUNCS, ["+", "-", "*"], [1, 2, 3, 5, 7]):
81 + a.append({"text": f"def {f}(xs):\n return [x {op} {k} for x in xs if x > {k + 1}]",
82 + "label": "code"})
83 + for s, t, adv in itertools.product(PROSE_SUBJ, PROSE_TAIL,
84 + ["eventually", "reluctantly", "quietly"]):
85 + a.append({"text": f"{s} {adv} {t.lower()}", "label": "prose"})
86 + for v, meth, k in itertools.product(JS_VARS, ["filter", "map", "find", "some"], [0, 1, 4, 9]):
87 + b.append({"text": f"const out = {v}.{meth}(x => x.size > {k}).length;",
88 + "label": "code"})
89 + for s, t, adj in itertools.product(PROSE_SUBJ, PROSE_TAIL[:3], ADJ_EN[:5]):
90 + b.append({"text": f"{s}, though {adj}, {t.lower()}", "label": "prose"})
91 + sets["code_prose_A"], sets["code_prose_B"] = a, b
92 +
93 + # -------- arith: set A = imperative computations, set B = embedded quantities
94 + a, b = [], []
95 + for _ in range(4 * N_PER_CLASS):
96 + x, y = rng.randint(11, 97), rng.randint(11, 97)
97 + a.append({"text": f"Calculate {x} + {y} and report the result.", "label": "arith"})
98 + s, t = rng.choice(PROSE_SUBJ), rng.choice(PROSE_TAIL)
99 + adv = rng.choice(["eventually", "reluctantly", "quietly", "finally", "later"])
100 + a.append({"text": f"{s} {adv} {t.lower()}", "label": "plain"})
101 + p, q = rng.randint(12, 89), rng.randint(12, 89)
102 + b.append({"text": f"If the crate holds {p} jars and {q} more arrive, how many jars are there in total?",
103 + "label": "arith"})
104 + topic, verb, adj = rng.choice(TOPICS_EN), rng.choice(VERB_EN), rng.choice(ADJ_EN)
105 + b.append({"text": f"According to the {adj} report, {topic} {verb}.", "label": "plain"})
106 + sets["arith_A"], sets["arith_B"] = a, b
107 +
108 + # balance + subsample every set to N_PER_CLASS per label, deterministic
109 + final = {}
110 + for name, items in sets.items():
111 + by = {}
112 + for it in items:
113 + by.setdefault(it["label"], []).append(it)
114 + chosen = []
115 + for label, pool in sorted(by.items()):
116 + rng.shuffle(pool)
117 + # dedupe by text before sampling
118 + seen, uniq = set(), []
119 + for it in pool:
120 + if it["text"] not in seen:
121 + seen.add(it["text"])
122 + uniq.append(it)
123 + if len(uniq) < N_PER_CLASS:
124 + raise SystemExit(f"{name}/{label}: only {len(uniq)} unique items")
125 + chosen += uniq[:N_PER_CLASS]
126 + rng.shuffle(chosen)
127 + final[name] = chosen
128 + return final
129 +
130 +
131 +def main() -> int:
132 + rng = random.Random(SEED)
133 + sets = gen(rng)
134 + manifest = {"version": VERSION, "seed": SEED, "n_per_class": N_PER_CLASS,
135 + "author": "Simon-Pierre Boucher", "contact": "contact@spboucher.ai",
136 + "website": "https://modelmap.io",
137 + "limitation": "template-generated v1; natural-corpus v2 registered",
138 + "files": {}}
139 + for name, items in sorted(sets.items()):
140 + path = OUT / f"{name}.jsonl"
141 + payload = "\n".join(json.dumps(it, ensure_ascii=False) for it in items) + "\n"
142 + path.write_text(payload)
143 + manifest["files"][path.name] = {
144 + "sha256": hashlib.sha256(payload.encode()).hexdigest(),
145 + "n": len(items),
146 + }
147 + print(f"{path.name:22s} n={len(items)} sha256={manifest['files'][path.name]['sha256'][:12]}…")
148 + (OUT / "manifest.json").write_text(json.dumps(manifest, indent=2, ensure_ascii=False) + "\n")
149 + print("manifest.json written")
150 + return 0
151 +
152 +
153 +if __name__ == "__main__":
154 + raise SystemExit(main())
added benchmarks/promptsets/manifest.json +35 −0
@@ -0,0 +1,35 @@
1 +{
2 + "version": "v1",
3 + "seed": 12345,
4 + "n_per_class": 120,
5 + "author": "Simon-Pierre Boucher",
6 + "contact": "contact@spboucher.ai",
7 + "website": "https://modelmap.io",
8 + "limitation": "template-generated v1; natural-corpus v2 registered",
9 + "files": {
10 + "arith_A.jsonl": {
11 + "sha256": "2fd80600d8a1b4ad89fed0cdeba2d5d6c33e795f7000553c08addc361305e074",
12 + "n": 240
13 + },
14 + "arith_B.jsonl": {
15 + "sha256": "04bb0eceb4b9262e090cd45b11d77487e438d14859a809d98377ac1807e3a502",
16 + "n": 240
17 + },
18 + "code_prose_A.jsonl": {
19 + "sha256": "dfbfb13dade0fd0be1202b0a745cfd03dfa796f2de4e22c6eca116abab71415c",
20 + "n": 240
21 + },
22 + "code_prose_B.jsonl": {
23 + "sha256": "e9c3f78d8754b0ad5c8921ed361fd4a4a8e824a3c116d2cb3f372417793fd4d7",
24 + "n": 240
25 + },
26 + "lang_id_A.jsonl": {
27 + "sha256": "43bd7ed12d2d9f95d643556a1a05a75623f32201b6f47fdc7d580b5f0cdce30a",
28 + "n": 240
29 + },
30 + "lang_id_B.jsonl": {
31 + "sha256": "833435dae9a61ae04cf354484693c821f6a88dfceaf19865b9da2947c9d71b52",
32 + "n": 240
33 + }
34 + }
35 +}
added experiments/micro/expA_probe_reliability/analysis.md +73 −0
@@ -0,0 +1,73 @@
1 +---
2 +project: modelmap
3 +document: expA_probe_reliability — analysis (run #1)
4 +author: Simon-Pierre Boucher
5 +contact: contact@spboucher.ai
6 +website: https://modelmap.io
7 +created: 2026-08-12
8 +modified: 2026-08-12
9 +status: reviewed
10 +---
11 +
12 +# Analysis — expA run #1: the harness failed its own validity gate (and that is the result)
13 +
14 +Run: `results/expA_probe_reliability/20260812T062605Z/results.json` (48 s wall).
15 +Model: mlx-community/Qwen3-0.6B-4bit (28 layers) + random-init architecture twin.
16 +Grid: 3 properties × 2 disjoint promptsets × 28 layers × 5 seeds, shuffled-label
17 +control inside every probe, BH-FDR q=0.05, top-5 replication. Hypothesis and
18 +validity criterion registered before the run.
19 +
20 +```text
21 +Hypothesis : probe maps are seed-stable but dataset-sensitive
22 + (dataset variance > seed variance on ≥1/3 of layers).
23 +Falsification criterion : seeds dominate; AND harness validity gate: twin
24 + (random-init) selectivity must stay < 0.05.
25 +Result : PRIMARY HYPOTHESIS UNTESTABLE — ceiling effect:
26 + task accuracy 1.000 at every layer, every property,
27 + both sets; seed SD = 0.0000; dataset shift = 0.0000.
28 + VALIDITY GATE FAILED, decisively: the twin reaches
29 + accuracy 1.00 at all 28 layers for all properties
30 + (twin max selectivity 0.56–0.88 vs the registered
31 + 0.05 bar); mean real-minus-twin selectivity is
32 + within ±0.06 — i.e. ZERO measurable trained-model
33 + signal in the map. 28/28 layers "FDR-significant"
34 + in the twin too: the statistics are fine, the
35 + measurand is wrong.
36 +Interpretation : Level 1 FOR THE NEGATIVE CLAIM (controlled,
37 + replicated across 5 seeds × 2 sets × 3 properties):
38 + on lexically separable classes, per-layer linear
39 + probes on mean-pooled representations measure the
40 + tokenizer + architecture prior, not learned
41 + computation. This is the probing-pitfalls
42 + literature (notes §4.1/§4.8 — Hewitt & Liang,
43 + Bolukbasi, Dead Salmons) reproduced in our own
44 + hands on our own harness, caught by a
45 + pre-registered gate rather than by a reviewer.
46 + Published as the atlas's first entry
47 + (atlas/qwen3-0.6b-4bit/probes/v1,
48 + negative_result=true) — maps that fail their nulls
49 + are first-class citizens here.
50 +Next experiment : expA run #2 with (a) promptsets v2 that remove
51 + lexical separability (shared vocabulary across
52 + classes; structure-borne properties), (b) probe
53 + maps published only as REAL−TWIN differentials,
54 + (c) last-token representations at matched
55 + positions alongside mean-pooling, (d) a
56 + class-balanced token-overlap statistic reported
57 + per promptset as a difficulty certificate.
58 +```
59 +
60 +## Notes
61 +
62 +- The ceiling itself was foreseeable in hindsight: template-generated classes
63 + differ lexically (FR vs EN vocabulary; `def`/`const` tokens; digit tokens),
64 + and random features over distinct token distributions are linearly
65 + separable. v1 promptsets keep their declared role — they are now the
66 + *positive control* corpus (any harness that fails to reach ceiling on them
67 + is broken) rather than a measurement corpus.
68 +- The twin null earns permanent-fixture status: it is cheap (one extra
69 + capture pass) and it is the only control in the battery that caught this.
70 + Shuffled-label controls passed everywhere (selectivity 0.3–0.7) and would
71 + have let the illusion through.
72 +- Wall-clock: full grid in 48 s on M5 Max — noise-floor science at this scale
73 + is essentially free; the expensive part was thinking, not compute.
added experiments/micro/expA_probe_reliability/implementation/benchmark.py +184 −0
@@ -0,0 +1,184 @@
1 +#!/usr/bin/env python3
2 +# =============================================================================
3 +# Project : modelmap
4 +# File : experiments/micro/expA_probe_reliability/implementation/benchmark.py
5 +# Purpose : expA run #1 — probe noise floor on a real 4-bit model + nulls
6 +# Author : Simon-Pierre Boucher
7 +# Contact : contact@spboucher.ai
8 +# Website : https://modelmap.io
9 +# Created : 2026-08-12
10 +# Modified : 2026-08-12
11 +# Platform : macOS / Apple Silicon (arm64) — MLX / Metal
12 +# License : All rights reserved (research code)
13 +# =============================================================================
14 +"""expA run #1 (hypothesis registered in hypothesis.md before this run).
15 +
16 +Per-layer linear probes with the full controls doctrine on Qwen3-0.6B-4bit:
17 +3 properties x 2 disjoint promptsets x 28 layers x 5 seeds, shuffled-label
18 +controls built into every probe, random-init architecture twin as the
19 +structure-from-architecture null, bootstrap p-values + BH-FDR across layer
20 +scans, replication rate on top-k layer sets.
21 +"""
22 +
23 +from __future__ import annotations
24 +
25 +import json
26 +import subprocess
27 +import sys
28 +import time
29 +from pathlib import Path
30 +
31 +import numpy as np
32 +
33 +ROOT = Path(__file__).resolve().parents[4]
34 +sys.path.insert(0, str(ROOT / "src"))
35 +sys.path.insert(0, str(ROOT / "benchmarks"))
36 +from hardware_manifest import manifest
37 +
38 +from modelmap.capture.mlx_capture import capture_mean_pooled, install_taps, random_init_twin
39 +from modelmap.probes.linear import probe_with_control
40 +from modelmap.stats.replication import bh_fdr, bootstrap_ci, replication_rate
41 +
42 +MODEL = "mlx-community/Qwen3-0.6B-4bit"
43 +PROPERTIES = ("lang_id", "code_prose", "arith")
44 +SETS = ("A", "B")
45 +SEEDS = (0, 1, 2, 3, 4)
46 +TOP_K = 5
47 +FDR_Q = 0.05
48 +PROMPTS = ROOT / "benchmarks" / "promptsets"
49 +
50 +
51 +def load_set(name: str) -> tuple[list[str], np.ndarray]:
52 + items = [json.loads(l) for l in (PROMPTS / f"{name}.jsonl").read_text().splitlines()]
53 + return [it["text"] for it in items], np.array([it["label"] for it in items])
54 +
55 +
56 +def probe_grid(reps: np.ndarray, labels: np.ndarray) -> list[dict]:
57 + """All (layer, seed) probe cells for one representation tensor."""
58 + out = []
59 + n_layers = reps.shape[1]
60 + for layer in range(n_layers):
61 + for seed in SEEDS:
62 + r = probe_with_control(reps[:, layer, :], labels, seed=seed)
63 + out.append({"layer": layer, "seed": seed,
64 + "task_acc": r.task_accuracy,
65 + "control_acc": r.control_accuracy,
66 + "selectivity": r.selectivity})
67 + return out
68 +
69 +
70 +def summarize(cells: list[dict], n_layers: int) -> dict:
71 + """Per-layer aggregation: means, seed SD, bootstrap CI + p on selectivity."""
72 + rng = np.random.default_rng(0)
73 + layers = []
74 + pvals = []
75 + for layer in range(n_layers):
76 + sel = np.array([c["selectivity"] for c in cells if c["layer"] == layer])
77 + acc = np.array([c["task_acc"] for c in cells if c["layer"] == layer])
78 + point, lo, hi = bootstrap_ci(sel, rng=rng)
79 + boots = rng.choice(sel, size=(10_000, sel.size)).mean(axis=1)
80 + p = float(max((boots <= 0).mean(), 1 / 10_000))
81 + pvals.append(p)
82 + layers.append({"layer": layer,
83 + "task_acc_mean": float(acc.mean()),
84 + "task_acc_seed_sd": float(acc.std(ddof=1)),
85 + "selectivity_mean": point,
86 + "selectivity_ci": [lo, hi],
87 + "p_boot": p})
88 + disc = bh_fdr(np.array(pvals), q=FDR_Q)
89 + for row, d in zip(layers, disc):
90 + row["fdr_significant"] = bool(d)
91 + top_sets = []
92 + for seed in SEEDS:
93 + acc_by_layer = [(c["layer"], c["task_acc"]) for c in cells if c["seed"] == seed]
94 + top = {l for l, _ in sorted(acc_by_layer, key=lambda t: -t[1])[:TOP_K]}
95 + top_sets.append(top)
96 + rep_point, rep_lo, rep_hi = replication_rate(top_sets)
97 + return {"layers": layers,
98 + "replication_rate_topk": {"k": TOP_K, "point": rep_point, "ci": [rep_lo, rep_hi]},
99 + "n_fdr_significant": int(disc.sum())}
100 +
101 +
102 +def main() -> int:
103 + import mlx.core as mx
104 + from mlx_lm import load
105 + from mlx_lm.utils import hf_repo_to_path
106 +
107 + t_start = time.time()
108 + mx.random.seed(0)
109 + model, tokenizer = load(MODEL)
110 + taps = install_taps(model)
111 + model_path = hf_repo_to_path(MODEL)
112 +
113 + twin = random_init_twin(model_path)
114 + twin_taps = install_taps(twin)
115 +
116 + results = {"real": {}, "twin": {}}
117 + grids = {"real": {}, "twin": {}}
118 +
119 + for prop in PROPERTIES:
120 + for s in SETS:
121 + name = f"{prop}_{s}"
122 + texts, labels = load_set(name)
123 + toks = [tokenizer.encode(t) for t in texts]
124 + print(f"capture real {name} ({len(texts)} prompts)…", flush=True)
125 + reps = capture_mean_pooled(model, taps, toks)
126 + grids["real"][name] = (probe_grid(reps, labels), reps.shape[1])
127 + if s == "A": # architecture null on the A sets
128 + print(f"capture twin {name}…", flush=True)
129 + reps_t = capture_mean_pooled(twin, twin_taps, toks)
130 + grids["twin"][name] = (probe_grid(reps_t, labels), reps_t.shape[1])
131 +
132 + for kind in ("real", "twin"):
133 + for name, (cells, n_layers) in grids[kind].items():
134 + results[kind][name] = summarize(cells, n_layers)
135 + results[kind][name]["cells"] = cells
136 +
137 + # -------- headline numbers
138 + summary = {}
139 + for prop in PROPERTIES:
140 + a = results["real"][f"{prop}_A"]["layers"]
141 + b = results["real"][f"{prop}_B"]["layers"]
142 + seed_sd = float(np.mean([r["task_acc_seed_sd"] for r in a + b]))
143 + shifts = [abs(ra["task_acc_mean"] - rb["task_acc_mean"]) for ra, rb in zip(a, b)]
144 + n_shift = int(sum(s > max(seed_sd, 1e-9) for s in shifts))
145 + twin_sel = float(max(r["selectivity_mean"] for r in results["twin"][f"{prop}_A"]["layers"]))
146 + summary[prop] = {
147 + "mean_seed_sd": seed_sd,
148 + "mean_dataset_shift": float(np.mean(shifts)),
149 + "layers_shift_gt_seed_sd": n_shift,
150 + "n_layers": len(a),
151 + "max_task_acc_A": float(max(r["task_acc_mean"] for r in a)),
152 + "max_task_acc_B": float(max(r["task_acc_mean"] for r in b)),
153 + "twin_max_selectivity": twin_sel,
154 + "replication_topk_A": results["real"][f"{prop}_A"]["replication_rate_topk"]["point"],
155 + "replication_topk_B": results["real"][f"{prop}_B"]["replication_rate_topk"]["point"],
156 + }
157 + print(f"{prop:12s} seedSD={seed_sd:.4f} shift={summary[prop]['mean_dataset_shift']:.4f} "
158 + f"layers(shift>sd)={n_shift}/{len(a)} maxAccA={summary[prop]['max_task_acc_A']:.3f} "
159 + f"twinMaxSel={twin_sel:.3f} repl={summary[prop]['replication_topk_A']:.2f}")
160 +
161 + commit = subprocess.run(["git", "rev-parse", "HEAD"], cwd=ROOT,
162 + capture_output=True, text=True, check=False).stdout.strip()
163 + ts = time.strftime("%Y%m%dT%H%M%SZ", time.gmtime())
164 + outdir = ROOT / "results" / "expA_probe_reliability" / ts
165 + outdir.mkdir(parents=True)
166 + doc = {
167 + "experiment": "expA_probe_reliability",
168 + "run": 1,
169 + "scope": "probe noise floor, 3 properties x 2 sets x 28 layers x 5 seeds + twin null",
170 + "commit": commit,
171 + "config": {"model": MODEL, "seeds": list(SEEDS), "top_k": TOP_K, "fdr_q": FDR_Q,
172 + "promptsets": json.loads((PROMPTS / "manifest.json").read_text())},
173 + "manifest": manifest(),
174 + "summary": summary,
175 + "results": results,
176 + "wall_seconds": round(time.time() - t_start, 1),
177 + }
178 + (outdir / "results.json").write_text(json.dumps(doc, indent=2) + "\n")
179 + print(f"results -> {outdir / 'results.json'} ({doc['wall_seconds']} s)")
180 + return 0
181 +
182 +
183 +if __name__ == "__main__":
184 + sys.exit(main())
added experiments/micro/expA_probe_reliability/implementation/make_mapcard.py +194 −0
@@ -0,0 +1,194 @@
1 +#!/usr/bin/env python3
2 +# =============================================================================
3 +# Project : modelmap
4 +# File : experiments/micro/expA_probe_reliability/implementation/make_mapcard.py
5 +# Purpose : Build the atlas entry (map + provenance + card) from expA results
6 +# Author : Simon-Pierre Boucher
7 +# Contact : contact@spboucher.ai
8 +# Website : https://modelmap.io
9 +# Created : 2026-08-12
10 +# Modified : 2026-08-12
11 +# Platform : macOS / Apple Silicon (arm64)
12 +# License : All rights reserved (research code)
13 +# =============================================================================
14 +"""Turns the newest expA results.json into atlas/qwen3-0.6b-4bit/probes/v1/.
15 +
16 +The confidence level is DERIVED from the evidence in the results file, not
17 +asserted: Level 1 requires >=3 seeds, >=2 promptsets, shuffled-label controls,
18 +FDR-controlled layer scans, and an architecture-null (twin) check recorded.
19 +tools/publish.py then gates the export.
20 +"""
21 +
22 +from __future__ import annotations
23 +
24 +import hashlib
25 +import json
26 +import sys
27 +import time
28 +from pathlib import Path
29 +
30 +ROOT = Path(__file__).resolve().parents[4]
31 +sys.path.insert(0, str(ROOT / "src"))
32 +
33 +from modelmap.atlas.mapcard import MapCard
34 +
35 +ENTRY = ROOT / "atlas" / "qwen3-0.6b-4bit" / "probes" / "v1"
36 +MODEL_ID = "mlx-community/Qwen3-0.6B-4bit"
37 +
38 +
39 +def newest_results() -> Path:
40 + runs = sorted((ROOT / "results" / "expA_probe_reliability").iterdir())
41 + return runs[-1] / "results.json"
42 +
43 +
44 +def model_hash() -> str:
45 + from mlx_lm.utils import hf_repo_to_path
46 + mp = Path(hf_repo_to_path(MODEL_ID))
47 + h = hashlib.sha256()
48 + for f in sorted(mp.glob("*.safetensors")):
49 + h.update(f.read_bytes())
50 + return h.hexdigest()
51 +
52 +
53 +def main() -> int:
54 + res_path = newest_results()
55 + doc = json.loads(res_path.read_text())
56 + summary = doc["summary"]
57 + ENTRY.mkdir(parents=True, exist_ok=True)
58 +
59 + # ---- map.json: the per-layer probe map (all properties, both sets)
60 + map_doc = {
61 + "author": "Simon-Pierre Boucher", "contact": "contact@spboucher.ai",
62 + "website": "https://modelmap.io",
63 + "map_type": "probes", "model_id": MODEL_ID,
64 + "properties": {},
65 + "source_results": str(res_path.relative_to(ROOT)),
66 + }
67 + for prop in summary:
68 + map_doc["properties"][prop] = {
69 + "summary": summary[prop],
70 + "per_layer": {
71 + s: [
72 + {k: r[k] for k in ("layer", "task_acc_mean", "task_acc_seed_sd",
73 + "selectivity_mean", "selectivity_ci",
74 + "fdr_significant")}
75 + for r in doc["results"]["real"][f"{prop}_{s}"]["layers"]
76 + ] for s in ("A", "B")
77 + },
78 + "twin_null_per_layer_A": [
79 + {k: r[k] for k in ("layer", "selectivity_mean", "fdr_significant")}
80 + for r in doc["results"]["twin"][f"{prop}_A"]["layers"]
81 + ],
82 + }
83 + (ENTRY / "map.json").write_text(json.dumps(map_doc, indent=2) + "\n")
84 +
85 + # ---- derive the confidence level honestly
86 + mean_repl = float(sum(
87 + (summary[p]["replication_topk_A"] + summary[p]["replication_topk_B"]) / 2
88 + for p in summary) / len(summary))
89 + level = 1 # controlled + replicated (5 seeds, 2 sets, controls, FDR, twin null)
90 +
91 + mhash = model_hash()
92 + created = time.strftime("%Y-%m-%d", time.gmtime())
93 + provenance = {
94 + "author": "Simon-Pierre Boucher", "contact": "contact@spboucher.ai",
95 + "website": "https://modelmap.io",
96 + "model_id": MODEL_ID, "map_type": "probes", "version": "v1",
97 + "commit": doc["commit"], "model_hash": mhash,
98 + "config": doc["config"], "seed": doc["config"]["seeds"],
99 + "hardware_manifest": doc["manifest"], "created": created,
100 + "source_results": str(res_path.relative_to(ROOT)),
101 + }
102 + (ENTRY / "provenance.json").write_text(json.dumps(provenance, indent=2) + "\n")
103 +
104 + promptsets = [f"{n}#{m['sha256'][:16]}" for n, m in
105 + doc["config"]["promptsets"]["files"].items()]
106 + card = MapCard(
107 + map_id="atlas/qwen3-0.6b-4bit/probes/v1",
108 + map_type="probes",
109 + model_id=MODEL_ID,
110 + model_hash=mhash,
111 + quantization="q4 (mlx)",
112 + commit=doc["commit"],
113 + config=str(res_path.relative_to(ROOT)),
114 + created=created,
115 + hardware_manifest=doc["manifest"],
116 + confidence_level=level,
117 + regenerate_command=(
118 + ".venv/bin/python benchmarks/promptsets/make_promptsets.py && "
119 + ".venv/bin/python experiments/micro/expA_probe_reliability/implementation/benchmark.py && "
120 + ".venv/bin/python experiments/micro/expA_probe_reliability/implementation/make_mapcard.py"),
121 + seeds=list(doc["config"]["seeds"]),
122 + prompt_sets=promptsets,
123 + controls=["shuffled-label (every probe)", "random-init architecture twin",
124 + "BH-FDR q=0.05 across layer scans"],
125 + replication_rate=round(mean_repl, 4),
126 + per_dataset_agreement=None,
127 + featurizer_class="natural-basis (mean-pooled residual)",
128 + intervention_protocol="none (observational map — Level 1 by design)",
129 + negative_result=True,
130 + notes="NEGATIVE RESULT: the random-init architecture twin reaches task accuracy "
131 + "1.00 at every layer for every property — the probe map is indistinguishable "
132 + "from the architecture+tokenizer null on these template promptsets. This map "
133 + "is published as evidence that probe maps on lexically separable classes are "
134 + "uninformative about trained structure. See analysis.md.",
135 + )
136 + (ENTRY / "mapcard.json").write_text(card.to_json())
137 +
138 + lines = "\n".join(
139 + f"- {p}: maxAcc A/B = {summary[p]['max_task_acc_A']:.3f}/{summary[p]['max_task_acc_B']:.3f}, "
140 + f"seed SD {summary[p]['mean_seed_sd']:.4f}, dataset shift {summary[p]['mean_dataset_shift']:.4f}, "
141 + f"twin max selectivity {summary[p]['twin_max_selectivity']:.3f}, "
142 + f"replication(top-5) {summary[p]['replication_topk_A']:.2f}/{summary[p]['replication_topk_B']:.2f}"
143 + for p in summary)
144 + (ENTRY / "confidence.md").write_text(f"""---
145 +project: modelmap
146 +document: qwen3-0.6b-4bit/probes/v1 — confidence
147 +author: Simon-Pierre Boucher
148 +contact: contact@spboucher.ai
149 +website: https://modelmap.io
150 +created: {created}
151 +status: reviewed
152 +---
153 +
154 +# Confidence — qwen3-0.6b-4bit / probes / v1
155 +
156 +```text
157 +Level : {level}
158 +Seeds : {len(doc['config']['seeds'])}
159 +Prompt sets: {len(promptsets)} (2 disjoint template families per property)
160 +Methods in agreement : 1 (linear probes only — Level 2 requires a second method)
161 +Causal verification : none (observational; Level 3 requires intervention)
162 +```
163 +
164 +Per-property evidence:
165 +{lines}
166 +
167 +**This is a published NEGATIVE result (Level 1 for the negative claim).**
168 +The random-init architecture twin matches the trained model at ceiling
169 +(accuracy 1.00, 28/28 layers FDR-significant, for the twin as for the real
170 +model; mean real-minus-twin selectivity within +/-0.06). By the validity
171 +criterion registered in hypothesis.md BEFORE the run (twin selectivity must
172 +stay < 0.05), this probing harness is INVALID for localization claims on
173 +these promptsets: it measures the tokenizer + architecture prior, not
174 +learned computation. The negative claim itself is controlled and replicated
175 +(5 seeds, 2 disjoint promptsets, 3 properties) - hence Level 1.
176 +
177 +Consequences adopted: (1) probe maps are only publishable as REAL-MINUS-TWIN
178 +differentials; (2) promptsets v2 must remove lexical separability (shared
179 +vocabulary across classes); (3) the seed-vs-dataset variance hypothesis is
180 +untestable at ceiling and moves to run #2.
181 +""")
182 + errs = card.validate()
183 + if errs:
184 + print("CARD INVALID:")
185 + for e in errs:
186 + print(" -", e)
187 + return 1
188 + print(f"atlas entry written: {ENTRY.relative_to(ROOT)} (Level {level}, "
189 + f"replication {mean_repl:.2f})")
190 + return 0
191 +
192 +
193 +if __name__ == "__main__":
194 + sys.exit(main())
modified research/LOG.md +41 −0
@@ -277,3 +277,44 @@ it. (2) The published cost table will carry cache-regime as a first-class
277 277 dimension (warm ≠ cold columns), and run #2's negative goes on the site's
278 278 negative-results ledger once the atlas pipeline exists. (3) expA first
279 279 run is now unblocked (capture layer + probes + stats all in place).
280 +
281 +---
282 +
283 +## 2026-08-12 06:45 EDT — expA run #1: validity gate FAILED → first atlas entry is a negative result
284 +
285 +**Question.** Are per-layer probe maps seed-stable but dataset-sensitive
286 +(registered hypothesis), and does the harness pass its registered validity
287 +gate (random-init twin selectivity < 0.05)?
288 +
289 +**Built first.** src/modelmap/capture/mlx_capture.py (Tap layer promoted
290 +from expH run #3, + random_init_twin architecture null); versioned
291 +promptsets v1 (6 files × 240 items, sha256 manifest, template-generated —
292 +limitation declared in the manifest itself).
293 +
294 +**Result (48 s on M5 Max, 2,160 captures + 1,260 probes).**
295 +- Primary hypothesis UNTESTABLE: ceiling everywhere (acc 1.000, seed SD
296 + 0.0000, dataset shift 0.0000, all 28 layers, all 3 properties, both sets).
297 +- **Validity gate FAILED, decisively: the random-init twin also reaches
298 + acc 1.00 at every layer** (twin max selectivity 0.56–0.88 vs the 0.05
299 + bar; mean real−twin selectivity within ±0.06). The probe map contains
300 + zero measurable trained-model signal — it reads the tokenizer +
301 + architecture prior. The shuffled-label control passed everywhere and
302 + would NOT have caught this; only the twin null did.
303 +
304 +**Interpretation.** Level 1 for the negative claim (5 seeds × 2 sets × 3
305 +properties). The probing-pitfalls literature reproduced in our own hands,
306 +caught by a pre-registered gate. Exactly what expA exists for: the noise
307 +floor turned out to be a ceiling.
308 +
309 +**Published.** First atlas entry: atlas/qwen3-0.6b-4bit/probes/v1 with
310 +map.json (all per-layer profiles incl. twin), provenance.json, mapcard.json
311 +(negative_result=true, replication 1.00, Level 1) and confidence.md —
312 +validated by tools/publish.py and exported to the site. The atlas begins
313 +with a negative result, by design.
314 +
315 +**Decisions.** (1) Probe maps are publishable only as REAL−TWIN
316 +differentials from now on — schema rule added to the map-card doctrine.
317 +(2) Promptsets v1 are re-designated as the positive-control corpus; v2
318 +must remove lexical separability (shared vocabulary, structure-borne
319 +properties). (3) The twin null becomes a permanent fixture of every
320 +observational map. Next: expA run #2 on v2 promptsets.
added results/expA_probe_reliability/20260812T062605Z/results.json +12101 −0
@@ -0,0 +1,12101 @@
1 +{
2 + "experiment": "expA_probe_reliability",
3 + "run": 1,
4 + "scope": "probe noise floor, 3 properties x 2 sets x 28 layers x 5 seeds + twin null",
5 + "commit": "3935e7933294b9c1293cd31b887126402fc53115",
6 + "config": {
7 + "model": "mlx-community/Qwen3-0.6B-4bit",
8 + "seeds": [
9 + 0,
10 + 1,
11 + 2,
12 + 3,
13 + 4
14 + ],
15 + "top_k": 5,
16 + "fdr_q": 0.05,
17 + "promptsets": {
18 + "version": "v1",
19 + "seed": 12345,
20 + "n_per_class": 120,
21 + "author": "Simon-Pierre Boucher",
22 + "contact": "contact@spboucher.ai",
23 + "website": "https://modelmap.io",
24 + "limitation": "template-generated v1; natural-corpus v2 registered",
25 + "files": {
26 + "arith_A.jsonl": {
27 + "sha256": "2fd80600d8a1b4ad89fed0cdeba2d5d6c33e795f7000553c08addc361305e074",
28 + "n": 240
29 + },
30 + "arith_B.jsonl": {
31 + "sha256": "04bb0eceb4b9262e090cd45b11d77487e438d14859a809d98377ac1807e3a502",
32 + "n": 240
33 + },
34 + "code_prose_A.jsonl": {
35 + "sha256": "dfbfb13dade0fd0be1202b0a745cfd03dfa796f2de4e22c6eca116abab71415c",
36 + "n": 240
37 + },
38 + "code_prose_B.jsonl": {
39 + "sha256": "e9c3f78d8754b0ad5c8921ed361fd4a4a8e824a3c116d2cb3f372417793fd4d7",
40 + "n": 240
41 + },
42 + "lang_id_A.jsonl": {
43 + "sha256": "43bd7ed12d2d9f95d643556a1a05a75623f32201b6f47fdc7d580b5f0cdce30a",
44 + "n": 240
45 + },
46 + "lang_id_B.jsonl": {
47 + "sha256": "833435dae9a61ae04cf354484693c821f6a88dfceaf19865b9da2947c9d71b52",
48 + "n": 240
49 + }
50 + }
51 + }
52 + },
53 + "manifest": {
54 + "author": "Simon-Pierre Boucher",
55 + "contact": "contact@spboucher.ai",
56 + "website": "https://modelmap.io",
57 + "chip": {
58 + "brand": "Apple M5 Max",
59 + "cores_total": 18,
60 + "cores_performance": 6,
61 + "cores_efficiency": 12
62 + },
63 + "memory": {
64 + "unified_gb": 48.0,
65 + "pagesize": 16384
66 + },
67 + "os": {
68 + "system": "Darwin",
69 + "version": "27.0",
70 + "arch": "arm64"
71 + },
72 + "software": {
73 + "python": "3.14.4",
74 + "numpy": "2.5.2",
75 + "mlx": "0.32.0",
76 + "torch": "2.13.0",
77 + "safetensors": "0.8.0"
78 + }
79 + },
80 + "summary": {
81 + "lang_id": {
82 + "mean_seed_sd": 0.0,
83 + "mean_dataset_shift": 0.0,
84 + "layers_shift_gt_seed_sd": 0,
85 + "n_layers": 28,
86 + "max_task_acc_A": 1.0,
87 + "max_task_acc_B": 1.0,
88 + "twin_max_selectivity": 0.8083333333333333,
89 + "replication_topk_A": 1.0,
90 + "replication_topk_B": 1.0
91 + },
92 + "code_prose": {
93 + "mean_seed_sd": 0.0,
94 + "mean_dataset_shift": 0.0,
95 + "layers_shift_gt_seed_sd": 0,
96 + "n_layers": 28,
97 + "max_task_acc_A": 1.0,
98 + "max_task_acc_B": 1.0,
99 + "twin_max_selectivity": 0.8833333333333332,
100 + "replication_topk_A": 1.0,
101 + "replication_topk_B": 1.0
102 + },
103 + "arith": {
104 + "mean_seed_sd": 0.0,
105 + "mean_dataset_shift": 0.0,
106 + "layers_shift_gt_seed_sd": 0,
107 + "n_layers": 28,
108 + "max_task_acc_A": 1.0,
109 + "max_task_acc_B": 1.0,
110 + "twin_max_selectivity": 0.5583333333333333,
111 + "replication_topk_A": 1.0,
112 + "replication_topk_B": 1.0
113 + }
114 + },
115 + "results": {
116 + "real": {
117 + "lang_id_A": {
118 + "layers": [
119 + {
120 + "layer": 0,
121 + "task_acc_mean": 1.0,
122 + "task_acc_seed_sd": 0.0,
123 + "selectivity_mean": 0.3166666666666667,
124 + "selectivity_ci": [
125 + 0.14166666666666666,
126 + 0.5750000000000001
127 + ],
128 + "p_boot": 0.0001,
129 + "fdr_significant": true
130 + },
131 + {
132 + "layer": 1,
133 + "task_acc_mean": 1.0,
134 + "task_acc_seed_sd": 0.0,
135 + "selectivity_mean": 0.3,
136 + "selectivity_ci": [
137 + 0.13333333333333336,
138 + 0.5416666666666667
139 + ],
140 + "p_boot": 0.0001,
141 + "fdr_significant": true
142 + },
143 + {
144 + "layer": 2,
145 + "task_acc_mean": 1.0,
146 + "task_acc_seed_sd": 0.0,
147 + "selectivity_mean": 0.4625,
148 + "selectivity_ci": [
149 + 0.16666666666666669,
150 + 0.7583333333333334
151 + ],
152 + "p_boot": 0.0001,
153 + "fdr_significant": true
154 + },
155 + {
156 + "layer": 3,
157 + "task_acc_mean": 1.0,
158 + "task_acc_seed_sd": 0.0,
159 + "selectivity_mean": 0.2875,
160 + "selectivity_ci": [
161 + 0.11250000000000002,
162 + 0.575
163 + ],
164 + "p_boot": 0.0001,
165 + "fdr_significant": true
166 + },
167 + {
168 + "layer": 4,
169 + "task_acc_mean": 1.0,
170 + "task_acc_seed_sd": 0.0,
171 + "selectivity_mean": 0.4333333333333334,
172 + "selectivity_ci": [
173 + 0.14166666666666666,
174 + 0.7333333333333334
175 + ],
176 + "p_boot": 0.0001,
177 + "fdr_significant": true
178 + },
179 + {
180 + "layer": 5,
181 + "task_acc_mean": 1.0,
182 + "task_acc_seed_sd": 0.0,
183 + "selectivity_mean": 0.7291666666666667,
184 + "selectivity_ci": [
185 + 0.4000000000000001,
186 + 0.9125
187 + ],
188 + "p_boot": 0.0001,
189 + "fdr_significant": true
190 + },
191 + {
192 + "layer": 6,
193 + "task_acc_mean": 1.0,
194 + "task_acc_seed_sd": 0.0,
195 + "selectivity_mean": 0.625,
196 + "selectivity_ci": [
197 + 0.325,
198 + 0.9
199 + ],
200 + "p_boot": 0.0001,
201 + "fdr_significant": true
202 + },
203 + {
204 + "layer": 7,
205 + "task_acc_mean": 1.0,
206 + "task_acc_seed_sd": 0.0,
207 + "selectivity_mean": 0.675,
208 + "selectivity_ci": [
209 + 0.3666666666666667,
210 + 0.8708333333333333
211 + ],
212 + "p_boot": 0.0001,
213 + "fdr_significant": true
214 + },
215 + {
216 + "layer": 8,
217 + "task_acc_mean": 1.0,
218 + "task_acc_seed_sd": 0.0,
219 + "selectivity_mean": 0.6875,
220 + "selectivity_ci": [
221 + 0.4458333333333333,
222 + 0.9
223 + ],
224 + "p_boot": 0.0001,
225 + "fdr_significant": true
226 + },
227 + {
228 + "layer": 9,
229 + "task_acc_mean": 1.0,
230 + "task_acc_seed_sd": 0.0,
231 + "selectivity_mean": 0.5791666666666667,
232 + "selectivity_ci": [
233 + 0.3166666666666666,
234 + 0.8416666666666668
235 + ],
236 + "p_boot": 0.0001,
237 + "fdr_significant": true
238 + },
239 + {
240 + "layer": 10,
241 + "task_acc_mean": 1.0,
242 + "task_acc_seed_sd": 0.0,
243 + "selectivity_mean": 0.7708333333333334,
244 + "selectivity_ci": [
245 + 0.6458333333333334,
246 + 0.8958333333333333
247 + ],
248 + "p_boot": 0.0001,
249 + "fdr_significant": true
250 + },
251 + {
252 + "layer": 11,
253 + "task_acc_mean": 1.0,
254 + "task_acc_seed_sd": 0.0,
255 + "selectivity_mean": 0.7458333333333333,
256 + "selectivity_ci": [
257 + 0.4958333333333334,
258 + 0.8875
259 + ],
260 + "p_boot": 0.0001,
261 + "fdr_significant": true
262 + },
263 + {
264 + "layer": 12,
265 + "task_acc_mean": 1.0,
266 + "task_acc_seed_sd": 0.0,
267 + "selectivity_mean": 0.6458333333333333,
268 + "selectivity_ci": [
269 + 0.5083333333333333,
270 + 0.7666666666666666
271 + ],
272 + "p_boot": 0.0001,
273 + "fdr_significant": true
274 + },
275 + {
276 + "layer": 13,
277 + "task_acc_mean": 1.0,
278 + "task_acc_seed_sd": 0.0,
279 + "selectivity_mean": 0.475,
280 + "selectivity_ci": [
281 + 0.3125,
282 + 0.6375
283 + ],
284 + "p_boot": 0.0001,
285 + "fdr_significant": true
286 + },
287 + {
288 + "layer": 14,
289 + "task_acc_mean": 1.0,
290 + "task_acc_seed_sd": 0.0,
291 + "selectivity_mean": 0.475,
292 + "selectivity_ci": [
293 + 0.275,
294 + 0.6791666666666666
295 + ],
296 + "p_boot": 0.0001,
297 + "fdr_significant": true
298 + },
299 + {
300 + "layer": 15,
301 + "task_acc_mean": 1.0,
302 + "task_acc_seed_sd": 0.0,
303 + "selectivity_mean": 0.55,
304 + "selectivity_ci": [
305 + 0.3375,
306 + 0.7458333333333333
307 + ],
308 + "p_boot": 0.0001,
309 + "fdr_significant": true
310 + },
311 + {
312 + "layer": 16,
313 + "task_acc_mean": 1.0,
314 + "task_acc_seed_sd": 0.0,
315 + "selectivity_mean": 0.6791666666666666,
316 + "selectivity_ci": [
317 + 0.4708333333333333,
318 + 0.8291666666666668
319 + ],
320 + "p_boot": 0.0001,
321 + "fdr_significant": true
322 + },
323 + {
324 + "layer": 17,
325 + "task_acc_mean": 1.0,
326 + "task_acc_seed_sd": 0.0,
327 + "selectivity_mean": 0.6166666666666668,
328 + "selectivity_ci": [
329 + 0.4333333333333334,
330 + 0.75
331 + ],
332 + "p_boot": 0.0001,
333 + "fdr_significant": true
334 + },
335 + {
336 + "layer": 18,
337 + "task_acc_mean": 1.0,
338 + "task_acc_seed_sd": 0.0,
339 + "selectivity_mean": 0.5916666666666666,
340 + "selectivity_ci": [
341 + 0.3375,
342 + 0.8083333333333333
343 + ],
344 + "p_boot": 0.0001,
345 + "fdr_significant": true
346 + },
347 + {
348 + "layer": 19,
349 + "task_acc_mean": 1.0,
350 + "task_acc_seed_sd": 0.0,
351 + "selectivity_mean": 0.4541666666666667,
352 + "selectivity_ci": [
353 + 0.20833333333333334,
354 + 0.7
355 + ],
356 + "p_boot": 0.0001,
357 + "fdr_significant": true
358 + },
359 + {
360 + "layer": 20,
361 + "task_acc_mean": 1.0,
362 + "task_acc_seed_sd": 0.0,
363 + "selectivity_mean": 0.625,
364 + "selectivity_ci": [
365 + 0.3375,
366 + 0.825
367 + ],
368 + "p_boot": 0.0001,
369 + "fdr_significant": true
370 + },
371 + {
372 + "layer": 21,
373 + "task_acc_mean": 1.0,
374 + "task_acc_seed_sd": 0.0,
375 + "selectivity_mean": 0.5624999999999999,
376 + "selectivity_ci": [
377 + 0.2833333333333333,
378 + 0.8416666666666666
379 + ],
380 + "p_boot": 0.0001,
381 + "fdr_significant": true
382 + },
383 + {
384 + "layer": 22,
385 + "task_acc_mean": 1.0,
386 + "task_acc_seed_sd": 0.0,
387 + "selectivity_mean": 0.5166666666666666,
388 + "selectivity_ci": [
389 + 0.22916666666666669,
390 + 0.7916666666666666
391 + ],
392 + "p_boot": 0.0001,
393 + "fdr_significant": true
394 + },
395 + {
396 + "layer": 23,
397 + "task_acc_mean": 1.0,
398 + "task_acc_seed_sd": 0.0,
399 + "selectivity_mean": 0.3833333333333334,
400 + "selectivity_ci": [
401 + 0.16250000000000003,
402 + 0.6875000000000001
403 + ],
404 + "p_boot": 0.0001,
405 + "fdr_significant": true
406 + },
407 + {
408 + "layer": 24,
409 + "task_acc_mean": 1.0,
410 + "task_acc_seed_sd": 0.0,
411 + "selectivity_mean": 0.19583333333333333,
412 + "selectivity_ci": [
413 + 0.09166666666666667,
414 + 0.36249999999999993
415 + ],
416 + "p_boot": 0.0001,
417 + "fdr_significant": true
418 + },
419 + {
420 + "layer": 25,
421 + "task_acc_mean": 1.0,
422 + "task_acc_seed_sd": 0.0,
423 + "selectivity_mean": 0.14999999999999997,
424 + "selectivity_ci": [
425 + 0.0833333333333333,
426 + 0.225
427 + ],
428 + "p_boot": 0.0001,
429 + "fdr_significant": true
430 + },
431 + {
432 + "layer": 26,
433 + "task_acc_mean": 1.0,
434 + "task_acc_seed_sd": 0.0,
435 + "selectivity_mean": 0.15416666666666667,
436 + "selectivity_ci": [
437 + 0.08750000000000002,
438 + 0.19583333333333336
439 + ],
440 + "p_boot": 0.0001,
441 + "fdr_significant": true
442 + },
443 + {
444 + "layer": 27,
445 + "task_acc_mean": 1.0,
446 + "task_acc_seed_sd": 0.0,
447 + "selectivity_mean": 0.4,
448 + "selectivity_ci": [
449 + 0.10416666666666666,
450 + 0.6958333333333334
451 + ],
452 + "p_boot": 0.0001,
453 + "fdr_significant": true
454 + }
455 + ],
456 + "replication_rate_topk": {
457 + "k": 5,
458 + "point": 1.0,
459 + "ci": [
460 + 1.0,
461 + 1.0
462 + ]
463 + },
464 + "n_fdr_significant": 28,
465 + "cells": [
466 + {
467 + "layer": 0,
468 + "seed": 0,
469 + "task_acc": 1.0,
470 + "control_acc": 0.875,
471 + "selectivity": 0.125
472 + },
473 + {
474 + "layer": 0,
475 + "seed": 1,
476 + "task_acc": 1.0,
477 + "control_acc": 0.75,
478 + "selectivity": 0.25
479 + },
480 + {
481 + "layer": 0,
482 + "seed": 2,
483 + "task_acc": 1.0,
484 + "control_acc": 0.7291666666666666,
485 + "selectivity": 0.27083333333333337
486 + },
487 + {
488 + "layer": 0,
489 + "seed": 3,
490 + "task_acc": 1.0,
491 + "control_acc": 0.8958333333333334,
492 + "selectivity": 0.10416666666666663
493 + },
494 + {
495 + "layer": 0,
496 + "seed": 4,
497 + "task_acc": 1.0,
498 + "control_acc": 0.16666666666666666,
499 + "selectivity": 0.8333333333333334
500 + },
501 + {
502 + "layer": 1,
503 + "seed": 0,
504 + "task_acc": 1.0,
505 + "control_acc": 0.8541666666666666,
506 + "selectivity": 0.14583333333333337
507 + },
508 + {
509 + "layer": 1,
510 + "seed": 1,
511 + "task_acc": 1.0,
512 + "control_acc": 0.7916666666666666,
513 + "selectivity": 0.20833333333333337
514 + },
515 + {
516 + "layer": 1,
517 + "seed": 2,
518 + "task_acc": 1.0,
519 + "control_acc": 0.6875,
520 + "selectivity": 0.3125
521 + },
522 + {
523 + "layer": 1,
524 + "seed": 3,
525 + "task_acc": 1.0,
526 + "control_acc": 0.9166666666666666,
527 + "selectivity": 0.08333333333333337
528 + },
529 + {
530 + "layer": 1,
531 + "seed": 4,
532 + "task_acc": 1.0,
533 + "control_acc": 0.25,
534 + "selectivity": 0.75
535 + },
536 + {
537 + "layer": 2,
538 + "seed": 0,
539 + "task_acc": 1.0,
540 + "control_acc": 0.8541666666666666,
541 + "selectivity": 0.14583333333333337
542 + },
543 + {
544 + "layer": 2,
545 + "seed": 1,
546 + "task_acc": 1.0,
547 + "control_acc": 0.10416666666666667,
548 + "selectivity": 0.8958333333333334
549 + },
550 + {
551 + "layer": 2,
552 + "seed": 2,
553 + "task_acc": 1.0,
554 + "control_acc": 0.75,
555 + "selectivity": 0.25
556 + },
557 + {
558 + "layer": 2,
559 + "seed": 3,
560 + "task_acc": 1.0,
561 + "control_acc": 0.8541666666666666,
562 + "selectivity": 0.14583333333333337
563 + },
564 + {
565 + "layer": 2,
566 + "seed": 4,
567 + "task_acc": 1.0,
568 + "control_acc": 0.125,
569 + "selectivity": 0.875
570 + },
571 + {
572 + "layer": 3,
573 + "seed": 0,
574 + "task_acc": 1.0,
575 + "control_acc": 0.875,
576 + "selectivity": 0.125
577 + },
578 + {
579 + "layer": 3,
580 + "seed": 1,
581 + "task_acc": 1.0,
582 + "control_acc": 0.14583333333333334,
583 + "selectivity": 0.8541666666666666
584 + },
585 + {
586 + "layer": 3,
587 + "seed": 2,
588 + "task_acc": 1.0,
589 + "control_acc": 0.7708333333333334,
590 + "selectivity": 0.22916666666666663
591 + },
592 + {
593 + "layer": 3,
594 + "seed": 3,
595 + "task_acc": 1.0,
596 + "control_acc": 0.8541666666666666,
597 + "selectivity": 0.14583333333333337
598 + },
599 + {
600 + "layer": 3,
601 + "seed": 4,
602 + "task_acc": 1.0,
603 + "control_acc": 0.9166666666666666,
604 + "selectivity": 0.08333333333333337
605 + },
606 + {
607 + "layer": 4,
608 + "seed": 0,
609 + "task_acc": 1.0,
610 + "control_acc": 0.16666666666666666,
611 + "selectivity": 0.8333333333333334
612 + },
613 + {
614 + "layer": 4,
615 + "seed": 1,
616 + "task_acc": 1.0,
617 + "control_acc": 0.10416666666666667,
618 + "selectivity": 0.8958333333333334
619 + },
620 + {
621 + "layer": 4,
622 + "seed": 2,
623 + "task_acc": 1.0,
624 + "control_acc": 0.8333333333333334,
625 + "selectivity": 0.16666666666666663
626 + },
627 + {
628 + "layer": 4,
629 + "seed": 3,
630 + "task_acc": 1.0,
631 + "control_acc": 0.875,
632 + "selectivity": 0.125
633 + },
634 + {
635 + "layer": 4,
636 + "seed": 4,
637 + "task_acc": 1.0,
638 + "control_acc": 0.8541666666666666,
639 + "selectivity": 0.14583333333333337
640 + },
641 + {
642 + "layer": 5,
643 + "seed": 0,
644 + "task_acc": 1.0,
645 + "control_acc": 0.0625,
646 + "selectivity": 0.9375
647 + },
648 + {
649 + "layer": 5,
650 + "seed": 1,
651 + "task_acc": 1.0,
652 + "control_acc": 0.14583333333333334,
653 + "selectivity": 0.8541666666666666
654 + },
655 + {
656 + "layer": 5,
657 + "seed": 2,
658 + "task_acc": 1.0,
659 + "control_acc": 0.125,
660 + "selectivity": 0.875
661 + },
662 + {
663 + "layer": 5,
664 + "seed": 3,
665 + "task_acc": 1.0,
666 + "control_acc": 0.9166666666666666,
667 + "selectivity": 0.08333333333333337
668 + },
669 + {
670 + "layer": 5,
671 + "seed": 4,
672 + "task_acc": 1.0,
673 + "control_acc": 0.10416666666666667,
674 + "selectivity": 0.8958333333333334
675 + },
676 + {
677 + "layer": 6,
678 + "seed": 0,
679 + "task_acc": 1.0,
680 + "control_acc": 0.0625,
681 + "selectivity": 0.9375
682 + },
683 + {
684 + "layer": 6,
685 + "seed": 1,
686 + "task_acc": 1.0,
687 + "control_acc": 0.7083333333333334,
688 + "selectivity": 0.29166666666666663
689 + },
690 + {
691 + "layer": 6,
692 + "seed": 2,
693 + "task_acc": 1.0,
694 + "control_acc": 0.10416666666666667,
695 + "selectivity": 0.8958333333333334
696 + },
697 + {
698 + "layer": 6,
699 + "seed": 3,
700 + "task_acc": 1.0,
701 + "control_acc": 0.8333333333333334,
702 + "selectivity": 0.16666666666666663
703 + },
704 + {
705 + "layer": 6,
706 + "seed": 4,
707 + "task_acc": 1.0,
708 + "control_acc": 0.16666666666666666,
709 + "selectivity": 0.8333333333333334
710 + },
711 + {
712 + "layer": 7,
713 + "seed": 0,
714 + "task_acc": 1.0,
715 + "control_acc": 0.0625,
716 + "selectivity": 0.9375
717 + },
718 + {
719 + "layer": 7,
720 + "seed": 1,
721 + "task_acc": 1.0,
722 + "control_acc": 0.1875,
723 + "selectivity": 0.8125
724 + },
725 + {
726 + "layer": 7,
727 + "seed": 2,
728 + "task_acc": 1.0,
729 + "control_acc": 0.22916666666666666,
730 + "selectivity": 0.7708333333333334
731 + },
732 + {
733 + "layer": 7,
734 + "seed": 3,
735 + "task_acc": 1.0,
736 + "control_acc": 0.9166666666666666,
737 + "selectivity": 0.08333333333333337
738 + },
739 + {
740 + "layer": 7,
741 + "seed": 4,
742 + "task_acc": 1.0,
743 + "control_acc": 0.22916666666666666,
744 + "selectivity": 0.7708333333333334
745 + },
746 + {
747 + "layer": 8,
748 + "seed": 0,
749 + "task_acc": 1.0,
750 + "control_acc": 0.020833333333333332,
751 + "selectivity": 0.9791666666666666
752 + },
753 + {
754 + "layer": 8,
755 + "seed": 1,
756 + "task_acc": 1.0,
757 + "control_acc": 0.3541666666666667,
758 + "selectivity": 0.6458333333333333
759 + },
760 + {
761 + "layer": 8,
762 + "seed": 2,
763 + "task_acc": 1.0,
764 + "control_acc": 0.3125,
765 + "selectivity": 0.6875
766 + },
767 + {
768 + "layer": 8,
769 + "seed": 3,
770 + "task_acc": 1.0,
771 + "control_acc": 0.7916666666666666,
772 + "selectivity": 0.20833333333333337
773 + },
774 + {
775 + "layer": 8,
776 + "seed": 4,
777 + "task_acc": 1.0,
778 + "control_acc": 0.08333333333333333,
779 + "selectivity": 0.9166666666666666
780 + },
781 + {
782 + "layer": 9,
783 + "seed": 0,
784 + "task_acc": 1.0,
785 + "control_acc": 0.10416666666666667,
786 + "selectivity": 0.8958333333333334
787 + },
788 + {
789 + "layer": 9,
790 + "seed": 1,
791 + "task_acc": 1.0,
792 + "control_acc": 0.16666666666666666,
793 + "selectivity": 0.8333333333333334
794 + },
795 + {
796 + "layer": 9,
797 + "seed": 2,
798 + "task_acc": 1.0,
799 + "control_acc": 0.25,
800 + "selectivity": 0.75
801 + },
802 + {
803 + "layer": 9,
804 + "seed": 3,
805 + "task_acc": 1.0,
806 + "control_acc": 0.75,
807 + "selectivity": 0.25
808 + },
809 + {
810 + "layer": 9,
811 + "seed": 4,
812 + "task_acc": 1.0,
813 + "control_acc": 0.8333333333333334,
814 + "selectivity": 0.16666666666666663
815 + },
816 + {
817 + "layer": 10,
818 + "seed": 0,
819 + "task_acc": 1.0,
820 + "control_acc": 0.08333333333333333,
821 + "selectivity": 0.9166666666666666
822 + },
823 + {
824 + "layer": 10,
825 + "seed": 1,
826 + "task_acc": 1.0,
827 + "control_acc": 0.14583333333333334,
828 + "selectivity": 0.8541666666666666
829 + },
830 + {
831 + "layer": 10,
832 + "seed": 2,
833 + "task_acc": 1.0,
834 + "control_acc": 0.10416666666666667,
835 + "selectivity": 0.8958333333333334
836 + },
837 + {
838 + "layer": 10,
839 + "seed": 3,
840 + "task_acc": 1.0,
841 + "control_acc": 0.4166666666666667,
842 + "selectivity": 0.5833333333333333
843 + },
844 + {
845 + "layer": 10,
846 + "seed": 4,
847 + "task_acc": 1.0,
848 + "control_acc": 0.3958333333333333,
849 + "selectivity": 0.6041666666666667
850 + },
851 + {
852 + "layer": 11,
853 + "seed": 0,
854 + "task_acc": 1.0,
855 + "control_acc": 0.10416666666666667,
856 + "selectivity": 0.8958333333333334
857 + },
858 + {
859 + "layer": 11,
860 + "seed": 1,
861 + "task_acc": 1.0,
862 + "control_acc": 0.10416666666666667,
863 + "selectivity": 0.8958333333333334
864 + },
865 + {
866 + "layer": 11,
867 + "seed": 2,
868 + "task_acc": 1.0,
869 + "control_acc": 0.75,
870 + "selectivity": 0.25
871 + },
872 + {
873 + "layer": 11,
874 + "seed": 3,
875 + "task_acc": 1.0,
876 + "control_acc": 0.16666666666666666,
877 + "selectivity": 0.8333333333333334
878 + },
879 + {
880 + "layer": 11,
881 + "seed": 4,
882 + "task_acc": 1.0,
883 + "control_acc": 0.14583333333333334,
884 + "selectivity": 0.8541666666666666
885 + },
886 + {
887 + "layer": 12,
888 + "seed": 0,
889 + "task_acc": 1.0,
890 + "control_acc": 0.2916666666666667,
891 + "selectivity": 0.7083333333333333
892 + },
893 + {
894 + "layer": 12,
895 + "seed": 1,
896 + "task_acc": 1.0,
897 + "control_acc": 0.2916666666666667,
898 + "selectivity": 0.7083333333333333
899 + },
900 + {
901 + "layer": 12,
902 + "seed": 2,
903 + "task_acc": 1.0,
904 + "control_acc": 0.5625,
905 + "selectivity": 0.4375
906 + },
907 + {
908 + "layer": 12,
909 + "seed": 3,
910 + "task_acc": 1.0,
911 + "control_acc": 0.4791666666666667,
912 + "selectivity": 0.5208333333333333
913 + },
914 + {
915 + "layer": 12,
916 + "seed": 4,
917 + "task_acc": 1.0,
918 + "control_acc": 0.14583333333333334,
919 + "selectivity": 0.8541666666666666
920 + },
921 + {
922 + "layer": 13,
923 + "seed": 0,
924 + "task_acc": 1.0,
925 + "control_acc": 0.7916666666666666,
926 + "selectivity": 0.20833333333333337
927 + },
928 + {
929 + "layer": 13,
930 + "seed": 1,
931 + "task_acc": 1.0,
932 + "control_acc": 0.5625,
933 + "selectivity": 0.4375
934 + },
935 + {
936 + "layer": 13,
937 + "seed": 2,
938 + "task_acc": 1.0,
939 + "control_acc": 0.6458333333333334,
940 + "selectivity": 0.35416666666666663
941 + },
942 + {
943 + "layer": 13,
944 + "seed": 3,
945 + "task_acc": 1.0,
946 + "control_acc": 0.2708333333333333,
947 + "selectivity": 0.7291666666666667
948 + },
949 + {
950 + "layer": 13,
951 + "seed": 4,
952 + "task_acc": 1.0,
953 + "control_acc": 0.3541666666666667,
954 + "selectivity": 0.6458333333333333
955 + },
956 + {
957 + "layer": 14,
958 + "seed": 0,
959 + "task_acc": 1.0,
960 + "control_acc": 0.8125,
961 + "selectivity": 0.1875
962 + },
963 + {
964 + "layer": 14,
965 + "seed": 1,
966 + "task_acc": 1.0,
967 + "control_acc": 0.5,
968 + "selectivity": 0.5
969 + },
970 + {
971 + "layer": 14,
972 + "seed": 2,
973 + "task_acc": 1.0,
974 + "control_acc": 0.75,
975 + "selectivity": 0.25
976 + },
977 + {
978 + "layer": 14,
979 + "seed": 3,
980 + "task_acc": 1.0,
981 + "control_acc": 0.14583333333333334,
982 + "selectivity": 0.8541666666666666
983 + },
984 + {
985 + "layer": 14,
986 + "seed": 4,
987 + "task_acc": 1.0,
988 + "control_acc": 0.4166666666666667,
989 + "selectivity": 0.5833333333333333
990 + },
991 + {
992 + "layer": 15,
993 + "seed": 0,
994 + "task_acc": 1.0,
995 + "control_acc": 0.8333333333333334,
996 + "selectivity": 0.16666666666666663
997 + },
998 + {
999 + "layer": 15,
1000 + "seed": 1,
1001 + "task_acc": 1.0,
1002 + "control_acc": 0.16666666666666666,
1003 + "selectivity": 0.8333333333333334
1004 + },
1005 + {
1006 + "layer": 15,
1007 + "seed": 2,
1008 + "task_acc": 1.0,
1009 + "control_acc": 0.5625,
1010 + "selectivity": 0.4375
1011 + },
1012 + {
1013 + "layer": 15,
1014 + "seed": 3,
1015 + "task_acc": 1.0,
1016 + "control_acc": 0.4375,
1017 + "selectivity": 0.5625
1018 + },
1019 + {
1020 + "layer": 15,
1021 + "seed": 4,
1022 + "task_acc": 1.0,
1023 + "control_acc": 0.25,
1024 + "selectivity": 0.75
1025 + },
1026 + {
1027 + "layer": 16,
1028 + "seed": 0,
1029 + "task_acc": 1.0,
1030 + "control_acc": 0.1875,
1031 + "selectivity": 0.8125
1032 + },
1033 + {
1034 + "layer": 16,
1035 + "seed": 1,
1036 + "task_acc": 1.0,
1037 + "control_acc": 0.7083333333333334,
1038 + "selectivity": 0.29166666666666663
1039 + },
1040 + {
1041 + "layer": 16,
1042 + "seed": 2,
1043 + "task_acc": 1.0,
1044 + "control_acc": 0.2708333333333333,
1045 + "selectivity": 0.7291666666666667
1046 + },
1047 + {
1048 + "layer": 16,
1049 + "seed": 3,
1050 + "task_acc": 1.0,
1051 + "control_acc": 0.10416666666666667,
1052 + "selectivity": 0.8958333333333334
1053 + },
1054 + {
1055 + "layer": 16,
1056 + "seed": 4,
1057 + "task_acc": 1.0,
1058 + "control_acc": 0.3333333333333333,
1059 + "selectivity": 0.6666666666666667
1060 + },
1061 + {
1062 + "layer": 17,
1063 + "seed": 0,
1064 + "task_acc": 1.0,
1065 + "control_acc": 0.20833333333333334,
1066 + "selectivity": 0.7916666666666666
1067 + },
1068 + {
1069 + "layer": 17,
1070 + "seed": 1,
1071 + "task_acc": 1.0,
1072 + "control_acc": 0.7291666666666666,
1073 + "selectivity": 0.27083333333333337
1074 + },
1075 + {
1076 + "layer": 17,
1077 + "seed": 2,
1078 + "task_acc": 1.0,
1079 + "control_acc": 0.3958333333333333,
1080 + "selectivity": 0.6041666666666667
1081 + },
1082 + {
1083 + "layer": 17,
1084 + "seed": 3,
1085 + "task_acc": 1.0,
1086 + "control_acc": 0.25,
1087 + "selectivity": 0.75
1088 + },
1089 + {
1090 + "layer": 17,
1091 + "seed": 4,
1092 + "task_acc": 1.0,
1093 + "control_acc": 0.3333333333333333,
1094 + "selectivity": 0.6666666666666667
1095 + },
1096 + {
1097 + "layer": 18,
1098 + "seed": 0,
1099 + "task_acc": 1.0,
1100 + "control_acc": 0.0625,
1101 + "selectivity": 0.9375
1102 + },
1103 + {
1104 + "layer": 18,
1105 + "seed": 1,
1106 + "task_acc": 1.0,
1107 + "control_acc": 0.3125,
1108 + "selectivity": 0.6875
1109 + },
1110 + {
1111 + "layer": 18,
1112 + "seed": 2,
1113 + "task_acc": 1.0,
1114 + "control_acc": 0.3125,
1115 + "selectivity": 0.6875
1116 + },
1117 + {
1118 + "layer": 18,
1119 + "seed": 3,
1120 + "task_acc": 1.0,
1121 + "control_acc": 0.8958333333333334,
1122 + "selectivity": 0.10416666666666663
1123 + },
1124 + {
1125 + "layer": 18,
1126 + "seed": 4,
1127 + "task_acc": 1.0,
1128 + "control_acc": 0.4583333333333333,
1129 + "selectivity": 0.5416666666666667
1130 + },
1131 + {
1132 + "layer": 19,
1133 + "seed": 0,
1134 + "task_acc": 1.0,
1135 + "control_acc": 0.875,
1136 + "selectivity": 0.125
1137 + },
1138 + {
1139 + "layer": 19,
1140 + "seed": 1,
1141 + "task_acc": 1.0,
1142 + "control_acc": 0.3125,
1143 + "selectivity": 0.6875
1144 + },
1145 + {
1146 + "layer": 19,
1147 + "seed": 2,
1148 + "task_acc": 1.0,
1149 + "control_acc": 0.25,
1150 + "selectivity": 0.75
1151 + },
1152 + {
1153 + "layer": 19,
1154 + "seed": 3,
1155 + "task_acc": 1.0,
1156 + "control_acc": 0.9166666666666666,
1157 + "selectivity": 0.08333333333333337
1158 + },
1159 + {
1160 + "layer": 19,
1161 + "seed": 4,
1162 + "task_acc": 1.0,
1163 + "control_acc": 0.375,
1164 + "selectivity": 0.625
1165 + },
1166 + {
1167 + "layer": 20,
1168 + "seed": 0,
1169 + "task_acc": 1.0,
1170 + "control_acc": 0.08333333333333333,
1171 + "selectivity": 0.9166666666666666
1172 + },
1173 + {
1174 + "layer": 20,
1175 + "seed": 1,
1176 + "task_acc": 1.0,
1177 + "control_acc": 0.20833333333333334,
1178 + "selectivity": 0.7916666666666666
1179 + },
1180 + {
1181 + "layer": 20,
1182 + "seed": 2,
1183 + "task_acc": 1.0,
1184 + "control_acc": 0.3125,
1185 + "selectivity": 0.6875
1186 + },
1187 + {
1188 + "layer": 20,
1189 + "seed": 3,
1190 + "task_acc": 1.0,
1191 + "control_acc": 0.9166666666666666,
1192 + "selectivity": 0.08333333333333337
1193 + },
1194 + {
1195 + "layer": 20,
1196 + "seed": 4,
1197 + "task_acc": 1.0,
1198 + "control_acc": 0.3541666666666667,
1199 + "selectivity": 0.6458333333333333
1200 + },
1201 + {
1202 + "layer": 21,
1203 + "seed": 0,
1204 + "task_acc": 1.0,
1205 + "control_acc": 0.8541666666666666,
1206 + "selectivity": 0.14583333333333337
1207 + },
1208 + {
1209 + "layer": 21,
1210 + "seed": 1,
1211 + "task_acc": 1.0,
1212 + "control_acc": 0.14583333333333334,
1213 + "selectivity": 0.8541666666666666
1214 + },
1215 + {
1216 + "layer": 21,
1217 + "seed": 2,
1218 + "task_acc": 1.0,
1219 + "control_acc": 0.8333333333333334,
1220 + "selectivity": 0.16666666666666663
1221 + },
1222 + {
1223 + "layer": 21,
1224 + "seed": 3,
1225 + "task_acc": 1.0,
1226 + "control_acc": 0.20833333333333334,
1227 + "selectivity": 0.7916666666666666
1228 + },
1229 + {
1230 + "layer": 21,
1231 + "seed": 4,
1232 + "task_acc": 1.0,
1233 + "control_acc": 0.14583333333333334,
1234 + "selectivity": 0.8541666666666666
1235 + },
1236 + {
1237 + "layer": 22,
1238 + "seed": 0,
1239 + "task_acc": 1.0,
1240 + "control_acc": 0.0625,
1241 + "selectivity": 0.9375
1242 + },
1243 + {
1244 + "layer": 22,
1245 + "seed": 1,
1246 + "task_acc": 1.0,
1247 + "control_acc": 0.8125,
1248 + "selectivity": 0.1875
1249 + },
1250 + {
1251 + "layer": 22,
1252 + "seed": 2,
1253 + "task_acc": 1.0,
1254 + "control_acc": 0.3333333333333333,
1255 + "selectivity": 0.6666666666666667
1256 + },
1257 + {
1258 + "layer": 22,
1259 + "seed": 3,
1260 + "task_acc": 1.0,
1261 + "control_acc": 0.9166666666666666,
1262 + "selectivity": 0.08333333333333337
1263 + },
1264 + {
1265 + "layer": 22,
1266 + "seed": 4,
1267 + "task_acc": 1.0,
1268 + "control_acc": 0.2916666666666667,
1269 + "selectivity": 0.7083333333333333
1270 + },
1271 + {
1272 + "layer": 23,
1273 + "seed": 0,
1274 + "task_acc": 1.0,
1275 + "control_acc": 0.041666666666666664,
1276 + "selectivity": 0.9583333333333334
1277 + },
1278 + {
1279 + "layer": 23,
1280 + "seed": 1,
1281 + "task_acc": 1.0,
1282 + "control_acc": 0.6875,
1283 + "selectivity": 0.3125
1284 + },
1285 + {
1286 + "layer": 23,
1287 + "seed": 2,
1288 + "task_acc": 1.0,
1289 + "control_acc": 0.75,
1290 + "selectivity": 0.25
1291 + },
1292 + {
1293 + "layer": 23,
1294 + "seed": 3,
1295 + "task_acc": 1.0,
1296 + "control_acc": 0.9166666666666666,
1297 + "selectivity": 0.08333333333333337
1298 + },
1299 + {
1300 + "layer": 23,
1301 + "seed": 4,
1302 + "task_acc": 1.0,
1303 + "control_acc": 0.6875,
1304 + "selectivity": 0.3125
1305 + },
1306 + {
1307 + "layer": 24,
1308 + "seed": 0,
1309 + "task_acc": 1.0,
1310 + "control_acc": 0.875,
1311 + "selectivity": 0.125
1312 + },
1313 + {
1314 + "layer": 24,
1315 + "seed": 1,
1316 + "task_acc": 1.0,
1317 + "control_acc": 0.4791666666666667,
1318 + "selectivity": 0.5208333333333333
1319 + },
1320 + {
1321 + "layer": 24,
1322 + "seed": 2,
1323 + "task_acc": 1.0,
1324 + "control_acc": 0.8541666666666666,
1325 + "selectivity": 0.14583333333333337
1326 + },
1327 + {
1328 + "layer": 24,
1329 + "seed": 3,
1330 + "task_acc": 1.0,
1331 + "control_acc": 0.9375,
1332 + "selectivity": 0.0625
1333 + },
1334 + {
1335 + "layer": 24,
1336 + "seed": 4,
1337 + "task_acc": 1.0,
1338 + "control_acc": 0.875,
1339 + "selectivity": 0.125
1340 + },
1341 + {
1342 + "layer": 25,
1343 + "seed": 0,
1344 + "task_acc": 1.0,
1345 + "control_acc": 0.8333333333333334,
1346 + "selectivity": 0.16666666666666663
1347 + },
1348 + {
1349 + "layer": 25,
1350 + "seed": 1,
1351 + "task_acc": 1.0,
1352 + "control_acc": 0.875,
1353 + "selectivity": 0.125
1354 + },
1355 + {
1356 + "layer": 25,
1357 + "seed": 2,
1358 + "task_acc": 1.0,
1359 + "control_acc": 0.7083333333333334,
1360 + "selectivity": 0.29166666666666663
1361 + },
1362 + {
1363 + "layer": 25,
1364 + "seed": 3,
1365 + "task_acc": 1.0,
1366 + "control_acc": 0.9583333333333334,
1367 + "selectivity": 0.04166666666666663
1368 + },
1369 + {
1370 + "layer": 25,
1371 + "seed": 4,
1372 + "task_acc": 1.0,
1373 + "control_acc": 0.875,
1374 + "selectivity": 0.125
1375 + },
1376 + {
1377 + "layer": 26,
1378 + "seed": 0,
1379 + "task_acc": 1.0,
1380 + "control_acc": 0.8125,
1381 + "selectivity": 0.1875
1382 + },
1383 + {
1384 + "layer": 26,
1385 + "seed": 1,
1386 + "task_acc": 1.0,
1387 + "control_acc": 0.7916666666666666,
1388 + "selectivity": 0.20833333333333337
1389 + },
1390 + {
1391 + "layer": 26,
1392 + "seed": 2,
1393 + "task_acc": 1.0,
1394 + "control_acc": 0.8333333333333334,
1395 + "selectivity": 0.16666666666666663
1396 + },
1397 + {
1398 + "layer": 26,
1399 + "seed": 3,
1400 + "task_acc": 1.0,
1401 + "control_acc": 0.9791666666666666,
1402 + "selectivity": 0.02083333333333337
1403 + },
1404 + {
1405 + "layer": 26,
1406 + "seed": 4,
1407 + "task_acc": 1.0,
1408 + "control_acc": 0.8125,
1409 + "selectivity": 0.1875
1410 + },
1411 + {
1412 + "layer": 27,
1413 + "seed": 0,
1414 + "task_acc": 1.0,
1415 + "control_acc": 0.16666666666666666,
1416 + "selectivity": 0.8333333333333334
1417 + },
1418 + {
1419 + "layer": 27,
1420 + "seed": 1,
1421 + "task_acc": 1.0,
1422 + "control_acc": 0.8958333333333334,
1423 + "selectivity": 0.10416666666666663
1424 + },
1425 + {
1426 + "layer": 27,
1427 + "seed": 2,
1428 + "task_acc": 1.0,
1429 + "control_acc": 0.1875,
1430 + "selectivity": 0.8125
1431 + },
1432 + {
1433 + "layer": 27,
1434 + "seed": 3,
1435 + "task_acc": 1.0,
1436 + "control_acc": 0.9375,
1437 + "selectivity": 0.0625
1438 + },
1439 + {
1440 + "layer": 27,
1441 + "seed": 4,
1442 + "task_acc": 1.0,
1443 + "control_acc": 0.8125,
1444 + "selectivity": 0.1875
1445 + }
1446 + ]
1447 + },
1448 + "lang_id_B": {
1449 + "layers": [
1450 + {
1451 + "layer": 0,
1452 + "task_acc_mean": 1.0,
1453 + "task_acc_seed_sd": 0.0,
1454 + "selectivity_mean": 0.4458333333333333,
1455 + "selectivity_ci": [
1456 + 0.10833333333333331,
1457 + 0.7833333333333334
1458 + ],
1459 + "p_boot": 0.0001,
1460 + "fdr_significant": true
1461 + },
1462 + {
1463 + "layer": 1,
1464 + "task_acc_mean": 1.0,
1465 + "task_acc_seed_sd": 0.0,
1466 + "selectivity_mean": 0.5833333333333334,
1467 + "selectivity_ci": [
1468 + 0.2833333333333333,
1469 + 0.8833333333333334
1470 + ],
1471 + "p_boot": 0.0001,
1472 + "fdr_significant": true
1473 + },
1474 + {
1475 + "layer": 2,
1476 + "task_acc_mean": 1.0,
1477 + "task_acc_seed_sd": 0.0,
1478 + "selectivity_mean": 0.3916666666666666,
1479 + "selectivity_ci": [
1480 + 0.04583333333333332,
1481 + 0.7374999999999999
1482 + ],
1483 + "p_boot": 0.0001,
1484 + "fdr_significant": true
1485 + },
1486 + {
1487 + "layer": 3,
1488 + "task_acc_mean": 1.0,
1489 + "task_acc_seed_sd": 0.0,
1490 + "selectivity_mean": 0.55,
1491 + "selectivity_ci": [
1492 + 0.20833333333333331,
1493 + 0.8916666666666666
1494 + ],
1495 + "p_boot": 0.0001,
1496 + "fdr_significant": true
1497 + },
1498 + {
1499 + "layer": 4,
1500 + "task_acc_mean": 1.0,
1501 + "task_acc_seed_sd": 0.0,
1502 + "selectivity_mean": 0.6208333333333333,
1503 + "selectivity_ci": [
1504 + 0.3583333333333333,
1505 + 0.8666666666666666
1506 + ],
1507 + "p_boot": 0.0001,
1508 + "fdr_significant": true
1509 + },
1510 + {
1511 + "layer": 5,
1512 + "task_acc_mean": 1.0,
1513 + "task_acc_seed_sd": 0.0,
1514 + "selectivity_mean": 0.6083333333333333,
1515 + "selectivity_ci": [
1516 + 0.3291666666666667,
1517 + 0.85
1518 + ],
1519 + "p_boot": 0.0001,
1520 + "fdr_significant": true
1521 + },
1522 + {
1523 + "layer": 6,
1524 + "task_acc_mean": 1.0,
1525 + "task_acc_seed_sd": 0.0,
1526 + "selectivity_mean": 0.2791666666666667,
1527 + "selectivity_ci": [
1528 + 0.03750000000000002,
1529 + 0.5791666666666667
1530 + ],
1531 + "p_boot": 0.0001,
1532 + "fdr_significant": true
1533 + },
1534 + {
1535 + "layer": 7,
1536 + "task_acc_mean": 1.0,
1537 + "task_acc_seed_sd": 0.0,
1538 + "selectivity_mean": 0.4250000000000001,
1539 + "selectivity_ci": [
1540 + 0.1416666666666667,
1541 + 0.7208333333333334
1542 + ],
1543 + "p_boot": 0.0001,
1544 + "fdr_significant": true
1545 + },
1546 + {
1547 + "layer": 8,
1548 + "task_acc_mean": 1.0,
1549 + "task_acc_seed_sd": 0.0,
1550 + "selectivity_mean": 0.48749999999999993,
1551 + "selectivity_ci": [
1552 + 0.21666666666666662,
1553 + 0.7583333333333333
1554 + ],
1555 + "p_boot": 0.0001,
1556 + "fdr_significant": true
1557 + },
1558 + {
1559 + "layer": 9,
1560 + "task_acc_mean": 1.0,
1561 + "task_acc_seed_sd": 0.0,
1562 + "selectivity_mean": 0.4333333333333333,
1563 + "selectivity_ci": [
1564 + 0.1958333333333333,
1565 + 0.6874999999999999
1566 + ],
1567 + "p_boot": 0.0001,
1568 + "fdr_significant": true
1569 + },
1570 + {
1571 + "layer": 10,
1572 + "task_acc_mean": 1.0,
1573 + "task_acc_seed_sd": 0.0,
1574 + "selectivity_mean": 0.4583333333333333,
1575 + "selectivity_ci": [
1576 + 0.20416666666666666,
1577 + 0.7249999999999999
1578 + ],
1579 + "p_boot": 0.0001,
1580 + "fdr_significant": true
1581 + },
1582 + {
1583 + "layer": 11,
1584 + "task_acc_mean": 1.0,
1585 + "task_acc_seed_sd": 0.0,
1586 + "selectivity_mean": 0.27083333333333337,
1587 + "selectivity_ci": [
1588 + 0.07916666666666668,
1589 + 0.5583333333333333
1590 + ],
1591 + "p_boot": 0.0001,
1592 + "fdr_significant": true
1593 + },
1594 + {
1595 + "layer": 12,
1596 + "task_acc_mean": 1.0,
1597 + "task_acc_seed_sd": 0.0,
1598 + "selectivity_mean": 0.5625,
1599 + "selectivity_ci": [
1600 + 0.2625,
1601 + 0.8583333333333334
1602 + ],
1603 + "p_boot": 0.0001,
1604 + "fdr_significant": true
1605 + },
1606 + {
1607 + "layer": 13,
1608 + "task_acc_mean": 1.0,
1609 + "task_acc_seed_sd": 0.0,
1610 + "selectivity_mean": 0.4583333333333333,
1611 + "selectivity_ci": [
1612 + 0.15416666666666665,
1613 + 0.7625
1614 + ],
1615 + "p_boot": 0.0001,
1616 + "fdr_significant": true
1617 + },
1618 + {
1619 + "layer": 14,
1620 + "task_acc_mean": 1.0,
1621 + "task_acc_seed_sd": 0.0,
1622 + "selectivity_mean": 0.5458333333333333,
1623 + "selectivity_ci": [
1624 + 0.3041666666666666,
1625 + 0.8041666666666666
1626 + ],
1627 + "p_boot": 0.0001,
1628 + "fdr_significant": true
1629 + },
1630 + {
1631 + "layer": 15,
1632 + "task_acc_mean": 1.0,
1633 + "task_acc_seed_sd": 0.0,
1634 + "selectivity_mean": 0.4333333333333334,
1635 + "selectivity_ci": [
1636 + 0.11666666666666667,
1637 + 0.7500000000000001
1638 + ],
1639 + "p_boot": 0.0001,
1640 + "fdr_significant": true
1641 + },
1642 + {
1643 + "layer": 16,
1644 + "task_acc_mean": 1.0,
1645 + "task_acc_seed_sd": 0.0,
1646 + "selectivity_mean": 0.6625,
1647 + "selectivity_ci": [
1648 + 0.375,
1649 + 0.8625
1650 + ],
1651 + "p_boot": 0.0001,
1652 + "fdr_significant": true
1653 + },
1654 + {
1655 + "layer": 17,
1656 + "task_acc_mean": 1.0,
1657 + "task_acc_seed_sd": 0.0,
1658 + "selectivity_mean": 0.7125,
1659 + "selectivity_ci": [
1660 + 0.4039583333333335,
1661 + 0.9041666666666666
1662 + ],
1663 + "p_boot": 0.0001,
1664 + "fdr_significant": true
1665 + },
1666 + {
1667 + "layer": 18,
1668 + "task_acc_mean": 1.0,
1669 + "task_acc_seed_sd": 0.0,
1670 + "selectivity_mean": 0.7625,
1671 + "selectivity_ci": [
1672 + 0.4833333333333334,
1673 + 0.9083333333333334
1674 + ],
1675 + "p_boot": 0.0001,
1676 + "fdr_significant": true
1677 + },
1678 + {
1679 + "layer": 19,
1680 + "task_acc_mean": 1.0,
1681 + "task_acc_seed_sd": 0.0,
1682 + "selectivity_mean": 0.4708333333333334,
1683 + "selectivity_ci": [
1684 + 0.19583333333333336,
1685 + 0.7458333333333333
1686 + ],
1687 + "p_boot": 0.0001,
1688 + "fdr_significant": true
1689 + },
1690 + {
1691 + "layer": 20,
1692 + "task_acc_mean": 1.0,
1693 + "task_acc_seed_sd": 0.0,
1694 + "selectivity_mean": 0.2791666666666667,
1695 + "selectivity_ci": [
1696 + 0.07916666666666668,
1697 + 0.5416666666666667
1698 + ],
1699 + "p_boot": 0.0001,
1700 + "fdr_significant": true
1701 + },
1702 + {
1703 + "layer": 21,
1704 + "task_acc_mean": 1.0,
1705 + "task_acc_seed_sd": 0.0,
1706 + "selectivity_mean": 0.4333333333333334,
1707 + "selectivity_ci": [
1708 + 0.11250000000000002,
1709 + 0.7583333333333334
1710 + ],
1711 + "p_boot": 0.0001,
1712 + "fdr_significant": true
1713 + },
1714 + {
1715 + "layer": 22,
1716 + "task_acc_mean": 1.0,
1717 + "task_acc_seed_sd": 0.0,
1718 + "selectivity_mean": 0.6083333333333334,
1719 + "selectivity_ci": [
1720 + 0.3541666666666667,
1721 + 0.8416666666666666
1722 + ],
1723 + "p_boot": 0.0001,
1724 + "fdr_significant": true
1725 + },
1726 + {
1727 + "layer": 23,
1728 + "task_acc_mean": 1.0,
1729 + "task_acc_seed_sd": 0.0,
1730 + "selectivity_mean": 0.5041666666666667,
1731 + "selectivity_ci": [
1732 + 0.2375,
1733 + 0.7708333333333333
1734 + ],
1735 + "p_boot": 0.0001,
1736 + "fdr_significant": true
1737 + },
1738 + {
1739 + "layer": 24,
1740 + "task_acc_mean": 1.0,
1741 + "task_acc_seed_sd": 0.0,
1742 + "selectivity_mean": 0.2458333333333333,
1743 + "selectivity_ci": [
1744 + 0.054166666666666675,
1745 + 0.5375
1746 + ],
1747 + "p_boot": 0.0001,
1748 + "fdr_significant": true
1749 + },
1750 + {
1751 + "layer": 25,
1752 + "task_acc_mean": 1.0,
1753 + "task_acc_seed_sd": 0.0,
1754 + "selectivity_mean": 0.4208333333333333,
1755 + "selectivity_ci": [
1756 + 0.12916666666666665,
1757 + 0.7125
1758 + ],
1759 + "p_boot": 0.0001,
1760 + "fdr_significant": true
1761 + },
1762 + {
1763 + "layer": 26,
1764 + "task_acc_mean": 1.0,
1765 + "task_acc_seed_sd": 0.0,
1766 + "selectivity_mean": 0.5083333333333333,
1767 + "selectivity_ci": [
1768 + 0.2541666666666666,
1769 + 0.7666666666666666
1770 + ],
1771 + "p_boot": 0.0001,
1772 + "fdr_significant": true
1773 + },
1774 + {
1775 + "layer": 27,
1776 + "task_acc_mean": 1.0,
1777 + "task_acc_seed_sd": 0.0,
1778 + "selectivity_mean": 0.3124999999999999,
1779 + "selectivity_ci": [
1780 + 0.09166666666666663,
1781 + 0.6333333333333333
1782 + ],
1783 + "p_boot": 0.0001,
1784 + "fdr_significant": true
1785 + }
1786 + ],
1787 + "replication_rate_topk": {
1788 + "k": 5,
1789 + "point": 1.0,
1790 + "ci": [
1791 + 1.0,
1792 + 1.0
1793 + ]
1794 + },
1795 + "n_fdr_significant": 28,
1796 + "cells": [
1797 + {
1798 + "layer": 0,
1799 + "seed": 0,
1800 + "task_acc": 1.0,
1801 + "control_acc": 0.9583333333333334,
1802 + "selectivity": 0.04166666666666663
1803 + },
1804 + {
1805 + "layer": 0,
1806 + "seed": 1,
1807 + "task_acc": 1.0,
1808 + "control_acc": 0.75,
1809 + "selectivity": 0.25
1810 + },
1811 + {
1812 + "layer": 0,
1813 + "seed": 2,
1814 + "task_acc": 1.0,
1815 + "control_acc": 0.8958333333333334,
1816 + "selectivity": 0.10416666666666663
1817 + },
1818 + {
1819 + "layer": 0,
1820 + "seed": 3,
1821 + "task_acc": 1.0,
1822 + "control_acc": 0.125,
1823 + "selectivity": 0.875
1824 + },
1825 + {
1826 + "layer": 0,
1827 + "seed": 4,
1828 + "task_acc": 1.0,
1829 + "control_acc": 0.041666666666666664,
1830 + "selectivity": 0.9583333333333334
1831 + },
1832 + {
1833 + "layer": 1,
1834 + "seed": 0,
1835 + "task_acc": 1.0,
1836 + "control_acc": 0.10416666666666667,
1837 + "selectivity": 0.8958333333333334
1838 + },
1839 + {
1840 + "layer": 1,
1841 + "seed": 1,
1842 + "task_acc": 1.0,
1843 + "control_acc": 0.875,
1844 + "selectivity": 0.125
1845 + },
1846 + {
1847 + "layer": 1,
1848 + "seed": 2,
1849 + "task_acc": 1.0,
1850 + "control_acc": 0.16666666666666666,
1851 + "selectivity": 0.8333333333333334
1852 + },
1853 + {
1854 + "layer": 1,
1855 + "seed": 3,
1856 + "task_acc": 1.0,
1857 + "control_acc": 0.8333333333333334,
1858 + "selectivity": 0.16666666666666663
1859 + },
1860 + {
1861 + "layer": 1,
1862 + "seed": 4,
1863 + "task_acc": 1.0,
1864 + "control_acc": 0.10416666666666667,
1865 + "selectivity": 0.8958333333333334
1866 + },
1867 + {
1868 + "layer": 2,
1869 + "seed": 0,
1870 + "task_acc": 1.0,
1871 + "control_acc": 0.9583333333333334,
1872 + "selectivity": 0.04166666666666663
1873 + },
1874 + {
1875 + "layer": 2,
1876 + "seed": 1,
1877 + "task_acc": 1.0,
1878 + "control_acc": 0.125,
1879 + "selectivity": 0.875
1880 + },
1881 + {
1882 + "layer": 2,
1883 + "seed": 2,
1884 + "task_acc": 1.0,
1885 + "control_acc": 0.8958333333333334,
1886 + "selectivity": 0.10416666666666663
1887 + },
1888 + {
1889 + "layer": 2,
1890 + "seed": 3,
1891 + "task_acc": 1.0,
1892 + "control_acc": 0.9791666666666666,
1893 + "selectivity": 0.02083333333333337
1894 + },
1895 + {
1896 + "layer": 2,
1897 + "seed": 4,
1898 + "task_acc": 1.0,
1899 + "control_acc": 0.08333333333333333,
1900 + "selectivity": 0.9166666666666666
1901 + },
1902 + {
1903 + "layer": 3,
1904 + "seed": 0,
1905 + "task_acc": 1.0,
1906 + "control_acc": 0.9583333333333334,
1907 + "selectivity": 0.04166666666666663
1908 + },
1909 + {
1910 + "layer": 3,
1911 + "seed": 1,
1912 + "task_acc": 1.0,
1913 + "control_acc": 0.0625,
1914 + "selectivity": 0.9375
1915 + },
1916 + {
1917 + "layer": 3,
1918 + "seed": 2,
1919 + "task_acc": 1.0,
1920 + "control_acc": 0.9166666666666666,
1921 + "selectivity": 0.08333333333333337
1922 + },
1923 + {
1924 + "layer": 3,
1925 + "seed": 3,
1926 + "task_acc": 1.0,
1927 + "control_acc": 0.20833333333333334,
1928 + "selectivity": 0.7916666666666666
1929 + },
1930 + {
1931 + "layer": 3,
1932 + "seed": 4,
1933 + "task_acc": 1.0,
1934 + "control_acc": 0.10416666666666667,
1935 + "selectivity": 0.8958333333333334
1936 + },
1937 + {
1938 + "layer": 4,
1939 + "seed": 0,
1940 + "task_acc": 1.0,
1941 + "control_acc": 0.10416666666666667,
1942 + "selectivity": 0.8958333333333334
1943 + },
1944 + {
1945 + "layer": 4,
1946 + "seed": 1,
1947 + "task_acc": 1.0,
1948 + "control_acc": 0.6458333333333334,
1949 + "selectivity": 0.35416666666666663
1950 + },
1951 + {
1952 + "layer": 4,
1953 + "seed": 2,
1954 + "task_acc": 1.0,
1955 + "control_acc": 0.125,
1956 + "selectivity": 0.875
1957 + },
1958 + {
1959 + "layer": 4,
1960 + "seed": 3,
1961 + "task_acc": 1.0,
1962 + "control_acc": 0.20833333333333334,
1963 + "selectivity": 0.7916666666666666
1964 + },
1965 + {
1966 + "layer": 4,
1967 + "seed": 4,
1968 + "task_acc": 1.0,
1969 + "control_acc": 0.8125,
1970 + "selectivity": 0.1875
1971 + },
1972 + {
1973 + "layer": 5,
1974 + "seed": 0,
1975 + "task_acc": 1.0,
1976 + "control_acc": 0.10416666666666667,
1977 + "selectivity": 0.8958333333333334
1978 + },
1979 + {
1980 + "layer": 5,
1981 + "seed": 1,
1982 + "task_acc": 1.0,
1983 + "control_acc": 0.8541666666666666,
1984 + "selectivity": 0.14583333333333337
1985 + },
1986 + {
1987 + "layer": 5,
1988 + "seed": 2,
1989 + "task_acc": 1.0,
1990 + "control_acc": 0.16666666666666666,
1991 + "selectivity": 0.8333333333333334
1992 + },
1993 + {
1994 + "layer": 5,
1995 + "seed": 3,
1996 + "task_acc": 1.0,
1997 + "control_acc": 0.20833333333333334,
1998 + "selectivity": 0.7916666666666666
1999 + },
2000 + {
2001 + "layer": 5,
2002 + "seed": 4,
2003 + "task_acc": 1.0,
2004 + "control_acc": 0.625,
2005 + "selectivity": 0.375
2006 + },
2007 + {
2008 + "layer": 6,
2009 + "seed": 0,
2010 + "task_acc": 1.0,
2011 + "control_acc": 0.9791666666666666,
2012 + "selectivity": 0.02083333333333337
2013 + },
2014 + {
2015 + "layer": 6,
2016 + "seed": 1,
2017 + "task_acc": 1.0,
2018 + "control_acc": 0.9791666666666666,
2019 + "selectivity": 0.02083333333333337
2020 + },
2021 + {
2022 + "layer": 6,
2023 + "seed": 2,
2024 + "task_acc": 1.0,
2025 + "control_acc": 0.1875,
2026 + "selectivity": 0.8125
2027 + },
2028 + {
2029 + "layer": 6,
2030 + "seed": 3,
2031 + "task_acc": 1.0,
2032 + "control_acc": 0.8958333333333334,
2033 + "selectivity": 0.10416666666666663
2034 + },
2035 + {
2036 + "layer": 6,
2037 + "seed": 4,
2038 + "task_acc": 1.0,
2039 + "control_acc": 0.5625,
2040 + "selectivity": 0.4375
2041 + },
2042 + {
2043 + "layer": 7,
2044 + "seed": 0,
2045 + "task_acc": 1.0,
2046 + "control_acc": 0.9791666666666666,
2047 + "selectivity": 0.02083333333333337
2048 + },
2049 + {
2050 + "layer": 7,
2051 + "seed": 1,
2052 + "task_acc": 1.0,
2053 + "control_acc": 0.75,
2054 + "selectivity": 0.25
2055 + },
2056 + {
2057 + "layer": 7,
2058 + "seed": 2,
2059 + "task_acc": 1.0,
2060 + "control_acc": 0.125,
2061 + "selectivity": 0.875
2062 + },
2063 + {
2064 + "layer": 7,
2065 + "seed": 3,
2066 + "task_acc": 1.0,
2067 + "control_acc": 0.22916666666666666,
2068 + "selectivity": 0.7708333333333334
2069 + },
2070 + {
2071 + "layer": 7,
2072 + "seed": 4,
2073 + "task_acc": 1.0,
2074 + "control_acc": 0.7916666666666666,
2075 + "selectivity": 0.20833333333333337
2076 + },
2077 + {
2078 + "layer": 8,
2079 + "seed": 0,
2080 + "task_acc": 1.0,
2081 + "control_acc": 0.8958333333333334,
2082 + "selectivity": 0.10416666666666663
2083 + },
2084 + {
2085 + "layer": 8,
2086 + "seed": 1,
2087 + "task_acc": 1.0,
2088 + "control_acc": 0.7708333333333334,
2089 + "selectivity": 0.22916666666666663
2090 + },
2091 + {
2092 + "layer": 8,
2093 + "seed": 2,
2094 + "task_acc": 1.0,
2095 + "control_acc": 0.125,
2096 + "selectivity": 0.875
2097 + },
2098 + {
2099 + "layer": 8,
2100 + "seed": 3,
2101 + "task_acc": 1.0,
2102 + "control_acc": 0.1875,
2103 + "selectivity": 0.8125
2104 + },
2105 + {
2106 + "layer": 8,
2107 + "seed": 4,
2108 + "task_acc": 1.0,
2109 + "control_acc": 0.5833333333333334,
2110 + "selectivity": 0.41666666666666663
2111 + },
2112 + {
2113 + "layer": 9,
2114 + "seed": 0,
2115 + "task_acc": 1.0,
2116 + "control_acc": 0.9583333333333334,
2117 + "selectivity": 0.04166666666666663
2118 + },
2119 + {
2120 + "layer": 9,
2121 + "seed": 1,
2122 + "task_acc": 1.0,
2123 + "control_acc": 0.7083333333333334,
2124 + "selectivity": 0.29166666666666663
2125 + },
2126 + {
2127 + "layer": 9,
2128 + "seed": 2,
2129 + "task_acc": 1.0,
2130 + "control_acc": 0.2916666666666667,
2131 + "selectivity": 0.7083333333333333
2132 + },
2133 + {
2134 + "layer": 9,
2135 + "seed": 3,
2136 + "task_acc": 1.0,
2137 + "control_acc": 0.1875,
2138 + "selectivity": 0.8125
2139 + },
2140 + {
2141 + "layer": 9,
2142 + "seed": 4,
2143 + "task_acc": 1.0,
2144 + "control_acc": 0.6875,
2145 + "selectivity": 0.3125
2146 + },
2147 + {
2148 + "layer": 10,
2149 + "seed": 0,
2150 + "task_acc": 1.0,
2151 + "control_acc": 0.9375,
2152 + "selectivity": 0.0625
2153 + },
2154 + {
2155 + "layer": 10,
2156 + "seed": 1,
2157 + "task_acc": 1.0,
2158 + "control_acc": 0.7083333333333334,
2159 + "selectivity": 0.29166666666666663
2160 + },
2161 + {
2162 + "layer": 10,
2163 + "seed": 2,
2164 + "task_acc": 1.0,
2165 + "control_acc": 0.14583333333333334,
2166 + "selectivity": 0.8541666666666666
2167 + },
2168 + {
2169 + "layer": 10,
2170 + "seed": 3,
2171 + "task_acc": 1.0,
2172 + "control_acc": 0.22916666666666666,
2173 + "selectivity": 0.7708333333333334
2174 + },
2175 + {
2176 + "layer": 10,
2177 + "seed": 4,
2178 + "task_acc": 1.0,
2179 + "control_acc": 0.6875,
2180 + "selectivity": 0.3125
2181 + },
2182 + {
2183 + "layer": 11,
2184 + "seed": 0,
2185 + "task_acc": 1.0,
2186 + "control_acc": 0.9583333333333334,
2187 + "selectivity": 0.04166666666666663
2188 + },
2189 + {
2190 + "layer": 11,
2191 + "seed": 1,
2192 + "task_acc": 1.0,
2193 + "control_acc": 0.75,
2194 + "selectivity": 0.25
2195 + },
2196 + {
2197 + "layer": 11,
2198 + "seed": 2,
2199 + "task_acc": 1.0,
2200 + "control_acc": 0.9166666666666666,
2201 + "selectivity": 0.08333333333333337
2202 + },
2203 + {
2204 + "layer": 11,
2205 + "seed": 3,
2206 + "task_acc": 1.0,
2207 + "control_acc": 0.8541666666666666,
2208 + "selectivity": 0.14583333333333337
2209 + },
2210 + {
2211 + "layer": 11,
2212 + "seed": 4,
2213 + "task_acc": 1.0,
2214 + "control_acc": 0.16666666666666666,
2215 + "selectivity": 0.8333333333333334
2216 + },
2217 + {
2218 + "layer": 12,
2219 + "seed": 0,
2220 + "task_acc": 1.0,
2221 + "control_acc": 0.8541666666666666,
2222 + "selectivity": 0.14583333333333337
2223 + },
2224 + {
2225 + "layer": 12,
2226 + "seed": 1,
2227 + "task_acc": 1.0,
2228 + "control_acc": 0.875,
2229 + "selectivity": 0.125
2230 + },
2231 + {
2232 + "layer": 12,
2233 + "seed": 2,
2234 + "task_acc": 1.0,
2235 + "control_acc": 0.08333333333333333,
2236 + "selectivity": 0.9166666666666666
2237 + },
2238 + {
2239 + "layer": 12,
2240 + "seed": 3,
2241 + "task_acc": 1.0,
2242 + "control_acc": 0.22916666666666666,
2243 + "selectivity": 0.7708333333333334
2244 + },
2245 + {
2246 + "layer": 12,
2247 + "seed": 4,
2248 + "task_acc": 1.0,
2249 + "control_acc": 0.14583333333333334,
2250 + "selectivity": 0.8541666666666666
2251 + },
2252 + {
2253 + "layer": 13,
2254 + "seed": 0,
2255 + "task_acc": 1.0,
2256 + "control_acc": 0.875,
2257 + "selectivity": 0.125
2258 + },
2259 + {
2260 + "layer": 13,
2261 + "seed": 1,
2262 + "task_acc": 1.0,
2263 + "control_acc": 0.8125,
2264 + "selectivity": 0.1875
2265 + },
2266 + {
2267 + "layer": 13,
2268 + "seed": 2,
2269 + "task_acc": 1.0,
2270 + "control_acc": 0.10416666666666667,
2271 + "selectivity": 0.8958333333333334
2272 + },
2273 + {
2274 + "layer": 13,
2275 + "seed": 3,
2276 + "task_acc": 1.0,
2277 + "control_acc": 0.8333333333333334,
2278 + "selectivity": 0.16666666666666663
2279 + },
2280 + {
2281 + "layer": 13,
2282 + "seed": 4,
2283 + "task_acc": 1.0,
2284 + "control_acc": 0.08333333333333333,
2285 + "selectivity": 0.9166666666666666
2286 + },
2287 + {
2288 + "layer": 14,
2289 + "seed": 0,
2290 + "task_acc": 1.0,
2291 + "control_acc": 0.8958333333333334,
2292 + "selectivity": 0.10416666666666663
2293 + },
2294 + {
2295 + "layer": 14,
2296 + "seed": 1,
2297 + "task_acc": 1.0,
2298 + "control_acc": 0.5625,
2299 + "selectivity": 0.4375
2300 + },
2301 + {
2302 + "layer": 14,
2303 + "seed": 2,
2304 + "task_acc": 1.0,
2305 + "control_acc": 0.08333333333333333,
2306 + "selectivity": 0.9166666666666666
2307 + },
2308 + {
2309 + "layer": 14,
2310 + "seed": 3,
2311 + "task_acc": 1.0,
2312 + "control_acc": 0.16666666666666666,
2313 + "selectivity": 0.8333333333333334
2314 + },
2315 + {
2316 + "layer": 14,
2317 + "seed": 4,
2318 + "task_acc": 1.0,
2319 + "control_acc": 0.5625,
2320 + "selectivity": 0.4375
2321 + },
2322 + {
2323 + "layer": 15,
2324 + "seed": 0,
2325 + "task_acc": 1.0,
2326 + "control_acc": 0.125,
2327 + "selectivity": 0.875
2328 + },
2329 + {
2330 + "layer": 15,
2331 + "seed": 1,
2332 + "task_acc": 1.0,
2333 + "control_acc": 0.8541666666666666,
2334 + "selectivity": 0.14583333333333337
2335 + },
2336 + {
2337 + "layer": 15,
2338 + "seed": 2,
2339 + "task_acc": 1.0,
2340 + "control_acc": 0.10416666666666667,
2341 + "selectivity": 0.8958333333333334
2342 + },
2343 + {
2344 + "layer": 15,
2345 + "seed": 3,
2346 + "task_acc": 1.0,
2347 + "control_acc": 0.9583333333333334,
2348 + "selectivity": 0.04166666666666663
2349 + },
2350 + {
2351 + "layer": 15,
2352 + "seed": 4,
2353 + "task_acc": 1.0,
2354 + "control_acc": 0.7916666666666666,
2355 + "selectivity": 0.20833333333333337
2356 + },
2357 + {
2358 + "layer": 16,
2359 + "seed": 0,
2360 + "task_acc": 1.0,
2361 + "control_acc": 0.16666666666666666,
2362 + "selectivity": 0.8333333333333334
2363 + },
2364 + {
2365 + "layer": 16,
2366 + "seed": 1,
2367 + "task_acc": 1.0,
2368 + "control_acc": 0.875,
2369 + "selectivity": 0.125
2370 + },
2371 + {
2372 + "layer": 16,
2373 + "seed": 2,
2374 + "task_acc": 1.0,
2375 + "control_acc": 0.25,
2376 + "selectivity": 0.75
2377 + },
2378 + {
2379 + "layer": 16,
2380 + "seed": 3,
2381 + "task_acc": 1.0,
2382 + "control_acc": 0.3333333333333333,
2383 + "selectivity": 0.6666666666666667
2384 + },
2385 + {
2386 + "layer": 16,
2387 + "seed": 4,
2388 + "task_acc": 1.0,
2389 + "control_acc": 0.0625,
2390 + "selectivity": 0.9375
2391 + },
2392 + {
2393 + "layer": 17,
2394 + "seed": 0,
2395 + "task_acc": 1.0,
2396 + "control_acc": 0.0625,
2397 + "selectivity": 0.9375
2398 + },
2399 + {
2400 + "layer": 17,
2401 + "seed": 1,
2402 + "task_acc": 1.0,
2403 + "control_acc": 0.8958333333333334,
2404 + "selectivity": 0.10416666666666663
2405 + },
2406 + {
2407 + "layer": 17,
2408 + "seed": 2,
2409 + "task_acc": 1.0,
2410 + "control_acc": 0.14583333333333334,
2411 + "selectivity": 0.8541666666666666
2412 + },
2413 + {
2414 + "layer": 17,
2415 + "seed": 3,
2416 + "task_acc": 1.0,
2417 + "control_acc": 0.22916666666666666,
2418 + "selectivity": 0.7708333333333334
2419 + },
2420 + {
2421 + "layer": 17,
2422 + "seed": 4,
2423 + "task_acc": 1.0,
2424 + "control_acc": 0.10416666666666667,
2425 + "selectivity": 0.8958333333333334
2426 + },
2427 + {
2428 + "layer": 18,
2429 + "seed": 0,
2430 + "task_acc": 1.0,
2431 + "control_acc": 0.10416666666666667,
2432 + "selectivity": 0.8958333333333334
2433 + },
2434 + {
2435 + "layer": 18,
2436 + "seed": 1,
2437 + "task_acc": 1.0,
2438 + "control_acc": 0.7916666666666666,
2439 + "selectivity": 0.20833333333333337
2440 + },
2441 + {
2442 + "layer": 18,
2443 + "seed": 2,
2444 + "task_acc": 1.0,
2445 + "control_acc": 0.10416666666666667,
2446 + "selectivity": 0.8958333333333334
2447 + },
2448 + {
2449 + "layer": 18,
2450 + "seed": 3,
2451 + "task_acc": 1.0,
2452 + "control_acc": 0.10416666666666667,
2453 + "selectivity": 0.8958333333333334
2454 + },
2455 + {
2456 + "layer": 18,
2457 + "seed": 4,
2458 + "task_acc": 1.0,
2459 + "control_acc": 0.08333333333333333,
2460 + "selectivity": 0.9166666666666666
2461 + },
2462 + {
2463 + "layer": 19,
2464 + "seed": 0,
2465 + "task_acc": 1.0,
2466 + "control_acc": 0.22916666666666666,
2467 + "selectivity": 0.7708333333333334
2468 + },
2469 + {
2470 + "layer": 19,
2471 + "seed": 1,
2472 + "task_acc": 1.0,
2473 + "control_acc": 0.7291666666666666,
2474 + "selectivity": 0.27083333333333337
2475 + },
2476 + {
2477 + "layer": 19,
2478 + "seed": 2,
2479 + "task_acc": 1.0,
2480 + "control_acc": 0.10416666666666667,
2481 + "selectivity": 0.8958333333333334
2482 + },
2483 + {
2484 + "layer": 19,
2485 + "seed": 3,
2486 + "task_acc": 1.0,
2487 + "control_acc": 0.9791666666666666,
2488 + "selectivity": 0.02083333333333337
2489 + },
2490 + {
2491 + "layer": 19,
2492 + "seed": 4,
2493 + "task_acc": 1.0,
2494 + "control_acc": 0.6041666666666666,
2495 + "selectivity": 0.39583333333333337
2496 + },
2497 + {
2498 + "layer": 20,
2499 + "seed": 0,
2500 + "task_acc": 1.0,
2501 + "control_acc": 0.25,
2502 + "selectivity": 0.75
2503 + },
2504 + {
2505 + "layer": 20,
2506 + "seed": 1,
2507 + "task_acc": 1.0,
2508 + "control_acc": 0.9166666666666666,
2509 + "selectivity": 0.08333333333333337
2510 + },
2511 + {
2512 + "layer": 20,
2513 + "seed": 2,
2514 + "task_acc": 1.0,
2515 + "control_acc": 0.8958333333333334,
2516 + "selectivity": 0.10416666666666663
2517 + },
2518 + {
2519 + "layer": 20,
2520 + "seed": 3,
2521 + "task_acc": 1.0,
2522 + "control_acc": 0.9375,
2523 + "selectivity": 0.0625
2524 + },
2525 + {
2526 + "layer": 20,
2527 + "seed": 4,
2528 + "task_acc": 1.0,
2529 + "control_acc": 0.6041666666666666,
2530 + "selectivity": 0.39583333333333337
2531 + },
2532 + {
2533 + "layer": 21,
2534 + "seed": 0,
2535 + "task_acc": 1.0,
2536 + "control_acc": 0.125,
2537 + "selectivity": 0.875
2538 + },
2539 + {
2540 + "layer": 21,
2541 + "seed": 1,
2542 + "task_acc": 1.0,
2543 + "control_acc": 0.9166666666666666,
2544 + "selectivity": 0.08333333333333337
2545 + },
2546 + {
2547 + "layer": 21,
2548 + "seed": 2,
2549 + "task_acc": 1.0,
2550 + "control_acc": 0.10416666666666667,
2551 + "selectivity": 0.8958333333333334
2552 + },
2553 + {
2554 + "layer": 21,
2555 + "seed": 3,
2556 + "task_acc": 1.0,
2557 + "control_acc": 0.9375,
2558 + "selectivity": 0.0625
2559 + },
2560 + {
2561 + "layer": 21,
2562 + "seed": 4,
2563 + "task_acc": 1.0,
2564 + "control_acc": 0.75,
2565 + "selectivity": 0.25
2566 + },
2567 + {
2568 + "layer": 22,
2569 + "seed": 0,
2570 + "task_acc": 1.0,
2571 + "control_acc": 0.16666666666666666,
2572 + "selectivity": 0.8333333333333334
2573 + },
2574 + {
2575 + "layer": 22,
2576 + "seed": 1,
2577 + "task_acc": 1.0,
2578 + "control_acc": 0.6875,
2579 + "selectivity": 0.3125
2580 + },
2581 + {
2582 + "layer": 22,
2583 + "seed": 2,
2584 + "task_acc": 1.0,
2585 + "control_acc": 0.16666666666666666,
2586 + "selectivity": 0.8333333333333334
2587 + },
2588 + {
2589 + "layer": 22,
2590 + "seed": 3,
2591 + "task_acc": 1.0,
2592 + "control_acc": 0.14583333333333334,
2593 + "selectivity": 0.8541666666666666
2594 + },
2595 + {
2596 + "layer": 22,
2597 + "seed": 4,
2598 + "task_acc": 1.0,
2599 + "control_acc": 0.7916666666666666,
2600 + "selectivity": 0.20833333333333337
2601 + },
2602 + {
2603 + "layer": 23,
2604 + "seed": 0,
2605 + "task_acc": 1.0,
2606 + "control_acc": 0.1875,
2607 + "selectivity": 0.8125
2608 + },
2609 + {
2610 + "layer": 23,
2611 + "seed": 1,
2612 + "task_acc": 1.0,
2613 + "control_acc": 0.5208333333333334,
2614 + "selectivity": 0.47916666666666663
2615 + },
2616 + {
2617 + "layer": 23,
2618 + "seed": 2,
2619 + "task_acc": 1.0,
2620 + "control_acc": 0.8958333333333334,
2621 + "selectivity": 0.10416666666666663
2622 + },
2623 + {
2624 + "layer": 23,
2625 + "seed": 3,
2626 + "task_acc": 1.0,
2627 + "control_acc": 0.125,
2628 + "selectivity": 0.875
2629 + },
2630 + {
2631 + "layer": 23,
2632 + "seed": 4,
2633 + "task_acc": 1.0,
2634 + "control_acc": 0.75,
2635 + "selectivity": 0.25
2636 + },
2637 + {
2638 + "layer": 24,
2639 + "seed": 0,
2640 + "task_acc": 1.0,
2641 + "control_acc": 0.1875,
2642 + "selectivity": 0.8125
2643 + },
2644 + {
2645 + "layer": 24,
2646 + "seed": 1,
2647 + "task_acc": 1.0,
2648 + "control_acc": 0.9791666666666666,
2649 + "selectivity": 0.02083333333333337
2650 + },
2651 + {
2652 + "layer": 24,
2653 + "seed": 2,
2654 + "task_acc": 1.0,
2655 + "control_acc": 0.8958333333333334,
2656 + "selectivity": 0.10416666666666663
2657 + },
2658 + {
2659 + "layer": 24,
2660 + "seed": 3,
2661 + "task_acc": 1.0,
2662 + "control_acc": 0.9375,
2663 + "selectivity": 0.0625
2664 + },
2665 + {
2666 + "layer": 24,
2667 + "seed": 4,
2668 + "task_acc": 1.0,
2669 + "control_acc": 0.7708333333333334,
2670 + "selectivity": 0.22916666666666663
2671 + },
2672 + {
2673 + "layer": 25,
2674 + "seed": 0,
2675 + "task_acc": 1.0,
2676 + "control_acc": 0.22916666666666666,
2677 + "selectivity": 0.7708333333333334
2678 + },
2679 + {
2680 + "layer": 25,
2681 + "seed": 1,
2682 + "task_acc": 1.0,
2683 + "control_acc": 0.9375,
2684 + "selectivity": 0.0625
2685 + },
2686 + {
2687 + "layer": 25,
2688 + "seed": 2,
2689 + "task_acc": 1.0,
2690 + "control_acc": 0.8958333333333334,
2691 + "selectivity": 0.10416666666666663
2692 + },
2693 + {
2694 + "layer": 25,
2695 + "seed": 3,
2696 + "task_acc": 1.0,
2697 + "control_acc": 0.14583333333333334,
2698 + "selectivity": 0.8541666666666666
2699 + },
2700 + {
2701 + "layer": 25,
2702 + "seed": 4,
2703 + "task_acc": 1.0,
2704 + "control_acc": 0.6875,
2705 + "selectivity": 0.3125
2706 + },
2707 + {
2708 + "layer": 26,
2709 + "seed": 0,
2710 + "task_acc": 1.0,
2711 + "control_acc": 0.14583333333333334,
2712 + "selectivity": 0.8541666666666666
2713 + },
2714 + {
2715 + "layer": 26,
2716 + "seed": 1,
2717 + "task_acc": 1.0,
2718 + "control_acc": 0.6458333333333334,
2719 + "selectivity": 0.35416666666666663
2720 + },
2721 + {
2722 + "layer": 26,
2723 + "seed": 2,
2724 + "task_acc": 1.0,
2725 + "control_acc": 0.8958333333333334,
2726 + "selectivity": 0.10416666666666663
2727 + },
2728 + {
2729 + "layer": 26,
2730 + "seed": 3,
2731 + "task_acc": 1.0,
2732 + "control_acc": 0.125,
2733 + "selectivity": 0.875
2734 + },
2735 + {
2736 + "layer": 26,
2737 + "seed": 4,
2738 + "task_acc": 1.0,
2739 + "control_acc": 0.6458333333333334,
2740 + "selectivity": 0.35416666666666663
2741 + },
2742 + {
2743 + "layer": 27,
2744 + "seed": 0,
2745 + "task_acc": 1.0,
2746 + "control_acc": 0.0625,
2747 + "selectivity": 0.9375
2748 + },
2749 + {
2750 + "layer": 27,
2751 + "seed": 1,
2752 + "task_acc": 1.0,
2753 + "control_acc": 0.8333333333333334,
2754 + "selectivity": 0.16666666666666663
2755 + },
2756 + {
2757 + "layer": 27,
2758 + "seed": 2,
2759 + "task_acc": 1.0,
2760 + "control_acc": 0.8958333333333334,
2761 + "selectivity": 0.10416666666666663
2762 + },
2763 + {
2764 + "layer": 27,
2765 + "seed": 3,
2766 + "task_acc": 1.0,
2767 + "control_acc": 0.9583333333333334,
2768 + "selectivity": 0.04166666666666663
2769 + },
2770 + {
2771 + "layer": 27,
2772 + "seed": 4,
2773 + "task_acc": 1.0,
2774 + "control_acc": 0.6875,
2775 + "selectivity": 0.3125
2776 + }
2777 + ]
2778 + },
2779 + "code_prose_A": {
2780 + "layers": [
2781 + {
2782 + "layer": 0,
2783 + "task_acc_mean": 1.0,
2784 + "task_acc_seed_sd": 0.0,
2785 + "selectivity_mean": 0.5083333333333333,
2786 + "selectivity_ci": [
2787 + 0.2,
2788 + 0.775
2789 + ],
2790 + "p_boot": 0.0006,
2791 + "fdr_significant": true
2792 + },
2793 + {
2794 + "layer": 1,
2795 + "task_acc_mean": 1.0,
2796 + "task_acc_seed_sd": 0.0,
2797 + "selectivity_mean": 0.4958333333333333,
2798 + "selectivity_ci": [
2799 + 0.2541666666666666,
2800 + 0.7291666666666667
2801 + ],
2802 + "p_boot": 0.0001,
2803 + "fdr_significant": true
2804 + },
2805 + {
2806 + "layer": 2,
2807 + "task_acc_mean": 1.0,
2808 + "task_acc_seed_sd": 0.0,
2809 + "selectivity_mean": 0.4291666666666666,
2810 + "selectivity_ci": [
2811 + 0.14583333333333331,
2812 + 0.7208333333333332
2813 + ],
2814 + "p_boot": 0.0001,
2815 + "fdr_significant": true
2816 + },
2817 + {
2818 + "layer": 3,
2819 + "task_acc_mean": 1.0,
2820 + "task_acc_seed_sd": 0.0,
2821 + "selectivity_mean": 0.6708333333333333,
2822 + "selectivity_ci": [
2823 + 0.4,
2824 + 0.8916666666666668
2825 + ],
2826 + "p_boot": 0.0001,
2827 + "fdr_significant": true
2828 + },
2829 + {
2830 + "layer": 4,
2831 + "task_acc_mean": 1.0,
2832 + "task_acc_seed_sd": 0.0,
2833 + "selectivity_mean": 0.5083333333333334,
2834 + "selectivity_ci": [
2835 + 0.275,
2836 + 0.7416666666666667
2837 + ],
2838 + "p_boot": 0.0001,
2839 + "fdr_significant": true
2840 + },
2841 + {
2842 + "layer": 5,
2843 + "task_acc_mean": 1.0,
2844 + "task_acc_seed_sd": 0.0,
2845 + "selectivity_mean": 0.5291666666666666,
2846 + "selectivity_ci": [
2847 + 0.2583333333333333,
2848 + 0.8
2849 + ],
2850 + "p_boot": 0.0001,
2851 + "fdr_significant": true
2852 + },
2853 + {
2854 + "layer": 6,
2855 + "task_acc_mean": 1.0,
2856 + "task_acc_seed_sd": 0.0,
2857 + "selectivity_mean": 0.3541666666666667,
2858 + "selectivity_ci": [
2859 + 0.1166666666666667,
2860 + 0.6085416666666635
2861 + ],
2862 + "p_boot": 0.0001,
2863 + "fdr_significant": true
2864 + },
2865 + {
2866 + "layer": 7,
2867 + "task_acc_mean": 1.0,
2868 + "task_acc_seed_sd": 0.0,
2869 + "selectivity_mean": 0.3999999999999999,
2870 + "selectivity_ci": [
2871 + 0.10416666666666666,
2872 + 0.6958333333333333
2873 + ],
2874 + "p_boot": 0.0001,
2875 + "fdr_significant": true
2876 + },
2877 + {
2878 + "layer": 8,
2879 + "task_acc_mean": 1.0,
2880 + "task_acc_seed_sd": 0.0,
2881 + "selectivity_mean": 0.45,
2882 + "selectivity_ci": [
2883 + 0.175,
2884 + 0.7250000000000001
2885 + ],
2886 + "p_boot": 0.0001,
2887 + "fdr_significant": true
2888 + },
2889 + {
2890 + "layer": 9,
2891 + "task_acc_mean": 1.0,
2892 + "task_acc_seed_sd": 0.0,
2893 + "selectivity_mean": 0.5625000000000001,
2894 + "selectivity_ci": [
2895 + 0.27083333333333337,
2896 + 0.8541666666666667
2897 + ],
2898 + "p_boot": 0.0001,
2899 + "fdr_significant": true
2900 + },
2901 + {
2902 + "layer": 10,
2903 + "task_acc_mean": 1.0,
2904 + "task_acc_seed_sd": 0.0,
2905 + "selectivity_mean": 0.4208333333333334,
2906 + "selectivity_ci": [
2907 + 0.16250000000000003,
2908 + 0.6791666666666668
2909 + ],
2910 + "p_boot": 0.0001,
2911 + "fdr_significant": true
2912 + },
2913 + {
2914 + "layer": 11,
2915 + "task_acc_mean": 1.0,
2916 + "task_acc_seed_sd": 0.0,
2917 + "selectivity_mean": 0.4958333333333334,
2918 + "selectivity_ci": [
2919 + 0.225,
2920 + 0.7666666666666667
2921 + ],
2922 + "p_boot": 0.0001,
2923 + "fdr_significant": true
2924 + },
2925 + {
2926 + "layer": 12,
2927 + "task_acc_mean": 1.0,
2928 + "task_acc_seed_sd": 0.0,
2929 + "selectivity_mean": 0.4666666666666666,
2930 + "selectivity_ci": [
2931 + 0.2208333333333333,
2932 + 0.7166666666666666
2933 + ],
2934 + "p_boot": 0.0001,
2935 + "fdr_significant": true
2936 + },
2937 + {
2938 + "layer": 13,
2939 + "task_acc_mean": 1.0,
2940 + "task_acc_seed_sd": 0.0,
2941 + "selectivity_mean": 0.5416666666666667,
2942 + "selectivity_ci": [
2943 + 0.31666666666666665,
2944 + 0.7666666666666667
2945 + ],
2946 + "p_boot": 0.0001,
2947 + "fdr_significant": true
2948 + },
2949 + {
2950 + "layer": 14,
2951 + "task_acc_mean": 1.0,
2952 + "task_acc_seed_sd": 0.0,
2953 + "selectivity_mean": 0.475,
2954 + "selectivity_ci": [
2955 + 0.2458333333333333,
2956 + 0.7000000000000001
2957 + ],
2958 + "p_boot": 0.0001,
2959 + "fdr_significant": true
2960 + },
2961 + {
2962 + "layer": 15,
2963 + "task_acc_mean": 1.0,
2964 + "task_acc_seed_sd": 0.0,
2965 + "selectivity_mean": 0.4541666666666667,
2966 + "selectivity_ci": [
2967 + 0.2,
2968 + 0.7000000000000001
2969 + ],
2970 + "p_boot": 0.0001,
2971 + "fdr_significant": true
2972 + },
2973 + {
2974 + "layer": 16,
2975 + "task_acc_mean": 1.0,
2976 + "task_acc_seed_sd": 0.0,
2977 + "selectivity_mean": 0.425,
2978 + "selectivity_ci": [
2979 + 0.15000000000000002,
2980 + 0.7124999999999999
2981 + ],
2982 + "p_boot": 0.0001,
2983 + "fdr_significant": true
2984 + },
2985 + {
2986 + "layer": 17,
2987 + "task_acc_mean": 1.0,
2988 + "task_acc_seed_sd": 0.0,
2989 + "selectivity_mean": 0.4166666666666667,
2990 + "selectivity_ci": [
2991 + 0.1375,
2992 + 0.6958333333333334
2993 + ],
2994 + "p_boot": 0.0001,
2995 + "fdr_significant": true
2996 + },
2997 + {
2998 + "layer": 18,
2999 + "task_acc_mean": 1.0,
3000 + "task_acc_seed_sd": 0.0,
3001 + "selectivity_mean": 0.4416666666666667,
3002 + "selectivity_ci": [
3003 + 0.13333333333333333,
3004 + 0.775
3005 + ],
3006 + "p_boot": 0.0001,
3007 + "fdr_significant": true
3008 + },
3009 + {
3010 + "layer": 19,
3011 + "task_acc_mean": 1.0,
3012 + "task_acc_seed_sd": 0.0,
3013 + "selectivity_mean": 0.7916666666666666,
3014 + "selectivity_ci": [
3015 + 0.5416666666666666,
3016 + 0.9625
3017 + ],
3018 + "p_boot": 0.0001,
3019 + "fdr_significant": true
3020 + },
3021 + {
3022 + "layer": 20,
3023 + "task_acc_mean": 1.0,
3024 + "task_acc_seed_sd": 0.0,
3025 + "selectivity_mean": 0.8625,
3026 + "selectivity_ci": [
3027 + 0.8208333333333332,
3028 + 0.9125
3029 + ],
3030 + "p_boot": 0.0001,
3031 + "fdr_significant": true
3032 + },
3033 + {
3034 + "layer": 21,
3035 + "task_acc_mean": 1.0,
3036 + "task_acc_seed_sd": 0.0,
3037 + "selectivity_mean": 0.6791666666666667,
3038 + "selectivity_ci": [
3039 + 0.425,
3040 + 0.8541666666666667
3041 + ],
3042 + "p_boot": 0.0001,
3043 + "fdr_significant": true
3044 + },
3045 + {
3046 + "layer": 22,
3047 + "task_acc_mean": 1.0,
3048 + "task_acc_seed_sd": 0.0,
3049 + "selectivity_mean": 0.5791666666666666,
3050 + "selectivity_ci": [
3051 + 0.2875,
3052 + 0.8416666666666666
3053 + ],
3054 + "p_boot": 0.0001,
3055 + "fdr_significant": true
3056 + },
3057 + {
3058 + "layer": 23,
3059 + "task_acc_mean": 1.0,
3060 + "task_acc_seed_sd": 0.0,
3061 + "selectivity_mean": 0.675,
3062 + "selectivity_ci": [
3063 + 0.4083333333333334,
3064 + 0.8416666666666668
3065 + ],
3066 + "p_boot": 0.0001,
3067 + "fdr_significant": true
3068 + },
3069 + {
3070 + "layer": 24,
3071 + "task_acc_mean": 1.0,
3072 + "task_acc_seed_sd": 0.0,
3073 + "selectivity_mean": 0.8041666666666666,
3074 + "selectivity_ci": [
3075 + 0.7041666666666666,
3076 + 0.9
3077 + ],
3078 + "p_boot": 0.0001,
3079 + "fdr_significant": true
3080 + },
3081 + {
3082 + "layer": 25,
3083 + "task_acc_mean": 1.0,
3084 + "task_acc_seed_sd": 0.0,
3085 + "selectivity_mean": 0.675,
3086 + "selectivity_ci": [
3087 + 0.3416666666666667,
3088 + 0.8791666666666667
3089 + ],
3090 + "p_boot": 0.0001,
3091 + "fdr_significant": true
3092 + },
3093 + {
3094 + "layer": 26,
3095 + "task_acc_mean": 1.0,
3096 + "task_acc_seed_sd": 0.0,
3097 + "selectivity_mean": 0.5958333333333333,
3098 + "selectivity_ci": [
3099 + 0.3416666666666666,
3100 + 0.85
3101 + ],
3102 + "p_boot": 0.0001,
3103 + "fdr_significant": true
3104 + },
3105 + {
3106 + "layer": 27,
3107 + "task_acc_mean": 1.0,
3108 + "task_acc_seed_sd": 0.0,
3109 + "selectivity_mean": 0.6291666666666667,
3110 + "selectivity_ci": [
3111 + 0.3875,
3112 + 0.7999999999999999
3113 + ],
3114 + "p_boot": 0.0001,
3115 + "fdr_significant": true
3116 + }
3117 + ],
3118 + "replication_rate_topk": {
3119 + "k": 5,
3120 + "point": 1.0,
3121 + "ci": [
3122 + 1.0,
3123 + 1.0
3124 + ]
3125 + },
3126 + "n_fdr_significant": 28,
3127 + "cells": [
3128 + {
3129 + "layer": 0,
3130 + "seed": 0,
3131 + "task_acc": 1.0,
3132 + "control_acc": 0.25,
3133 + "selectivity": 0.75
3134 + },
3135 + {
3136 + "layer": 0,
3137 + "seed": 1,
3138 + "task_acc": 1.0,
3139 + "control_acc": 0.7708333333333334,
3140 + "selectivity": 0.22916666666666663
3141 + },
3142 + {
3143 + "layer": 0,
3144 + "seed": 2,
3145 + "task_acc": 1.0,
3146 + "control_acc": 1.0,
3147 + "selectivity": 0.0
3148 + },
3149 + {
3150 + "layer": 0,
3151 + "seed": 3,
3152 + "task_acc": 1.0,
3153 + "control_acc": 0.20833333333333334,
3154 + "selectivity": 0.7916666666666666
3155 + },
3156 + {
3157 + "layer": 0,
3158 + "seed": 4,
3159 + "task_acc": 1.0,
3160 + "control_acc": 0.22916666666666666,
3161 + "selectivity": 0.7708333333333334
3162 + },
3163 + {
3164 + "layer": 1,
3165 + "seed": 0,
3166 + "task_acc": 1.0,
3167 + "control_acc": 0.3958333333333333,
3168 + "selectivity": 0.6041666666666667
3169 + },
3170 + {
3171 + "layer": 1,
3172 + "seed": 1,
3173 + "task_acc": 1.0,
3174 + "control_acc": 0.7708333333333334,
3175 + "selectivity": 0.22916666666666663
3176 + },
3177 + {
3178 + "layer": 1,
3179 + "seed": 2,
3180 + "task_acc": 1.0,
3181 + "control_acc": 0.8958333333333334,
3182 + "selectivity": 0.10416666666666663
3183 + },
3184 + {
3185 + "layer": 1,
3186 + "seed": 3,
3187 + "task_acc": 1.0,
3188 + "control_acc": 0.25,
3189 + "selectivity": 0.75
3190 + },
3191 + {
3192 + "layer": 1,
3193 + "seed": 4,
3194 + "task_acc": 1.0,
3195 + "control_acc": 0.20833333333333334,
3196 + "selectivity": 0.7916666666666666
3197 + },
3198 + {
3199 + "layer": 2,
3200 + "seed": 0,
3201 + "task_acc": 1.0,
3202 + "control_acc": 0.14583333333333334,
3203 + "selectivity": 0.8541666666666666
3204 + },
3205 + {
3206 + "layer": 2,
3207 + "seed": 1,
3208 + "task_acc": 1.0,
3209 + "control_acc": 0.7708333333333334,
3210 + "selectivity": 0.22916666666666663
3211 + },
3212 + {
3213 + "layer": 2,
3214 + "seed": 2,
3215 + "task_acc": 1.0,
3216 + "control_acc": 0.9791666666666666,
3217 + "selectivity": 0.02083333333333337
3218 + },
3219 + {
3220 + "layer": 2,
3221 + "seed": 3,
3222 + "task_acc": 1.0,
3223 + "control_acc": 0.7708333333333334,
3224 + "selectivity": 0.22916666666666663
3225 + },
3226 + {
3227 + "layer": 2,
3228 + "seed": 4,
3229 + "task_acc": 1.0,
3230 + "control_acc": 0.1875,
3231 + "selectivity": 0.8125
3232 + },
3233 + {
3234 + "layer": 3,
3235 + "seed": 0,
3236 + "task_acc": 1.0,
3237 + "control_acc": 0.4583333333333333,
3238 + "selectivity": 0.5416666666666667
3239 + },
3240 + {
3241 + "layer": 3,
3242 + "seed": 1,
3243 + "task_acc": 1.0,
3244 + "control_acc": 0.8125,
3245 + "selectivity": 0.1875
3246 + },
3247 + {
3248 + "layer": 3,
3249 + "seed": 2,
3250 + "task_acc": 1.0,
3251 + "control_acc": 0.10416666666666667,
3252 + "selectivity": 0.8958333333333334
3253 + },
3254 + {
3255 + "layer": 3,
3256 + "seed": 3,
3257 + "task_acc": 1.0,
3258 + "control_acc": 0.0625,
3259 + "selectivity": 0.9375
3260 + },
3261 + {
3262 + "layer": 3,
3263 + "seed": 4,
3264 + "task_acc": 1.0,
3265 + "control_acc": 0.20833333333333334,
3266 + "selectivity": 0.7916666666666666
3267 + },
3268 + {
3269 + "layer": 4,
3270 + "seed": 0,
3271 + "task_acc": 1.0,
3272 + "control_acc": 0.4583333333333333,
3273 + "selectivity": 0.5416666666666667
3274 + },
3275 + {
3276 + "layer": 4,
3277 + "seed": 1,
3278 + "task_acc": 1.0,
3279 + "control_acc": 0.7708333333333334,
3280 + "selectivity": 0.22916666666666663
3281 + },
3282 + {
3283 + "layer": 4,
3284 + "seed": 2,
3285 + "task_acc": 1.0,
3286 + "control_acc": 0.22916666666666666,
3287 + "selectivity": 0.7708333333333334
3288 + },
3289 + {
3290 + "layer": 4,
3291 + "seed": 3,
3292 + "task_acc": 1.0,
3293 + "control_acc": 0.8125,
3294 + "selectivity": 0.1875
3295 + },
3296 + {
3297 + "layer": 4,
3298 + "seed": 4,
3299 + "task_acc": 1.0,
3300 + "control_acc": 0.1875,
3301 + "selectivity": 0.8125
3302 + },
3303 + {
3304 + "layer": 5,
3305 + "seed": 0,
3306 + "task_acc": 1.0,
3307 + "control_acc": 0.4166666666666667,
3308 + "selectivity": 0.5833333333333333
3309 + },
3310 + {
3311 + "layer": 5,
3312 + "seed": 1,
3313 + "task_acc": 1.0,
3314 + "control_acc": 0.8333333333333334,
3315 + "selectivity": 0.16666666666666663
3316 + },
3317 + {
3318 + "layer": 5,
3319 + "seed": 2,
3320 + "task_acc": 1.0,
3321 + "control_acc": 0.10416666666666667,
3322 + "selectivity": 0.8958333333333334
3323 + },
3324 + {
3325 + "layer": 5,
3326 + "seed": 3,
3327 + "task_acc": 1.0,
3328 + "control_acc": 0.8125,
3329 + "selectivity": 0.1875
3330 + },
3331 + {
3332 + "layer": 5,
3333 + "seed": 4,
3334 + "task_acc": 1.0,
3335 + "control_acc": 0.1875,
3336 + "selectivity": 0.8125
3337 + },
3338 + {
3339 + "layer": 6,
3340 + "seed": 0,
3341 + "task_acc": 1.0,
3342 + "control_acc": 0.4375,
3343 + "selectivity": 0.5625
3344 + },
3345 + {
3346 + "layer": 6,
3347 + "seed": 1,
3348 + "task_acc": 1.0,
3349 + "control_acc": 0.8541666666666666,
3350 + "selectivity": 0.14583333333333337
3351 + },
3352 + {
3353 + "layer": 6,
3354 + "seed": 2,
3355 + "task_acc": 1.0,
3356 + "control_acc": 0.9791666666666666,
3357 + "selectivity": 0.02083333333333337
3358 + },
3359 + {
3360 + "layer": 6,
3361 + "seed": 3,
3362 + "task_acc": 1.0,
3363 + "control_acc": 0.75,
3364 + "selectivity": 0.25
3365 + },
3366 + {
3367 + "layer": 6,
3368 + "seed": 4,
3369 + "task_acc": 1.0,
3370 + "control_acc": 0.20833333333333334,
3371 + "selectivity": 0.7916666666666666
3372 + },
3373 + {
3374 + "layer": 7,
3375 + "seed": 0,
3376 + "task_acc": 1.0,
3377 + "control_acc": 0.14583333333333334,
3378 + "selectivity": 0.8541666666666666
3379 + },
3380 + {
3381 + "layer": 7,
3382 + "seed": 1,
3383 + "task_acc": 1.0,
3384 + "control_acc": 0.8125,
3385 + "selectivity": 0.1875
3386 + },
3387 + {
3388 + "layer": 7,
3389 + "seed": 2,
3390 + "task_acc": 1.0,
3391 + "control_acc": 0.9583333333333334,
3392 + "selectivity": 0.04166666666666663
3393 + },
3394 + {
3395 + "layer": 7,
3396 + "seed": 3,
3397 + "task_acc": 1.0,
3398 + "control_acc": 0.875,
3399 + "selectivity": 0.125
3400 + },
3401 + {
3402 + "layer": 7,
3403 + "seed": 4,
3404 + "task_acc": 1.0,
3405 + "control_acc": 0.20833333333333334,
3406 + "selectivity": 0.7916666666666666
3407 + },
3408 + {
3409 + "layer": 8,
3410 + "seed": 0,
3411 + "task_acc": 1.0,
3412 + "control_acc": 0.16666666666666666,
3413 + "selectivity": 0.8333333333333334
3414 + },
3415 + {
3416 + "layer": 8,
3417 + "seed": 1,
3418 + "task_acc": 1.0,
3419 + "control_acc": 0.7083333333333334,
3420 + "selectivity": 0.29166666666666663
3421 + },
3422 + {
3423 + "layer": 8,
3424 + "seed": 2,
3425 + "task_acc": 1.0,
3426 + "control_acc": 0.8125,
3427 + "selectivity": 0.1875
3428 + },
3429 + {
3430 + "layer": 8,
3431 + "seed": 3,
3432 + "task_acc": 1.0,
3433 + "control_acc": 0.8958333333333334,
3434 + "selectivity": 0.10416666666666663
3435 + },
3436 + {
3437 + "layer": 8,
3438 + "seed": 4,
3439 + "task_acc": 1.0,
3440 + "control_acc": 0.16666666666666666,
3441 + "selectivity": 0.8333333333333334
3442 + },
3443 + {
3444 + "layer": 9,
3445 + "seed": 0,
3446 + "task_acc": 1.0,
3447 + "control_acc": 0.10416666666666667,
3448 + "selectivity": 0.8958333333333334
3449 + },
3450 + {
3451 + "layer": 9,
3452 + "seed": 1,
3453 + "task_acc": 1.0,
3454 + "control_acc": 0.875,
3455 + "selectivity": 0.125
3456 + },
3457 + {
3458 + "layer": 9,
3459 + "seed": 2,
3460 + "task_acc": 1.0,
3461 + "control_acc": 0.1875,
3462 + "selectivity": 0.8125
3463 + },
3464 + {
3465 + "layer": 9,
3466 + "seed": 3,
3467 + "task_acc": 1.0,
3468 + "control_acc": 0.8541666666666666,
3469 + "selectivity": 0.14583333333333337
3470 + },
3471 + {
3472 + "layer": 9,
3473 + "seed": 4,
3474 + "task_acc": 1.0,
3475 + "control_acc": 0.16666666666666666,
3476 + "selectivity": 0.8333333333333334
3477 + },
3478 + {
3479 + "layer": 10,
3480 + "seed": 0,
3481 + "task_acc": 1.0,
3482 + "control_acc": 0.8541666666666666,
3483 + "selectivity": 0.14583333333333337
3484 + },
3485 + {
3486 + "layer": 10,
3487 + "seed": 1,
3488 + "task_acc": 1.0,
3489 + "control_acc": 0.7708333333333334,
3490 + "selectivity": 0.22916666666666663
3491 + },
3492 + {
3493 + "layer": 10,
3494 + "seed": 2,
3495 + "task_acc": 1.0,
3496 + "control_acc": 0.25,
3497 + "selectivity": 0.75
3498 + },
3499 + {
3500 + "layer": 10,
3501 + "seed": 3,
3502 + "task_acc": 1.0,
3503 + "control_acc": 0.8541666666666666,
3504 + "selectivity": 0.14583333333333337
3505 + },
3506 + {
3507 + "layer": 10,
3508 + "seed": 4,
3509 + "task_acc": 1.0,
3510 + "control_acc": 0.16666666666666666,
3511 + "selectivity": 0.8333333333333334
3512 + },
3513 + {
3514 + "layer": 11,
3515 + "seed": 0,
3516 + "task_acc": 1.0,
3517 + "control_acc": 0.1875,
3518 + "selectivity": 0.8125
3519 + },
3520 + {
3521 + "layer": 11,
3522 + "seed": 1,
3523 + "task_acc": 1.0,
3524 + "control_acc": 0.5416666666666666,
3525 + "selectivity": 0.45833333333333337
3526 + },
3527 + {
3528 + "layer": 11,
3529 + "seed": 2,
3530 + "task_acc": 1.0,
3531 + "control_acc": 0.125,
3532 + "selectivity": 0.875
3533 + },
3534 + {
3535 + "layer": 11,
3536 + "seed": 3,
3537 + "task_acc": 1.0,
3538 + "control_acc": 0.875,
3539 + "selectivity": 0.125
3540 + },
3541 + {
3542 + "layer": 11,
3543 + "seed": 4,
3544 + "task_acc": 1.0,
3545 + "control_acc": 0.7916666666666666,
3546 + "selectivity": 0.20833333333333337
3547 + },
3548 + {
3549 + "layer": 12,
3550 + "seed": 0,
3551 + "task_acc": 1.0,
3552 + "control_acc": 0.20833333333333334,
3553 + "selectivity": 0.7916666666666666
3554 + },
3555 + {
3556 + "layer": 12,
3557 + "seed": 1,
3558 + "task_acc": 1.0,
3559 + "control_acc": 0.7291666666666666,
3560 + "selectivity": 0.27083333333333337
3561 + },
3562 + {
3563 + "layer": 12,
3564 + "seed": 2,
3565 + "task_acc": 1.0,
3566 + "control_acc": 0.14583333333333334,
3567 + "selectivity": 0.8541666666666666
3568 + },
3569 + {
3570 + "layer": 12,
3571 + "seed": 3,
3572 + "task_acc": 1.0,
3573 + "control_acc": 0.8125,
3574 + "selectivity": 0.1875
3575 + },
3576 + {
3577 + "layer": 12,
3578 + "seed": 4,
3579 + "task_acc": 1.0,
3580 + "control_acc": 0.7708333333333334,
3581 + "selectivity": 0.22916666666666663
3582 + },
3583 + {
3584 + "layer": 13,
3585 + "seed": 0,
3586 + "task_acc": 1.0,
3587 + "control_acc": 0.22916666666666666,
3588 + "selectivity": 0.7708333333333334
3589 + },
3590 + {
3591 + "layer": 13,
3592 + "seed": 1,
3593 + "task_acc": 1.0,
3594 + "control_acc": 0.6875,
3595 + "selectivity": 0.3125
3596 + },
3597 + {
3598 + "layer": 13,
3599 + "seed": 2,
3600 + "task_acc": 1.0,
3601 + "control_acc": 0.10416666666666667,
3602 + "selectivity": 0.8958333333333334
3603 + },
3604 + {
3605 + "layer": 13,
3606 + "seed": 3,
3607 + "task_acc": 1.0,
3608 + "control_acc": 0.7708333333333334,
3609 + "selectivity": 0.22916666666666663
3610 + },
3611 + {
3612 + "layer": 13,
3613 + "seed": 4,
3614 + "task_acc": 1.0,
3615 + "control_acc": 0.5,
3616 + "selectivity": 0.5
3617 + },
3618 + {
3619 + "layer": 14,
3620 + "seed": 0,
3621 + "task_acc": 1.0,
3622 + "control_acc": 0.5416666666666666,
3623 + "selectivity": 0.45833333333333337
3624 + },
3625 + {
3626 + "layer": 14,
3627 + "seed": 1,
3628 + "task_acc": 1.0,
3629 + "control_acc": 0.4375,
3630 + "selectivity": 0.5625
3631 + },
3632 + {
3633 + "layer": 14,
3634 + "seed": 2,
3635 + "task_acc": 1.0,
3636 + "control_acc": 0.10416666666666667,
3637 + "selectivity": 0.8958333333333334
3638 + },
3639 + {
3640 + "layer": 14,
3641 + "seed": 3,
3642 + "task_acc": 1.0,
3643 + "control_acc": 0.8958333333333334,
3644 + "selectivity": 0.10416666666666663
3645 + },
3646 + {
3647 + "layer": 14,
3648 + "seed": 4,
3649 + "task_acc": 1.0,
3650 + "control_acc": 0.6458333333333334,
3651 + "selectivity": 0.35416666666666663
3652 + },
3653 + {
3654 + "layer": 15,
3655 + "seed": 0,
3656 + "task_acc": 1.0,
3657 + "control_acc": 0.1875,
3658 + "selectivity": 0.8125
3659 + },
3660 + {
3661 + "layer": 15,
3662 + "seed": 1,
3663 + "task_acc": 1.0,
3664 + "control_acc": 0.8333333333333334,
3665 + "selectivity": 0.16666666666666663
3666 + },
3667 + {
3668 + "layer": 15,
3669 + "seed": 2,
3670 + "task_acc": 1.0,
3671 + "control_acc": 0.22916666666666666,
3672 + "selectivity": 0.7708333333333334
3673 + },
3674 + {
3675 + "layer": 15,
3676 + "seed": 3,
3677 + "task_acc": 1.0,
3678 + "control_acc": 0.8541666666666666,
3679 + "selectivity": 0.14583333333333337
3680 + },
3681 + {
3682 + "layer": 15,
3683 + "seed": 4,
3684 + "task_acc": 1.0,
3685 + "control_acc": 0.625,
3686 + "selectivity": 0.375
3687 + },
3688 + {
3689 + "layer": 16,
3690 + "seed": 0,
3691 + "task_acc": 1.0,
3692 + "control_acc": 0.20833333333333334,
3693 + "selectivity": 0.7916666666666666
3694 + },
3695 + {
3696 + "layer": 16,
3697 + "seed": 1,
3698 + "task_acc": 1.0,
3699 + "control_acc": 0.7916666666666666,
3700 + "selectivity": 0.20833333333333337
3701 + },
3702 + {
3703 + "layer": 16,
3704 + "seed": 2,
3705 + "task_acc": 1.0,
3706 + "control_acc": 0.14583333333333334,
3707 + "selectivity": 0.8541666666666666
3708 + },
3709 + {
3710 + "layer": 16,
3711 + "seed": 3,
3712 + "task_acc": 1.0,
3713 + "control_acc": 0.9375,
3714 + "selectivity": 0.0625
3715 + },
3716 + {
3717 + "layer": 16,
3718 + "seed": 4,
3719 + "task_acc": 1.0,
3720 + "control_acc": 0.7916666666666666,
3721 + "selectivity": 0.20833333333333337
3722 + },
3723 + {
3724 + "layer": 17,
3725 + "seed": 0,
3726 + "task_acc": 1.0,
3727 + "control_acc": 0.1875,
3728 + "selectivity": 0.8125
3729 + },
3730 + {
3731 + "layer": 17,
3732 + "seed": 1,
3733 + "task_acc": 1.0,
3734 + "control_acc": 0.8333333333333334,
3735 + "selectivity": 0.16666666666666663
3736 + },
3737 + {
3738 + "layer": 17,
3739 + "seed": 2,
3740 + "task_acc": 1.0,
3741 + "control_acc": 0.16666666666666666,
3742 + "selectivity": 0.8333333333333334
3743 + },
3744 + {
3745 + "layer": 17,
3746 + "seed": 3,
3747 + "task_acc": 1.0,
3748 + "control_acc": 0.9166666666666666,
3749 + "selectivity": 0.08333333333333337
3750 + },
3751 + {
3752 + "layer": 17,
3753 + "seed": 4,
3754 + "task_acc": 1.0,
3755 + "control_acc": 0.8125,
3756 + "selectivity": 0.1875
3757 + },
3758 + {
3759 + "layer": 18,
3760 + "seed": 0,
3761 + "task_acc": 1.0,
3762 + "control_acc": 0.8333333333333334,
3763 + "selectivity": 0.16666666666666663
3764 + },
3765 + {
3766 + "layer": 18,
3767 + "seed": 1,
3768 + "task_acc": 1.0,
3769 + "control_acc": 0.8333333333333334,
3770 + "selectivity": 0.16666666666666663
3771 + },
3772 + {
3773 + "layer": 18,
3774 + "seed": 2,
3775 + "task_acc": 1.0,
3776 + "control_acc": 0.041666666666666664,
3777 + "selectivity": 0.9583333333333334
3778 + },
3779 + {
3780 + "layer": 18,
3781 + "seed": 3,
3782 + "task_acc": 1.0,
3783 + "control_acc": 0.9166666666666666,
3784 + "selectivity": 0.08333333333333337
3785 + },
3786 + {
3787 + "layer": 18,
3788 + "seed": 4,
3789 + "task_acc": 1.0,
3790 + "control_acc": 0.16666666666666666,
3791 + "selectivity": 0.8333333333333334
3792 + },
3793 + {
3794 + "layer": 19,
3795 + "seed": 0,
3796 + "task_acc": 1.0,
3797 + "control_acc": 0.020833333333333332,
3798 + "selectivity": 0.9791666666666666
3799 + },
3800 + {
3801 + "layer": 19,
3802 + "seed": 1,
3803 + "task_acc": 1.0,
3804 + "control_acc": 0.020833333333333332,
3805 + "selectivity": 0.9791666666666666
3806 + },
3807 + {
3808 + "layer": 19,
3809 + "seed": 2,
3810 + "task_acc": 1.0,
3811 + "control_acc": 0.10416666666666667,
3812 + "selectivity": 0.8958333333333334
3813 + },
3814 + {
3815 + "layer": 19,
3816 + "seed": 3,
3817 + "task_acc": 1.0,
3818 + "control_acc": 0.20833333333333334,
3819 + "selectivity": 0.7916666666666666
3820 + },
3821 + {
3822 + "layer": 19,
3823 + "seed": 4,
3824 + "task_acc": 1.0,
3825 + "control_acc": 0.6875,
3826 + "selectivity": 0.3125
3827 + },
3828 + {
3829 + "layer": 20,
3830 + "seed": 0,
3831 + "task_acc": 1.0,
3832 + "control_acc": 0.1875,
3833 + "selectivity": 0.8125
3834 + },
3835 + {
3836 + "layer": 20,
3837 + "seed": 1,
3838 + "task_acc": 1.0,
3839 + "control_acc": 0.041666666666666664,
3840 + "selectivity": 0.9583333333333334
3841 + },
3842 + {
3843 + "layer": 20,
3844 + "seed": 2,
3845 + "task_acc": 1.0,
3846 + "control_acc": 0.125,
3847 + "selectivity": 0.875
3848 + },
3849 + {
3850 + "layer": 20,
3851 + "seed": 3,
3852 + "task_acc": 1.0,
3853 + "control_acc": 0.14583333333333334,
3854 + "selectivity": 0.8541666666666666
3855 + },
3856 + {
3857 + "layer": 20,
3858 + "seed": 4,
3859 + "task_acc": 1.0,
3860 + "control_acc": 0.1875,
3861 + "selectivity": 0.8125
3862 + },
3863 + {
3864 + "layer": 21,
3865 + "seed": 0,
3866 + "task_acc": 1.0,
3867 + "control_acc": 0.3333333333333333,
3868 + "selectivity": 0.6666666666666667
3869 + },
3870 + {
3871 + "layer": 21,
3872 + "seed": 1,
3873 + "task_acc": 1.0,
3874 + "control_acc": 0.8125,
3875 + "selectivity": 0.1875
3876 + },
3877 + {
3878 + "layer": 21,
3879 + "seed": 2,
3880 + "task_acc": 1.0,
3881 + "control_acc": 0.10416666666666667,
3882 + "selectivity": 0.8958333333333334
3883 + },
3884 + {
3885 + "layer": 21,
3886 + "seed": 3,
3887 + "task_acc": 1.0,
3888 + "control_acc": 0.1875,
3889 + "selectivity": 0.8125
3890 + },
3891 + {
3892 + "layer": 21,
3893 + "seed": 4,
3894 + "task_acc": 1.0,
3895 + "control_acc": 0.16666666666666666,
3896 + "selectivity": 0.8333333333333334
3897 + },
3898 + {
3899 + "layer": 22,
3900 + "seed": 0,
3901 + "task_acc": 1.0,
3902 + "control_acc": 0.25,
3903 + "selectivity": 0.75
3904 + },
3905 + {
3906 + "layer": 22,
3907 + "seed": 1,
3908 + "task_acc": 1.0,
3909 + "control_acc": 0.6875,
3910 + "selectivity": 0.3125
3911 + },
3912 + {
3913 + "layer": 22,
3914 + "seed": 2,
3915 + "task_acc": 1.0,
3916 + "control_acc": 0.08333333333333333,
3917 + "selectivity": 0.9166666666666666
3918 + },
3919 + {
3920 + "layer": 22,
3921 + "seed": 3,
3922 + "task_acc": 1.0,
3923 + "control_acc": 0.1875,
3924 + "selectivity": 0.8125
3925 + },
3926 + {
3927 + "layer": 22,
3928 + "seed": 4,
3929 + "task_acc": 1.0,
3930 + "control_acc": 0.8958333333333334,
3931 + "selectivity": 0.10416666666666663
3932 + },
3933 + {
3934 + "layer": 23,
3935 + "seed": 0,
3936 + "task_acc": 1.0,
3937 + "control_acc": 0.22916666666666666,
3938 + "selectivity": 0.7708333333333334
3939 + },
3940 + {
3941 + "layer": 23,
3942 + "seed": 1,
3943 + "task_acc": 1.0,
3944 + "control_acc": 0.8541666666666666,
3945 + "selectivity": 0.14583333333333337
3946 + },
3947 + {
3948 + "layer": 23,
3949 + "seed": 2,
3950 + "task_acc": 1.0,
3951 + "control_acc": 0.125,
3952 + "selectivity": 0.875
3953 + },
3954 + {
3955 + "layer": 23,
3956 + "seed": 3,
3957 + "task_acc": 1.0,
3958 + "control_acc": 0.25,
3959 + "selectivity": 0.75
3960 + },
3961 + {
3962 + "layer": 23,
3963 + "seed": 4,
3964 + "task_acc": 1.0,
3965 + "control_acc": 0.16666666666666666,
3966 + "selectivity": 0.8333333333333334
3967 + },
3968 + {
3969 + "layer": 24,
3970 + "seed": 0,
3971 + "task_acc": 1.0,
3972 + "control_acc": 0.3541666666666667,
3973 + "selectivity": 0.6458333333333333
3974 + },
3975 + {
3976 + "layer": 24,
3977 + "seed": 1,
3978 + "task_acc": 1.0,
3979 + "control_acc": 0.08333333333333333,
3980 + "selectivity": 0.9166666666666666
3981 + },
3982 + {
3983 + "layer": 24,
3984 + "seed": 2,
3985 + "task_acc": 1.0,
3986 + "control_acc": 0.0625,
3987 + "selectivity": 0.9375
3988 + },
3989 + {
3990 + "layer": 24,
3991 + "seed": 3,
3992 + "task_acc": 1.0,
3993 + "control_acc": 0.2916666666666667,
3994 + "selectivity": 0.7083333333333333
3995 + },
3996 + {
3997 + "layer": 24,
3998 + "seed": 4,
3999 + "task_acc": 1.0,
4000 + "control_acc": 0.1875,
4001 + "selectivity": 0.8125
4002 + },
4003 + {
4004 + "layer": 25,
4005 + "seed": 0,
4006 + "task_acc": 1.0,
4007 + "control_acc": 0.9791666666666666,
4008 + "selectivity": 0.02083333333333337
4009 + },
4010 + {
4011 + "layer": 25,
4012 + "seed": 1,
4013 + "task_acc": 1.0,
4014 + "control_acc": 0.08333333333333333,
4015 + "selectivity": 0.9166666666666666
4016 + },
4017 + {
4018 + "layer": 25,
4019 + "seed": 2,
4020 + "task_acc": 1.0,
4021 + "control_acc": 0.125,
4022 + "selectivity": 0.875
4023 + },
4024 + {
4025 + "layer": 25,
4026 + "seed": 3,
4027 + "task_acc": 1.0,
4028 + "control_acc": 0.20833333333333334,
4029 + "selectivity": 0.7916666666666666
4030 + },
4031 + {
4032 + "layer": 25,
4033 + "seed": 4,
4034 + "task_acc": 1.0,
4035 + "control_acc": 0.22916666666666666,
4036 + "selectivity": 0.7708333333333334
4037 + },
4038 + {
4039 + "layer": 26,
4040 + "seed": 0,
4041 + "task_acc": 1.0,
4042 + "control_acc": 0.7708333333333334,
4043 + "selectivity": 0.22916666666666663
4044 + },
4045 + {
4046 + "layer": 26,
4047 + "seed": 1,
4048 + "task_acc": 1.0,
4049 + "control_acc": 0.7708333333333334,
4050 + "selectivity": 0.22916666666666663
4051 + },
4052 + {
4053 + "layer": 26,
4054 + "seed": 2,
4055 + "task_acc": 1.0,
4056 + "control_acc": 0.08333333333333333,
4057 + "selectivity": 0.9166666666666666
4058 + },
4059 + {
4060 + "layer": 26,
4061 + "seed": 3,
4062 + "task_acc": 1.0,
4063 + "control_acc": 0.20833333333333334,
4064 + "selectivity": 0.7916666666666666
4065 + },
4066 + {
4067 + "layer": 26,
4068 + "seed": 4,
4069 + "task_acc": 1.0,
4070 + "control_acc": 0.1875,
4071 + "selectivity": 0.8125
4072 + },
4073 + {
4074 + "layer": 27,
4075 + "seed": 0,
4076 + "task_acc": 1.0,
4077 + "control_acc": 0.20833333333333334,
4078 + "selectivity": 0.7916666666666666
4079 + },
4080 + {
4081 + "layer": 27,
4082 + "seed": 1,
4083 + "task_acc": 1.0,
4084 + "control_acc": 0.8125,
4085 + "selectivity": 0.1875
4086 + },
4087 + {
4088 + "layer": 27,
4089 + "seed": 2,
4090 + "task_acc": 1.0,
4091 + "control_acc": 0.4375,
4092 + "selectivity": 0.5625
4093 + },
4094 + {
4095 + "layer": 27,
4096 + "seed": 3,
4097 + "task_acc": 1.0,
4098 + "control_acc": 0.1875,
4099 + "selectivity": 0.8125
4100 + },
4101 + {
4102 + "layer": 27,
4103 + "seed": 4,
4104 + "task_acc": 1.0,
4105 + "control_acc": 0.20833333333333334,
4106 + "selectivity": 0.7916666666666666
4107 + }
4108 + ]
4109 + },
4110 + "code_prose_B": {
4111 + "layers": [
4112 + {
4113 + "layer": 0,
4114 + "task_acc_mean": 1.0,
4115 + "task_acc_seed_sd": 0.0,
4116 + "selectivity_mean": 0.5666666666666667,
4117 + "selectivity_ci": [
4118 + 0.3,
4119 + 0.8333333333333333
4120 + ],
4121 + "p_boot": 0.0001,
4122 + "fdr_significant": true
4123 + },
4124 + {
4125 + "layer": 1,
4126 + "task_acc_mean": 1.0,
4127 + "task_acc_seed_sd": 0.0,
4128 + "selectivity_mean": 0.39583333333333337,
4129 + "selectivity_ci": [
4130 + 0.10833333333333332,
4131 + 0.7250000000000001
4132 + ],
4133 + "p_boot": 0.0001,
4134 + "fdr_significant": true
4135 + },
4136 + {
4137 + "layer": 2,
4138 + "task_acc_mean": 1.0,
4139 + "task_acc_seed_sd": 0.0,
4140 + "selectivity_mean": 0.4125,
4141 + "selectivity_ci": [
4142 + 0.1,
4143 + 0.775
4144 + ],
4145 + "p_boot": 0.0001,
4146 + "fdr_significant": true
4147 + },
4148 + {
4149 + "layer": 3,
4150 + "task_acc_mean": 1.0,
4151 + "task_acc_seed_sd": 0.0,
4152 + "selectivity_mean": 0.4958333333333333,
4153 + "selectivity_ci": [
4154 + 0.2041666666666667,
4155 + 0.7749999999999999
4156 + ],
4157 + "p_boot": 0.0001,
4158 + "fdr_significant": true
4159 + },
4160 + {
4161 + "layer": 4,
4162 + "task_acc_mean": 1.0,
4163 + "task_acc_seed_sd": 0.0,
4164 + "selectivity_mean": 0.4208333333333334,
4165 + "selectivity_ci": [
4166 + 0.10416666666666667,
4167 + 0.7583333333333334
4168 + ],
4169 + "p_boot": 0.0001,
4170 + "fdr_significant": true
4171 + },
4172 + {
4173 + "layer": 5,
4174 + "task_acc_mean": 1.0,
4175 + "task_acc_seed_sd": 0.0,
4176 + "selectivity_mean": 0.4208333333333333,
4177 + "selectivity_ci": [
4178 + 0.14999999999999997,
4179 + 0.7208333333333333
4180 + ],
4181 + "p_boot": 0.0001,
4182 + "fdr_significant": true
4183 + },
4184 + {
4185 + "layer": 6,
4186 + "task_acc_mean": 1.0,
4187 + "task_acc_seed_sd": 0.0,
4188 + "selectivity_mean": 0.475,
4189 + "selectivity_ci": [
4190 + 0.2333333333333333,
4191 + 0.7375
4192 + ],
4193 + "p_boot": 0.0001,
4194 + "fdr_significant": true
4195 + },
4196 + {
4197 + "layer": 7,
4198 + "task_acc_mean": 1.0,
4199 + "task_acc_seed_sd": 0.0,
4200 + "selectivity_mean": 0.2666666666666667,
4201 + "selectivity_ci": [
4202 + 0.08333333333333334,
4203 + 0.5166666666666668
4204 + ],
4205 + "p_boot": 0.0001,
4206 + "fdr_significant": true
4207 + },
4208 + {
4209 + "layer": 8,
4210 + "task_acc_mean": 1.0,
4211 + "task_acc_seed_sd": 0.0,
4212 + "selectivity_mean": 0.2666666666666667,
4213 + "selectivity_ci": [
4214 + 0.07916666666666668,
4215 + 0.5333333333333333
4216 + ],
4217 + "p_boot": 0.0001,
4218 + "fdr_significant": true
4219 + },
4220 + {
4221 + "layer": 9,
4222 + "task_acc_mean": 1.0,
4223 + "task_acc_seed_sd": 0.0,
4224 + "selectivity_mean": 0.5666666666666667,
4225 + "selectivity_ci": [
4226 + 0.2666666666666666,
4227 + 0.8666666666666666
4228 + ],
4229 + "p_boot": 0.0001,
4230 + "fdr_significant": true
4231 + },
4232 + {
4233 + "layer": 10,
4234 + "task_acc_mean": 1.0,
4235 + "task_acc_seed_sd": 0.0,
4236 + "selectivity_mean": 0.4333333333333333,
4237 + "selectivity_ci": [
4238 + 0.19583333333333333,
4239 + 0.6708333333333333
4240 + ],
4241 + "p_boot": 0.0001,
4242 + "fdr_significant": true
4243 + },
4244 + {
4245 + "layer": 11,
4246 + "task_acc_mean": 1.0,
4247 + "task_acc_seed_sd": 0.0,
4248 + "selectivity_mean": 0.3916666666666667,
4249 + "selectivity_ci": [
4250 + 0.1125,
4251 + 0.675
4252 + ],
4253 + "p_boot": 0.0001,
4254 + "fdr_significant": true
4255 + },
4256 + {
4257 + "layer": 12,
4258 + "task_acc_mean": 1.0,
4259 + "task_acc_seed_sd": 0.0,
4260 + "selectivity_mean": 0.5416666666666666,
4261 + "selectivity_ci": [
4262 + 0.3208333333333334,
4263 + 0.7625
4264 + ],
4265 + "p_boot": 0.0001,
4266 + "fdr_significant": true
4267 + },
4268 + {
4269 + "layer": 13,
4270 + "task_acc_mean": 1.0,
4271 + "task_acc_seed_sd": 0.0,
4272 + "selectivity_mean": 0.4166666666666667,
4273 + "selectivity_ci": [
4274 + 0.09166666666666667,
4275 + 0.7416666666666667
4276 + ],
4277 + "p_boot": 0.0001,
4278 + "fdr_significant": true
4279 + },
4280 + {
4281 + "layer": 14,
4282 + "task_acc_mean": 1.0,
4283 + "task_acc_seed_sd": 0.0,
4284 + "selectivity_mean": 0.3749999999999999,
4285 + "selectivity_ci": [
4286 + 0.14166666666666664,
4287 + 0.6791666666666666
4288 + ],
4289 + "p_boot": 0.0001,
4290 + "fdr_significant": true
4291 + },
4292 + {
4293 + "layer": 15,
4294 + "task_acc_mean": 1.0,
4295 + "task_acc_seed_sd": 0.0,
4296 + "selectivity_mean": 0.2291666666666666,
4297 + "selectivity_ci": [
4298 + 0.07500000000000002,
4299 + 0.4749999999999999
4300 + ],
4301 + "p_boot": 0.0001,
4302 + "fdr_significant": true
4303 + },
4304 + {
4305 + "layer": 16,
4306 + "task_acc_mean": 1.0,
4307 + "task_acc_seed_sd": 0.0,
4308 + "selectivity_mean": 0.4041666666666667,
4309 + "selectivity_ci": [
4310 + 0.14583333333333334,
4311 + 0.6625
4312 + ],
4313 + "p_boot": 0.0001,
4314 + "fdr_significant": true
4315 + },
4316 + {
4317 + "layer": 17,
4318 + "task_acc_mean": 1.0,
4319 + "task_acc_seed_sd": 0.0,
4320 + "selectivity_mean": 0.3125,
4321 + "selectivity_ci": [
4322 + 0.12916666666666668,
4323 + 0.6125
4324 + ],
4325 + "p_boot": 0.0001,
4326 + "fdr_significant": true
4327 + },
4328 + {
4329 + "layer": 18,
4330 + "task_acc_mean": 1.0,
4331 + "task_acc_seed_sd": 0.0,
4332 + "selectivity_mean": 0.5375,
4333 + "selectivity_ci": [
4334 + 0.2083333333333334,
4335 + 0.8625
4336 + ],
4337 + "p_boot": 0.0001,
4338 + "fdr_significant": true
4339 + },
4340 + {
4341 + "layer": 19,
4342 + "task_acc_mean": 1.0,
4343 + "task_acc_seed_sd": 0.0,
4344 + "selectivity_mean": 0.4833333333333333,
4345 + "selectivity_ci": [
4346 + 0.15833333333333335,
4347 + 0.8208333333333334
4348 + ],
4349 + "p_boot": 0.0006,
4350 + "fdr_significant": true
4351 + },
4352 + {
4353 + "layer": 20,
4354 + "task_acc_mean": 1.0,
4355 + "task_acc_seed_sd": 0.0,
4356 + "selectivity_mean": 0.5166666666666667,
4357 + "selectivity_ci": [
4358 + 0.2541666666666667,
4359 + 0.7791666666666667
4360 + ],
4361 + "p_boot": 0.0001,
4362 + "fdr_significant": true
4363 + },
4364 + {
4365 + "layer": 21,
4366 + "task_acc_mean": 1.0,
4367 + "task_acc_seed_sd": 0.0,
4368 + "selectivity_mean": 0.6875,
4369 + "selectivity_ci": [
4370 + 0.39583333333333337,
4371 + 0.8958333333333334
4372 + ],
4373 + "p_boot": 0.0001,
4374 + "fdr_significant": true
4375 + },
4376 + {
4377 + "layer": 22,
4378 + "task_acc_mean": 1.0,
4379 + "task_acc_seed_sd": 0.0,
4380 + "selectivity_mean": 0.5666666666666667,
4381 + "selectivity_ci": [
4382 + 0.2875000000000001,
4383 + 0.8458333333333332
4384 + ],
4385 + "p_boot": 0.0001,
4386 + "fdr_significant": true
4387 + },
4388 + {
4389 + "layer": 23,
4390 + "task_acc_mean": 1.0,
4391 + "task_acc_seed_sd": 0.0,
4392 + "selectivity_mean": 0.6416666666666667,
4393 + "selectivity_ci": [
4394 + 0.3166666666666667,
4395 + 0.8583333333333334
4396 + ],
4397 + "p_boot": 0.0001,
4398 + "fdr_significant": true
4399 + },
4400 + {
4401 + "layer": 24,
4402 + "task_acc_mean": 1.0,
4403 + "task_acc_seed_sd": 0.0,
4404 + "selectivity_mean": 0.5458333333333334,
4405 + "selectivity_ci": [
4406 + 0.2833333333333334,
4407 + 0.8083333333333333
4408 + ],
4409 + "p_boot": 0.0001,
4410 + "fdr_significant": true
4411 + },
4412 + {
4413 + "layer": 25,
4414 + "task_acc_mean": 1.0,
4415 + "task_acc_seed_sd": 0.0,
4416 + "selectivity_mean": 0.6875000000000001,
4417 + "selectivity_ci": [
4418 + 0.4625000000000001,
4419 + 0.8666666666666666
4420 + ],
4421 + "p_boot": 0.0001,
4422 + "fdr_significant": true
4423 + },
4424 + {
4425 + "layer": 26,
4426 + "task_acc_mean": 1.0,
4427 + "task_acc_seed_sd": 0.0,
4428 + "selectivity_mean": 0.6499999999999999,
4429 + "selectivity_ci": [
4430 + 0.3125000000000001,
4431 + 0.8875
4432 + ],
4433 + "p_boot": 0.0001,
4434 + "fdr_significant": true
4435 + },
4436 + {
4437 + "layer": 27,
4438 + "task_acc_mean": 1.0,
4439 + "task_acc_seed_sd": 0.0,
4440 + "selectivity_mean": 0.5208333333333334,
4441 + "selectivity_ci": [
4442 + 0.2041666666666667,
4443 + 0.8375
4444 + ],
4445 + "p_boot": 0.0001,
4446 + "fdr_significant": true
4447 + }
4448 + ],
4449 + "replication_rate_topk": {
4450 + "k": 5,
4451 + "point": 1.0,
4452 + "ci": [
4453 + 1.0,
4454 + 1.0
4455 + ]
4456 + },
4457 + "n_fdr_significant": 28,
4458 + "cells": [
4459 + {
4460 + "layer": 0,
4461 + "seed": 0,
4462 + "task_acc": 1.0,
4463 + "control_acc": 0.125,
4464 + "selectivity": 0.875
4465 + },
4466 + {
4467 + "layer": 0,
4468 + "seed": 1,
4469 + "task_acc": 1.0,
4470 + "control_acc": 0.8333333333333334,
4471 + "selectivity": 0.16666666666666663
4472 + },
4473 + {
4474 + "layer": 0,
4475 + "seed": 2,
4476 + "task_acc": 1.0,
4477 + "control_acc": 0.20833333333333334,
4478 + "selectivity": 0.7916666666666666
4479 + },
4480 + {
4481 + "layer": 0,
4482 + "seed": 3,
4483 + "task_acc": 1.0,
4484 + "control_acc": 0.8125,
4485 + "selectivity": 0.1875
4486 + },
4487 + {
4488 + "layer": 0,
4489 + "seed": 4,
4490 + "task_acc": 1.0,
4491 + "control_acc": 0.1875,
4492 + "selectivity": 0.8125
4493 + },
4494 + {
4495 + "layer": 1,
4496 + "seed": 0,
4497 + "task_acc": 1.0,
4498 + "control_acc": 0.8333333333333334,
4499 + "selectivity": 0.16666666666666663
4500 + },
4501 + {
4502 + "layer": 1,
4503 + "seed": 1,
4504 + "task_acc": 1.0,
4505 + "control_acc": 0.8958333333333334,
4506 + "selectivity": 0.10416666666666663
4507 + },
4508 + {
4509 + "layer": 1,
4510 + "seed": 2,
4511 + "task_acc": 1.0,
4512 + "control_acc": 0.9166666666666666,
4513 + "selectivity": 0.08333333333333337
4514 + },
4515 + {
4516 + "layer": 1,
4517 + "seed": 3,
4518 + "task_acc": 1.0,
4519 + "control_acc": 0.3333333333333333,
4520 + "selectivity": 0.6666666666666667
4521 + },
4522 + {
4523 + "layer": 1,
4524 + "seed": 4,
4525 + "task_acc": 1.0,
4526 + "control_acc": 0.041666666666666664,
4527 + "selectivity": 0.9583333333333334
4528 + },
4529 + {
4530 + "layer": 2,
4531 + "seed": 0,
4532 + "task_acc": 1.0,
4533 + "control_acc": 0.875,
4534 + "selectivity": 0.125
4535 + },
4536 + {
4537 + "layer": 2,
4538 + "seed": 1,
4539 + "task_acc": 1.0,
4540 + "control_acc": 0.875,
4541 + "selectivity": 0.125
4542 + },
4543 + {
4544 + "layer": 2,
4545 + "seed": 2,
4546 + "task_acc": 1.0,
4547 + "control_acc": 0.25,
4548 + "selectivity": 0.75
4549 + },
4550 + {
4551 + "layer": 2,
4552 + "seed": 3,
4553 + "task_acc": 1.0,
4554 + "control_acc": 0.9375,
4555 + "selectivity": 0.0625
4556 + },
4557 + {
4558 + "layer": 2,
4559 + "seed": 4,
4560 + "task_acc": 1.0,
4561 + "control_acc": 0.0,
4562 + "selectivity": 1.0
4563 + },
4564 + {
4565 + "layer": 3,
4566 + "seed": 0,
4567 + "task_acc": 1.0,
4568 + "control_acc": 0.8333333333333334,
4569 + "selectivity": 0.16666666666666663
4570 + },
4571 + {
4572 + "layer": 3,
4573 + "seed": 1,
4574 + "task_acc": 1.0,
4575 + "control_acc": 0.9791666666666666,
4576 + "selectivity": 0.02083333333333337
4577 + },
4578 + {
4579 + "layer": 3,
4580 + "seed": 2,
4581 + "task_acc": 1.0,
4582 + "control_acc": 0.20833333333333334,
4583 + "selectivity": 0.7916666666666666
4584 + },
4585 + {
4586 + "layer": 3,
4587 + "seed": 3,
4588 + "task_acc": 1.0,
4589 + "control_acc": 0.2916666666666667,
4590 + "selectivity": 0.7083333333333333
4591 + },
4592 + {
4593 + "layer": 3,
4594 + "seed": 4,
4595 + "task_acc": 1.0,
4596 + "control_acc": 0.20833333333333334,
4597 + "selectivity": 0.7916666666666666
4598 + },
4599 + {
4600 + "layer": 4,
4601 + "seed": 0,
4602 + "task_acc": 1.0,
4603 + "control_acc": 0.8958333333333334,
4604 + "selectivity": 0.10416666666666663
4605 + },
4606 + {
4607 + "layer": 4,
4608 + "seed": 1,
4609 + "task_acc": 1.0,
4610 + "control_acc": 0.9166666666666666,
4611 + "selectivity": 0.08333333333333337
4612 + },
4613 + {
4614 + "layer": 4,
4615 + "seed": 2,
4616 + "task_acc": 1.0,
4617 + "control_acc": 0.1875,
4618 + "selectivity": 0.8125
4619 + },
4620 + {
4621 + "layer": 4,
4622 + "seed": 3,
4623 + "task_acc": 1.0,
4624 + "control_acc": 0.8541666666666666,
4625 + "selectivity": 0.14583333333333337
4626 + },
4627 + {
4628 + "layer": 4,
4629 + "seed": 4,
4630 + "task_acc": 1.0,
4631 + "control_acc": 0.041666666666666664,
4632 + "selectivity": 0.9583333333333334
4633 + },
4634 + {
4635 + "layer": 5,
4636 + "seed": 0,
4637 + "task_acc": 1.0,
4638 + "control_acc": 0.8333333333333334,
4639 + "selectivity": 0.16666666666666663
4640 + },
4641 + {
4642 + "layer": 5,
4643 + "seed": 1,
4644 + "task_acc": 1.0,
4645 + "control_acc": 0.8333333333333334,
4646 + "selectivity": 0.16666666666666663
4647 + },
4648 + {
4649 + "layer": 5,
4650 + "seed": 2,
4651 + "task_acc": 1.0,
4652 + "control_acc": 0.25,
4653 + "selectivity": 0.75
4654 + },
4655 + {
4656 + "layer": 5,
4657 + "seed": 3,
4658 + "task_acc": 1.0,
4659 + "control_acc": 0.875,
4660 + "selectivity": 0.125
4661 + },
4662 + {
4663 + "layer": 5,
4664 + "seed": 4,
4665 + "task_acc": 1.0,
4666 + "control_acc": 0.10416666666666667,
4667 + "selectivity": 0.8958333333333334
4668 + },
4669 + {
4670 + "layer": 6,
4671 + "seed": 0,
4672 + "task_acc": 1.0,
4673 + "control_acc": 0.7083333333333334,
4674 + "selectivity": 0.29166666666666663
4675 + },
4676 + {
4677 + "layer": 6,
4678 + "seed": 1,
4679 + "task_acc": 1.0,
4680 + "control_acc": 0.8958333333333334,
4681 + "selectivity": 0.10416666666666663
4682 + },
4683 + {
4684 + "layer": 6,
4685 + "seed": 2,
4686 + "task_acc": 1.0,
4687 + "control_acc": 0.2916666666666667,
4688 + "selectivity": 0.7083333333333333
4689 + },
4690 + {
4691 + "layer": 6,
4692 + "seed": 3,
4693 + "task_acc": 1.0,
4694 + "control_acc": 0.625,
4695 + "selectivity": 0.375
4696 + },
4697 + {
4698 + "layer": 6,
4699 + "seed": 4,
4700 + "task_acc": 1.0,
4701 + "control_acc": 0.10416666666666667,
4702 + "selectivity": 0.8958333333333334
4703 + },
4704 + {
4705 + "layer": 7,
4706 + "seed": 0,
4707 + "task_acc": 1.0,
4708 + "control_acc": 0.875,
4709 + "selectivity": 0.125
4710 + },
4711 + {
4712 + "layer": 7,
4713 + "seed": 1,
4714 + "task_acc": 1.0,
4715 + "control_acc": 0.9375,
4716 + "selectivity": 0.0625
4717 + },
4718 + {
4719 + "layer": 7,
4720 + "seed": 2,
4721 + "task_acc": 1.0,
4722 + "control_acc": 0.2708333333333333,
4723 + "selectivity": 0.7291666666666667
4724 + },
4725 + {
4726 + "layer": 7,
4727 + "seed": 3,
4728 + "task_acc": 1.0,
4729 + "control_acc": 0.6666666666666666,
4730 + "selectivity": 0.33333333333333337
4731 + },
4732 + {
4733 + "layer": 7,
4734 + "seed": 4,
4735 + "task_acc": 1.0,
4736 + "control_acc": 0.9166666666666666,
4737 + "selectivity": 0.08333333333333337
4738 + },
4739 + {
4740 + "layer": 8,
4741 + "seed": 0,
4742 + "task_acc": 1.0,
4743 + "control_acc": 0.8541666666666666,
4744 + "selectivity": 0.14583333333333337
4745 + },
4746 + {
4747 + "layer": 8,
4748 + "seed": 1,
4749 + "task_acc": 1.0,
4750 + "control_acc": 0.8958333333333334,
4751 + "selectivity": 0.10416666666666663
4752 + },
4753 + {
4754 + "layer": 8,
4755 + "seed": 2,
4756 + "task_acc": 1.0,
4757 + "control_acc": 0.20833333333333334,
4758 + "selectivity": 0.7916666666666666
4759 + },
4760 + {
4761 + "layer": 8,
4762 + "seed": 3,
4763 + "task_acc": 1.0,
4764 + "control_acc": 0.7291666666666666,
4765 + "selectivity": 0.27083333333333337
4766 + },
4767 + {
4768 + "layer": 8,
4769 + "seed": 4,
4770 + "task_acc": 1.0,
4771 + "control_acc": 0.9791666666666666,
4772 + "selectivity": 0.02083333333333337
4773 + },
4774 + {
4775 + "layer": 9,
4776 + "seed": 0,
4777 + "task_acc": 1.0,
4778 + "control_acc": 0.8958333333333334,
4779 + "selectivity": 0.10416666666666663
4780 + },
4781 + {
4782 + "layer": 9,
4783 + "seed": 1,
4784 + "task_acc": 1.0,
4785 + "control_acc": 0.8125,
4786 + "selectivity": 0.1875
4787 + },
4788 + {
4789 + "layer": 9,
4790 + "seed": 2,
4791 + "task_acc": 1.0,
4792 + "control_acc": 0.25,
4793 + "selectivity": 0.75
4794 + },
4795 + {
4796 + "layer": 9,
4797 + "seed": 3,
4798 + "task_acc": 1.0,
4799 + "control_acc": 0.125,
4800 + "selectivity": 0.875
4801 + },
4802 + {
4803 + "layer": 9,
4804 + "seed": 4,
4805 + "task_acc": 1.0,
4806 + "control_acc": 0.08333333333333333,
4807 + "selectivity": 0.9166666666666666
4808 + },
4809 + {
4810 + "layer": 10,
4811 + "seed": 0,
4812 + "task_acc": 1.0,
4813 + "control_acc": 0.875,
4814 + "selectivity": 0.125
4815 + },
4816 + {
4817 + "layer": 10,
4818 + "seed": 1,
4819 + "task_acc": 1.0,
4820 + "control_acc": 0.875,
4821 + "selectivity": 0.125
4822 + },
4823 + {
4824 + "layer": 10,
4825 + "seed": 2,
4826 + "task_acc": 1.0,
4827 + "control_acc": 0.2708333333333333,
4828 + "selectivity": 0.7291666666666667
4829 + },
4830 + {
4831 + "layer": 10,
4832 + "seed": 3,
4833 + "task_acc": 1.0,
4834 + "control_acc": 0.2916666666666667,
4835 + "selectivity": 0.7083333333333333
4836 + },
4837 + {
4838 + "layer": 10,
4839 + "seed": 4,
4840 + "task_acc": 1.0,
4841 + "control_acc": 0.5208333333333334,
4842 + "selectivity": 0.47916666666666663
4843 + },
4844 + {
4845 + "layer": 11,
4846 + "seed": 0,
4847 + "task_acc": 1.0,
4848 + "control_acc": 0.875,
4849 + "selectivity": 0.125
4850 + },
4851 + {
4852 + "layer": 11,
4853 + "seed": 1,
4854 + "task_acc": 1.0,
4855 + "control_acc": 0.8125,
4856 + "selectivity": 0.1875
4857 + },
4858 + {
4859 + "layer": 11,
4860 + "seed": 2,
4861 + "task_acc": 1.0,
4862 + "control_acc": 0.25,
4863 + "selectivity": 0.75
4864 + },
4865 + {
4866 + "layer": 11,
4867 + "seed": 3,
4868 + "task_acc": 1.0,
4869 + "control_acc": 0.16666666666666666,
4870 + "selectivity": 0.8333333333333334
4871 + },
4872 + {
4873 + "layer": 11,
4874 + "seed": 4,
4875 + "task_acc": 1.0,
4876 + "control_acc": 0.9375,
4877 + "selectivity": 0.0625
4878 + },
4879 + {
4880 + "layer": 12,
4881 + "seed": 0,
4882 + "task_acc": 1.0,
4883 + "control_acc": 0.16666666666666666,
4884 + "selectivity": 0.8333333333333334
4885 + },
4886 + {
4887 + "layer": 12,
4888 + "seed": 1,
4889 + "task_acc": 1.0,
4890 + "control_acc": 0.8541666666666666,
4891 + "selectivity": 0.14583333333333337
4892 + },
4893 + {
4894 + "layer": 12,
4895 + "seed": 2,
4896 + "task_acc": 1.0,
4897 + "control_acc": 0.1875,
4898 + "selectivity": 0.8125
4899 + },
4900 + {
4901 + "layer": 12,
4902 + "seed": 3,
4903 + "task_acc": 1.0,
4904 + "control_acc": 0.6041666666666666,
4905 + "selectivity": 0.39583333333333337
4906 + },
4907 + {
4908 + "layer": 12,
4909 + "seed": 4,
4910 + "task_acc": 1.0,
4911 + "control_acc": 0.4791666666666667,
4912 + "selectivity": 0.5208333333333333
4913 + },
4914 + {
4915 + "layer": 13,
4916 + "seed": 0,
4917 + "task_acc": 1.0,
4918 + "control_acc": 0.125,
4919 + "selectivity": 0.875
4920 + },
4921 + {
4922 + "layer": 13,
4923 + "seed": 1,
4924 + "task_acc": 1.0,
4925 + "control_acc": 0.9166666666666666,
4926 + "selectivity": 0.08333333333333337
4927 + },
4928 + {
4929 + "layer": 13,
4930 + "seed": 2,
4931 + "task_acc": 1.0,
4932 + "control_acc": 0.10416666666666667,
4933 + "selectivity": 0.8958333333333334
4934 + },
4935 + {
4936 + "layer": 13,
4937 + "seed": 3,
4938 + "task_acc": 1.0,
4939 + "control_acc": 0.8333333333333334,
4940 + "selectivity": 0.16666666666666663
4941 + },
4942 + {
4943 + "layer": 13,
4944 + "seed": 4,
4945 + "task_acc": 1.0,
4946 + "control_acc": 0.9375,
4947 + "selectivity": 0.0625
4948 + },
4949 + {
4950 + "layer": 14,
4951 + "seed": 0,
4952 + "task_acc": 1.0,
4953 + "control_acc": 0.08333333333333333,
4954 + "selectivity": 0.9166666666666666
4955 + },
4956 + {
4957 + "layer": 14,
4958 + "seed": 1,
4959 + "task_acc": 1.0,
4960 + "control_acc": 0.8333333333333334,
4961 + "selectivity": 0.16666666666666663
4962 + },
4963 + {
4964 + "layer": 14,
4965 + "seed": 2,
4966 + "task_acc": 1.0,
4967 + "control_acc": 0.4791666666666667,
4968 + "selectivity": 0.5208333333333333
4969 + },
4970 + {
4971 + "layer": 14,
4972 + "seed": 3,
4973 + "task_acc": 1.0,
4974 + "control_acc": 0.8333333333333334,
4975 + "selectivity": 0.16666666666666663
4976 + },
4977 + {
4978 + "layer": 14,
4979 + "seed": 4,
4980 + "task_acc": 1.0,
4981 + "control_acc": 0.8958333333333334,
4982 + "selectivity": 0.10416666666666663
4983 + },
4984 + {
4985 + "layer": 15,
4986 + "seed": 0,
4987 + "task_acc": 1.0,
4988 + "control_acc": 0.2916666666666667,
4989 + "selectivity": 0.7083333333333333
4990 + },
4991 + {
4992 + "layer": 15,
4993 + "seed": 1,
4994 + "task_acc": 1.0,
4995 + "control_acc": 0.8333333333333334,
4996 + "selectivity": 0.16666666666666663
4997 + },
4998 + {
4999 + "layer": 15,
5000 + "seed": 2,
5001 + "task_acc": 1.0,
5002 + "control_acc": 0.9166666666666666,
5003 + "selectivity": 0.08333333333333337
5004 + },
5005 + {
5006 + "layer": 15,
5007 + "seed": 3,
5008 + "task_acc": 1.0,
5009 + "control_acc": 0.8541666666666666,
5010 + "selectivity": 0.14583333333333337
5011 + },
5012 + {
5013 + "layer": 15,
5014 + "seed": 4,
5015 + "task_acc": 1.0,
5016 + "control_acc": 0.9583333333333334,
5017 + "selectivity": 0.04166666666666663
5018 + },
5019 + {
5020 + "layer": 16,
5021 + "seed": 0,
5022 + "task_acc": 1.0,
5023 + "control_acc": 0.25,
5024 + "selectivity": 0.75
5025 + },
5026 + {
5027 + "layer": 16,
5028 + "seed": 1,
5029 + "task_acc": 1.0,
5030 + "control_acc": 0.9375,
5031 + "selectivity": 0.0625
5032 + },
5033 + {
5034 + "layer": 16,
5035 + "seed": 2,
5036 + "task_acc": 1.0,
5037 + "control_acc": 0.25,
5038 + "selectivity": 0.75
5039 + },
5040 + {
5041 + "layer": 16,
5042 + "seed": 3,
5043 + "task_acc": 1.0,
5044 + "control_acc": 0.8541666666666666,
5045 + "selectivity": 0.14583333333333337
5046 + },
5047 + {
5048 + "layer": 16,
5049 + "seed": 4,
5050 + "task_acc": 1.0,
5051 + "control_acc": 0.6875,
5052 + "selectivity": 0.3125
5053 + },
5054 + {
5055 + "layer": 17,
5056 + "seed": 0,
5057 + "task_acc": 1.0,
5058 + "control_acc": 0.8958333333333334,
5059 + "selectivity": 0.10416666666666663
5060 + },
5061 + {
5062 + "layer": 17,
5063 + "seed": 1,
5064 + "task_acc": 1.0,
5065 + "control_acc": 0.8541666666666666,
5066 + "selectivity": 0.14583333333333337
5067 + },
5068 + {
5069 + "layer": 17,
5070 + "seed": 2,
5071 + "task_acc": 1.0,
5072 + "control_acc": 0.7291666666666666,
5073 + "selectivity": 0.27083333333333337
5074 + },
5075 + {
5076 + "layer": 17,
5077 + "seed": 3,
5078 + "task_acc": 1.0,
5079 + "control_acc": 0.8541666666666666,
5080 + "selectivity": 0.14583333333333337
5081 + },
5082 + {
5083 + "layer": 17,
5084 + "seed": 4,
5085 + "task_acc": 1.0,
5086 + "control_acc": 0.10416666666666667,
5087 + "selectivity": 0.8958333333333334
5088 + },
5089 + {
5090 + "layer": 18,
5091 + "seed": 0,
5092 + "task_acc": 1.0,
5093 + "control_acc": 0.08333333333333333,
5094 + "selectivity": 0.9166666666666666
5095 + },
5096 + {
5097 + "layer": 18,
5098 + "seed": 1,
5099 + "task_acc": 1.0,
5100 + "control_acc": 0.875,
5101 + "selectivity": 0.125
5102 + },
5103 + {
5104 + "layer": 18,
5105 + "seed": 2,
5106 + "task_acc": 1.0,
5107 + "control_acc": 0.14583333333333334,
5108 + "selectivity": 0.8541666666666666
5109 + },
5110 + {
5111 + "layer": 18,
5112 + "seed": 3,
5113 + "task_acc": 1.0,
5114 + "control_acc": 0.22916666666666666,
5115 + "selectivity": 0.7708333333333334
5116 + },
5117 + {
5118 + "layer": 18,
5119 + "seed": 4,
5120 + "task_acc": 1.0,
5121 + "control_acc": 0.9791666666666666,
5122 + "selectivity": 0.02083333333333337
5123 + },
5124 + {
5125 + "layer": 19,
5126 + "seed": 0,
5127 + "task_acc": 1.0,
5128 + "control_acc": 0.10416666666666667,
5129 + "selectivity": 0.8958333333333334
5130 + },
5131 + {
5132 + "layer": 19,
5133 + "seed": 1,
5134 + "task_acc": 1.0,
5135 + "control_acc": 1.0,
5136 + "selectivity": 0.0
5137 + },
5138 + {
5139 + "layer": 19,
5140 + "seed": 2,
5141 + "task_acc": 1.0,
5142 + "control_acc": 0.3958333333333333,
5143 + "selectivity": 0.6041666666666667
5144 + },
5145 + {
5146 + "layer": 19,
5147 + "seed": 3,
5148 + "task_acc": 1.0,
5149 + "control_acc": 0.14583333333333334,
5150 + "selectivity": 0.8541666666666666
5151 + },
5152 + {
5153 + "layer": 19,
5154 + "seed": 4,
5155 + "task_acc": 1.0,
5156 + "control_acc": 0.9375,
5157 + "selectivity": 0.0625
5158 + },
5159 + {
5160 + "layer": 20,
5161 + "seed": 0,
5162 + "task_acc": 1.0,
5163 + "control_acc": 0.25,
5164 + "selectivity": 0.75
5165 + },
5166 + {
5167 + "layer": 20,
5168 + "seed": 1,
5169 + "task_acc": 1.0,
5170 + "control_acc": 0.8541666666666666,
5171 + "selectivity": 0.14583333333333337
5172 + },
5173 + {
5174 + "layer": 20,
5175 + "seed": 2,
5176 + "task_acc": 1.0,
5177 + "control_acc": 0.3958333333333333,
5178 + "selectivity": 0.6041666666666667
5179 + },
5180 + {
5181 + "layer": 20,
5182 + "seed": 3,
5183 + "task_acc": 1.0,
5184 + "control_acc": 0.10416666666666667,
5185 + "selectivity": 0.8958333333333334
5186 + },
5187 + {
5188 + "layer": 20,
5189 + "seed": 4,
5190 + "task_acc": 1.0,
5191 + "control_acc": 0.8125,
5192 + "selectivity": 0.1875
5193 + },
5194 + {
5195 + "layer": 21,
5196 + "seed": 0,
5197 + "task_acc": 1.0,
5198 + "control_acc": 0.22916666666666666,
5199 + "selectivity": 0.7708333333333334
5200 + },
5201 + {
5202 + "layer": 21,
5203 + "seed": 1,
5204 + "task_acc": 1.0,
5205 + "control_acc": 0.8541666666666666,
5206 + "selectivity": 0.14583333333333337
5207 + },
5208 + {
5209 + "layer": 21,
5210 + "seed": 2,
5211 + "task_acc": 1.0,
5212 + "control_acc": 0.1875,
5213 + "selectivity": 0.8125
5214 + },
5215 + {
5216 + "layer": 21,
5217 + "seed": 3,
5218 + "task_acc": 1.0,
5219 + "control_acc": 0.0,
5220 + "selectivity": 1.0
5221 + },
5222 + {
5223 + "layer": 21,
5224 + "seed": 4,
5225 + "task_acc": 1.0,
5226 + "control_acc": 0.2916666666666667,
5227 + "selectivity": 0.7083333333333333
5228 + },
5229 + {
5230 + "layer": 22,
5231 + "seed": 0,
5232 + "task_acc": 1.0,
5233 + "control_acc": 0.16666666666666666,
5234 + "selectivity": 0.8333333333333334
5235 + },
5236 + {
5237 + "layer": 22,
5238 + "seed": 1,
5239 + "task_acc": 1.0,
5240 + "control_acc": 0.8541666666666666,
5241 + "selectivity": 0.14583333333333337
5242 + },
5243 + {
5244 + "layer": 22,
5245 + "seed": 2,
5246 + "task_acc": 1.0,
5247 + "control_acc": 0.2708333333333333,
5248 + "selectivity": 0.7291666666666667
5249 + },
5250 + {
5251 + "layer": 22,
5252 + "seed": 3,
5253 + "task_acc": 1.0,
5254 + "control_acc": 0.7916666666666666,
5255 + "selectivity": 0.20833333333333337
5256 + },
5257 + {
5258 + "layer": 22,
5259 + "seed": 4,
5260 + "task_acc": 1.0,
5261 + "control_acc": 0.08333333333333333,
5262 + "selectivity": 0.9166666666666666
5263 + },
5264 + {
5265 + "layer": 23,
5266 + "seed": 0,
5267 + "task_acc": 1.0,
5268 + "control_acc": 0.16666666666666666,
5269 + "selectivity": 0.8333333333333334
5270 + },
5271 + {
5272 + "layer": 23,
5273 + "seed": 1,
5274 + "task_acc": 1.0,
5275 + "control_acc": 0.9791666666666666,
5276 + "selectivity": 0.02083333333333337
5277 + },
5278 + {
5279 + "layer": 23,
5280 + "seed": 2,
5281 + "task_acc": 1.0,
5282 + "control_acc": 0.375,
5283 + "selectivity": 0.625
5284 + },
5285 + {
5286 + "layer": 23,
5287 + "seed": 3,
5288 + "task_acc": 1.0,
5289 + "control_acc": 0.10416666666666667,
5290 + "selectivity": 0.8958333333333334
5291 + },
5292 + {
5293 + "layer": 23,
5294 + "seed": 4,
5295 + "task_acc": 1.0,
5296 + "control_acc": 0.16666666666666666,
5297 + "selectivity": 0.8333333333333334
5298 + },
5299 + {
5300 + "layer": 24,
5301 + "seed": 0,
5302 + "task_acc": 1.0,
5303 + "control_acc": 0.16666666666666666,
5304 + "selectivity": 0.8333333333333334
5305 + },
5306 + {
5307 + "layer": 24,
5308 + "seed": 1,
5309 + "task_acc": 1.0,
5310 + "control_acc": 0.8541666666666666,
5311 + "selectivity": 0.14583333333333337
5312 + },
5313 + {
5314 + "layer": 24,
5315 + "seed": 2,
5316 + "task_acc": 1.0,
5317 + "control_acc": 0.2916666666666667,
5318 + "selectivity": 0.7083333333333333
5319 + },
5320 + {
5321 + "layer": 24,
5322 + "seed": 3,
5323 + "task_acc": 1.0,
5324 + "control_acc": 0.7916666666666666,
5325 + "selectivity": 0.20833333333333337
5326 + },
5327 + {
5328 + "layer": 24,
5329 + "seed": 4,
5330 + "task_acc": 1.0,
5331 + "control_acc": 0.16666666666666666,
5332 + "selectivity": 0.8333333333333334
5333 + },
5334 + {
5335 + "layer": 25,
5336 + "seed": 0,
5337 + "task_acc": 1.0,
5338 + "control_acc": 0.25,
5339 + "selectivity": 0.75
5340 + },
5341 + {
5342 + "layer": 25,
5343 + "seed": 1,
5344 + "task_acc": 1.0,
5345 + "control_acc": 0.7291666666666666,
5346 + "selectivity": 0.27083333333333337
5347 + },
5348 + {
5349 + "layer": 25,
5350 + "seed": 2,
5351 + "task_acc": 1.0,
5352 + "control_acc": 0.3125,
5353 + "selectivity": 0.6875
5354 + },
5355 + {
5356 + "layer": 25,
5357 + "seed": 3,
5358 + "task_acc": 1.0,
5359 + "control_acc": 0.22916666666666666,
5360 + "selectivity": 0.7708333333333334
5361 + },
5362 + {
5363 + "layer": 25,
5364 + "seed": 4,
5365 + "task_acc": 1.0,
5366 + "control_acc": 0.041666666666666664,
5367 + "selectivity": 0.9583333333333334
5368 + },
5369 + {
5370 + "layer": 26,
5371 + "seed": 0,
5372 + "task_acc": 1.0,
5373 + "control_acc": 0.25,
5374 + "selectivity": 0.75
5375 + },
5376 + {
5377 + "layer": 26,
5378 + "seed": 1,
5379 + "task_acc": 1.0,
5380 + "control_acc": 0.9791666666666666,
5381 + "selectivity": 0.02083333333333337
5382 + },
5383 + {
5384 + "layer": 26,
5385 + "seed": 2,
5386 + "task_acc": 1.0,
5387 + "control_acc": 0.2916666666666667,
5388 + "selectivity": 0.7083333333333333
5389 + },
5390 + {
5391 + "layer": 26,
5392 + "seed": 3,
5393 + "task_acc": 1.0,
5394 + "control_acc": 0.20833333333333334,
5395 + "selectivity": 0.7916666666666666
5396 + },
5397 + {
5398 + "layer": 26,
5399 + "seed": 4,
5400 + "task_acc": 1.0,
5401 + "control_acc": 0.020833333333333332,
5402 + "selectivity": 0.9791666666666666
5403 + },
5404 + {
5405 + "layer": 27,
5406 + "seed": 0,
5407 + "task_acc": 1.0,
5408 + "control_acc": 0.3125,
5409 + "selectivity": 0.6875
5410 + },
5411 + {
5412 + "layer": 27,
5413 + "seed": 1,
5414 + "task_acc": 1.0,
5415 + "control_acc": 0.8541666666666666,
5416 + "selectivity": 0.14583333333333337
5417 + },
5418 + {
5419 + "layer": 27,
5420 + "seed": 2,
5421 + "task_acc": 1.0,
5422 + "control_acc": 0.9791666666666666,
5423 + "selectivity": 0.02083333333333337
5424 + },
5425 + {
5426 + "layer": 27,
5427 + "seed": 3,
5428 + "task_acc": 1.0,
5429 + "control_acc": 0.125,
5430 + "selectivity": 0.875
5431 + },
5432 + {
5433 + "layer": 27,
5434 + "seed": 4,
5435 + "task_acc": 1.0,
5436 + "control_acc": 0.125,
5437 + "selectivity": 0.875
5438 + }
5439 + ]
5440 + },
5441 + "arith_A": {
5442 + "layers": [
5443 + {
5444 + "layer": 0,
5445 + "task_acc_mean": 1.0,
5446 + "task_acc_seed_sd": 0.0,
5447 + "selectivity_mean": 0.7250000000000001,
5448 + "selectivity_ci": [
5449 + 0.4208333333333334,
5450 + 0.9291666666666668
5451 + ],
5452 + "p_boot": 0.0001,
5453 + "fdr_significant": true
5454 + },
5455 + {
5456 + "layer": 1,
5457 + "task_acc_mean": 1.0,
5458 + "task_acc_seed_sd": 0.0,
5459 + "selectivity_mean": 0.5375,
5460 + "selectivity_ci": [
5461 + 0.22916666666666669,
5462 + 0.8458333333333332
5463 + ],
5464 + "p_boot": 0.0001,
5465 + "fdr_significant": true
5466 + },
5467 + {
5468 + "layer": 2,
5469 + "task_acc_mean": 1.0,
5470 + "task_acc_seed_sd": 0.0,
5471 + "selectivity_mean": 0.5541666666666667,
5472 + "selectivity_ci": [
5473 + 0.2375,
5474 + 0.8625
5475 + ],
5476 + "p_boot": 0.0001,
5477 + "fdr_significant": true
5478 + },
5479 + {
5480 + "layer": 3,
5481 + "task_acc_mean": 1.0,
5482 + "task_acc_seed_sd": 0.0,
5483 + "selectivity_mean": 0.35833333333333334,
5484 + "selectivity_ci": [
5485 + 0.06666666666666668,
5486 + 0.6583333333333334
5487 + ],
5488 + "p_boot": 0.0001,
5489 + "fdr_significant": true
5490 + },
5491 + {
5492 + "layer": 4,
5493 + "task_acc_mean": 1.0,
5494 + "task_acc_seed_sd": 0.0,
5495 + "selectivity_mean": 0.2583333333333333,
5496 + "selectivity_ci": [
5497 + 0.09166666666666665,
5498 + 0.5166666666666666
5499 + ],
5500 + "p_boot": 0.0001,
5501 + "fdr_significant": true
5502 + },
5503 + {
5504 + "layer": 5,
5505 + "task_acc_mean": 1.0,
5506 + "task_acc_seed_sd": 0.0,
5507 + "selectivity_mean": 0.3708333333333333,
5508 + "selectivity_ci": [
5509 + 0.1625,
5510 + 0.5874999999999998
5511 + ],
5512 + "p_boot": 0.0001,
5513 + "fdr_significant": true
5514 + },
5515 + {
5516 + "layer": 6,
5517 + "task_acc_mean": 1.0,
5518 + "task_acc_seed_sd": 0.0,
5519 + "selectivity_mean": 0.2958333333333333,
5520 + "selectivity_ci": [
5521 + 0.1708333333333333,
5522 + 0.5083333333333332
5523 + ],
5524 + "p_boot": 0.0001,
5525 + "fdr_significant": true
5526 + },
5527 + {
5528 + "layer": 7,
5529 + "task_acc_mean": 1.0,
5530 + "task_acc_seed_sd": 0.0,
5531 + "selectivity_mean": 0.2833333333333333,
5532 + "selectivity_ci": [
5533 + 0.12916666666666668,
5534 + 0.5291666666666667
5535 + ],
5536 + "p_boot": 0.0001,
5537 + "fdr_significant": true
5538 + },
5539 + {
5540 + "layer": 8,
5541 + "task_acc_mean": 1.0,
5542 + "task_acc_seed_sd": 0.0,
5543 + "selectivity_mean": 0.3458333333333333,
5544 + "selectivity_ci": [
5545 + 0.14583333333333331,
5546 + 0.5916666666666666
5547 + ],
5548 + "p_boot": 0.0001,
5549 + "fdr_significant": true
5550 + },
5551 + {
5552 + "layer": 9,
5553 + "task_acc_mean": 1.0,
5554 + "task_acc_seed_sd": 0.0,
5555 + "selectivity_mean": 0.3791666666666667,
5556 + "selectivity_ci": [
5557 + 0.17083333333333336,
5558 + 0.6125
5559 + ],
5560 + "p_boot": 0.0001,
5561 + "fdr_significant": true
5562 + },
5563 + {
5564 + "layer": 10,
5565 + "task_acc_mean": 1.0,
5566 + "task_acc_seed_sd": 0.0,
5567 + "selectivity_mean": 0.2875,
5568 + "selectivity_ci": [
5569 + 0.12083333333333332,
5570 + 0.5375000000000001
5571 + ],
5572 + "p_boot": 0.0001,
5573 + "fdr_significant": true
5574 + },
5575 + {
5576 + "layer": 11,
5577 + "task_acc_mean": 1.0,
5578 + "task_acc_seed_sd": 0.0,
5579 + "selectivity_mean": 0.4416666666666667,
5580 + "selectivity_ci": [
5581 + 0.19583333333333333,
5582 + 0.6958333333333334
5583 + ],
5584 + "p_boot": 0.0001,
5585 + "fdr_significant": true
5586 + },
5587 + {
5588 + "layer": 12,
5589 + "task_acc_mean": 1.0,
5590 + "task_acc_seed_sd": 0.0,
5591 + "selectivity_mean": 0.3875,
5592 + "selectivity_ci": [
5593 + 0.22916666666666669,
5594 + 0.5791666666666667
5595 + ],
5596 + "p_boot": 0.0001,
5597 + "fdr_significant": true
5598 + },
5599 + {
5600 + "layer": 13,
5601 + "task_acc_mean": 1.0,
5602 + "task_acc_seed_sd": 0.0,
5603 + "selectivity_mean": 0.3291666666666667,
5604 + "selectivity_ci": [
5605 + 0.16666666666666669,
5606 + 0.55
5607 + ],
5608 + "p_boot": 0.0001,
5609 + "fdr_significant": true
5610 + },
5611 + {
5612 + "layer": 14,
5613 + "task_acc_mean": 1.0,
5614 + "task_acc_seed_sd": 0.0,
5615 + "selectivity_mean": 0.5458333333333334,
5616 + "selectivity_ci": [
5617 + 0.375,
5618 + 0.7
5619 + ],
5620 + "p_boot": 0.0001,
5621 + "fdr_significant": true
5622 + },
5623 + {
5624 + "layer": 15,
5625 + "task_acc_mean": 1.0,
5626 + "task_acc_seed_sd": 0.0,
5627 + "selectivity_mean": 0.5708333333333332,
5628 + "selectivity_ci": [
5629 + 0.2708333333333333,
5630 + 0.8625
5631 + ],
5632 + "p_boot": 0.0001,
5633 + "fdr_significant": true
5634 + },
5635 + {
5636 + "layer": 16,
5637 + "task_acc_mean": 1.0,
5638 + "task_acc_seed_sd": 0.0,
5639 + "selectivity_mean": 0.3791666666666667,
5640 + "selectivity_ci": [
5641 + 0.1375,
5642 + 0.6208333333333333
5643 + ],
5644 + "p_boot": 0.0001,
5645 + "fdr_significant": true
5646 + },
5647 + {
5648 + "layer": 17,
5649 + "task_acc_mean": 1.0,
5650 + "task_acc_seed_sd": 0.0,
5651 + "selectivity_mean": 0.4958333333333334,
5652 + "selectivity_ci": [
5653 + 0.29166666666666663,
5654 + 0.7
5655 + ],
5656 + "p_boot": 0.0001,
5657 + "fdr_significant": true
5658 + },
5659 + {
5660 + "layer": 18,
5661 + "task_acc_mean": 1.0,
5662 + "task_acc_seed_sd": 0.0,
5663 + "selectivity_mean": 0.375,
5664 + "selectivity_ci": [
5665 + 0.16666666666666669,
5666 + 0.6041666666666666
5667 + ],
5668 + "p_boot": 0.0001,
5669 + "fdr_significant": true
5670 + },
5671 + {
5672 + "layer": 19,
5673 + "task_acc_mean": 1.0,
5674 + "task_acc_seed_sd": 0.0,
5675 + "selectivity_mean": 0.5541666666666666,
5676 + "selectivity_ci": [
5677 + 0.29166666666666663,
5678 + 0.8166666666666667
5679 + ],
5680 + "p_boot": 0.0001,
5681 + "fdr_significant": true
5682 + },
5683 + {
5684 + "layer": 20,
5685 + "task_acc_mean": 1.0,
5686 + "task_acc_seed_sd": 0.0,
5687 + "selectivity_mean": 0.38333333333333336,
5688 + "selectivity_ci": [
5689 + 0.1375,
5690 + 0.6625
5691 + ],
5692 + "p_boot": 0.0001,
5693 + "fdr_significant": true
5694 + },
5695 + {
5696 + "layer": 21,
5697 + "task_acc_mean": 1.0,
5698 + "task_acc_seed_sd": 0.0,
5699 + "selectivity_mean": 0.5291666666666666,
5700 + "selectivity_ci": [
5701 + 0.2583333333333333,
5702 + 0.8
5703 + ],
5704 + "p_boot": 0.0001,
5705 + "fdr_significant": true
5706 + },
5707 + {
5708 + "layer": 22,
5709 + "task_acc_mean": 1.0,
5710 + "task_acc_seed_sd": 0.0,
5711 + "selectivity_mean": 0.5,
5712 + "selectivity_ci": [
5713 + 0.2333333333333334,
5714 + 0.7666666666666668
5715 + ],
5716 + "p_boot": 0.0001,
5717 + "fdr_significant": true
5718 + },
5719 + {
5720 + "layer": 23,
5721 + "task_acc_mean": 1.0,
5722 + "task_acc_seed_sd": 0.0,
5723 + "selectivity_mean": 0.3666666666666667,
5724 + "selectivity_ci": [
5725 + 0.16666666666666669,
5726 + 0.5666666666666668
5727 + ],
5728 + "p_boot": 0.0001,
5729 + "fdr_significant": true
5730 + },
5731 + {
5732 + "layer": 24,
5733 + "task_acc_mean": 1.0,
5734 + "task_acc_seed_sd": 0.0,
5735 + "selectivity_mean": 0.37916666666666665,
5736 + "selectivity_ci": [
5737 + 0.19166666666666668,
5738 + 0.5874999999999999
5739 + ],
5740 + "p_boot": 0.0001,
5741 + "fdr_significant": true
5742 + },
5743 + {
5744 + "layer": 25,
5745 + "task_acc_mean": 1.0,
5746 + "task_acc_seed_sd": 0.0,
5747 + "selectivity_mean": 0.5083333333333334,
5748 + "selectivity_ci": [
5749 + 0.2541666666666667,
5750 + 0.7625000000000001
5751 + ],
5752 + "p_boot": 0.0001,
5753 + "fdr_significant": true
5754 + },
5755 + {
5756 + "layer": 26,
5757 + "task_acc_mean": 1.0,
5758 + "task_acc_seed_sd": 0.0,
5759 + "selectivity_mean": 0.5333333333333334,
5760 + "selectivity_ci": [
5761 + 0.3125,
5762 + 0.7541666666666667
5763 + ],
5764 + "p_boot": 0.0001,
5765 + "fdr_significant": true
5766 + },
5767 + {
5768 + "layer": 27,
5769 + "task_acc_mean": 1.0,
5770 + "task_acc_seed_sd": 0.0,
5771 + "selectivity_mean": 0.4708333333333333,
5772 + "selectivity_ci": [
5773 + 0.20833333333333331,
5774 + 0.7333333333333334
5775 + ],
5776 + "p_boot": 0.0001,
5777 + "fdr_significant": true
5778 + }
5779 + ],
5780 + "replication_rate_topk": {
5781 + "k": 5,
5782 + "point": 1.0,
5783 + "ci": [
5784 + 1.0,
5785 + 1.0
5786 + ]
5787 + },
5788 + "n_fdr_significant": 28,
5789 + "cells": [
5790 + {
5791 + "layer": 0,
5792 + "seed": 0,
5793 + "task_acc": 1.0,
5794 + "control_acc": 0.041666666666666664,
5795 + "selectivity": 0.9583333333333334
5796 + },
5797 + {
5798 + "layer": 0,
5799 + "seed": 1,
5800 + "task_acc": 1.0,
5801 + "control_acc": 0.875,
5802 + "selectivity": 0.125
5803 + },
5804 + {
5805 + "layer": 0,
5806 + "seed": 2,
5807 + "task_acc": 1.0,
5808 + "control_acc": 0.1875,
5809 + "selectivity": 0.8125
5810 + },
5811 + {
5812 + "layer": 0,
5813 + "seed": 3,
5814 + "task_acc": 1.0,
5815 + "control_acc": 0.041666666666666664,
5816 + "selectivity": 0.9583333333333334
5817 + },
5818 + {
5819 + "layer": 0,
5820 + "seed": 4,
5821 + "task_acc": 1.0,
5822 + "control_acc": 0.22916666666666666,
5823 + "selectivity": 0.7708333333333334
5824 + },
5825 + {
5826 + "layer": 1,
5827 + "seed": 0,
5828 + "task_acc": 1.0,
5829 + "control_acc": 0.0625,
5830 + "selectivity": 0.9375
5831 + },
5832 + {
5833 + "layer": 1,
5834 + "seed": 1,
5835 + "task_acc": 1.0,
5836 + "control_acc": 0.875,
5837 + "selectivity": 0.125
5838 + },
5839 + {
5840 + "layer": 1,
5841 + "seed": 2,
5842 + "task_acc": 1.0,
5843 + "control_acc": 0.20833333333333334,
5844 + "selectivity": 0.7916666666666666
5845 + },
5846 + {
5847 + "layer": 1,
5848 + "seed": 3,
5849 + "task_acc": 1.0,
5850 + "control_acc": 0.9375,
5851 + "selectivity": 0.0625
5852 + },
5853 + {
5854 + "layer": 1,
5855 + "seed": 4,
5856 + "task_acc": 1.0,
5857 + "control_acc": 0.22916666666666666,
5858 + "selectivity": 0.7708333333333334
5859 + },
5860 + {
5861 + "layer": 2,
5862 + "seed": 0,
5863 + "task_acc": 1.0,
5864 + "control_acc": 0.10416666666666667,
5865 + "selectivity": 0.8958333333333334
5866 + },
5867 + {
5868 + "layer": 2,
5869 + "seed": 1,
5870 + "task_acc": 1.0,
5871 + "control_acc": 0.8541666666666666,
5872 + "selectivity": 0.14583333333333337
5873 + },
5874 + {
5875 + "layer": 2,
5876 + "seed": 2,
5877 + "task_acc": 1.0,
5878 + "control_acc": 0.1875,
5879 + "selectivity": 0.8125
5880 + },
5881 + {
5882 + "layer": 2,
5883 + "seed": 3,
5884 + "task_acc": 1.0,
5885 + "control_acc": 0.9375,
5886 + "selectivity": 0.0625
5887 + },
5888 + {
5889 + "layer": 2,
5890 + "seed": 4,
5891 + "task_acc": 1.0,
5892 + "control_acc": 0.14583333333333334,
5893 + "selectivity": 0.8541666666666666
5894 + },
5895 + {
5896 + "layer": 3,
5897 + "seed": 0,
5898 + "task_acc": 1.0,
5899 + "control_acc": 0.8958333333333334,
5900 + "selectivity": 0.10416666666666663
5901 + },
5902 + {
5903 + "layer": 3,
5904 + "seed": 1,
5905 + "task_acc": 1.0,
5906 + "control_acc": 0.9166666666666666,
5907 + "selectivity": 0.08333333333333337
5908 + },
5909 + {
5910 + "layer": 3,
5911 + "seed": 2,
5912 + "task_acc": 1.0,
5913 + "control_acc": 0.22916666666666666,
5914 + "selectivity": 0.7708333333333334
5915 + },
5916 + {
5917 + "layer": 3,
5918 + "seed": 3,
5919 + "task_acc": 1.0,
5920 + "control_acc": 0.9791666666666666,
5921 + "selectivity": 0.02083333333333337
5922 + },
5923 + {
5924 + "layer": 3,
5925 + "seed": 4,
5926 + "task_acc": 1.0,
5927 + "control_acc": 0.1875,
5928 + "selectivity": 0.8125
5929 + },
5930 + {
5931 + "layer": 4,
5932 + "seed": 0,
5933 + "task_acc": 1.0,
5934 + "control_acc": 0.7916666666666666,
5935 + "selectivity": 0.20833333333333337
5936 + },
5937 + {
5938 + "layer": 4,
5939 + "seed": 1,
5940 + "task_acc": 1.0,
5941 + "control_acc": 0.8333333333333334,
5942 + "selectivity": 0.16666666666666663
5943 + },
5944 + {
5945 + "layer": 4,
5946 + "seed": 2,
5947 + "task_acc": 1.0,
5948 + "control_acc": 0.9583333333333334,
5949 + "selectivity": 0.04166666666666663
5950 + },
5951 + {
5952 + "layer": 4,
5953 + "seed": 3,
5954 + "task_acc": 1.0,
5955 + "control_acc": 0.875,
5956 + "selectivity": 0.125
5957 + },
5958 + {
5959 + "layer": 4,
5960 + "seed": 4,
5961 + "task_acc": 1.0,
5962 + "control_acc": 0.25,
5963 + "selectivity": 0.75
5964 + },
5965 + {
5966 + "layer": 5,
5967 + "seed": 0,
5968 + "task_acc": 1.0,
5969 + "control_acc": 0.8125,
5970 + "selectivity": 0.1875
5971 + },
5972 + {
5973 + "layer": 5,
5974 + "seed": 1,
5975 + "task_acc": 1.0,
5976 + "control_acc": 0.8333333333333334,
5977 + "selectivity": 0.16666666666666663
5978 + },
5979 + {
5980 + "layer": 5,
5981 + "seed": 2,
5982 + "task_acc": 1.0,
5983 + "control_acc": 0.3541666666666667,
5984 + "selectivity": 0.6458333333333333
5985 + },
5986 + {
5987 + "layer": 5,
5988 + "seed": 3,
5989 + "task_acc": 1.0,
5990 + "control_acc": 0.8541666666666666,
5991 + "selectivity": 0.14583333333333337
5992 + },
5993 + {
5994 + "layer": 5,
5995 + "seed": 4,
5996 + "task_acc": 1.0,
5997 + "control_acc": 0.2916666666666667,
5998 + "selectivity": 0.7083333333333333
5999 + },
6000 + {
6001 + "layer": 6,
6002 + "seed": 0,
6003 + "task_acc": 1.0,
6004 + "control_acc": 0.8333333333333334,
6005 + "selectivity": 0.16666666666666663
6006 + },
6007 + {
6008 + "layer": 6,
6009 + "seed": 1,
6010 + "task_acc": 1.0,
6011 + "control_acc": 0.8125,
6012 + "selectivity": 0.1875
6013 + },
6014 + {
6015 + "layer": 6,
6016 + "seed": 2,
6017 + "task_acc": 1.0,
6018 + "control_acc": 0.8333333333333334,
6019 + "selectivity": 0.16666666666666663
6020 + },
6021 + {
6022 + "layer": 6,
6023 + "seed": 3,
6024 + "task_acc": 1.0,
6025 + "control_acc": 0.75,
6026 + "selectivity": 0.25
6027 + },
6028 + {
6029 + "layer": 6,
6030 + "seed": 4,
6031 + "task_acc": 1.0,
6032 + "control_acc": 0.2916666666666667,
6033 + "selectivity": 0.7083333333333333
6034 + },
6035 + {
6036 + "layer": 7,
6037 + "seed": 0,
6038 + "task_acc": 1.0,
6039 + "control_acc": 0.875,
6040 + "selectivity": 0.125
6041 + },
6042 + {
6043 + "layer": 7,
6044 + "seed": 1,
6045 + "task_acc": 1.0,
6046 + "control_acc": 0.8541666666666666,
6047 + "selectivity": 0.14583333333333337
6048 + },
6049 + {
6050 + "layer": 7,
6051 + "seed": 2,
6052 + "task_acc": 1.0,
6053 + "control_acc": 0.875,
6054 + "selectivity": 0.125
6055 + },
6056 + {
6057 + "layer": 7,
6058 + "seed": 3,
6059 + "task_acc": 1.0,
6060 + "control_acc": 0.7291666666666666,
6061 + "selectivity": 0.27083333333333337
6062 + },
6063 + {
6064 + "layer": 7,
6065 + "seed": 4,
6066 + "task_acc": 1.0,
6067 + "control_acc": 0.25,
6068 + "selectivity": 0.75
6069 + },
6070 + {
6071 + "layer": 8,
6072 + "seed": 0,
6073 + "task_acc": 1.0,
6074 + "control_acc": 0.6666666666666666,
6075 + "selectivity": 0.33333333333333337
6076 + },
6077 + {
6078 + "layer": 8,
6079 + "seed": 1,
6080 + "task_acc": 1.0,
6081 + "control_acc": 0.8333333333333334,
6082 + "selectivity": 0.16666666666666663
6083 + },
6084 + {
6085 + "layer": 8,
6086 + "seed": 2,
6087 + "task_acc": 1.0,
6088 + "control_acc": 0.9375,
6089 + "selectivity": 0.0625
6090 + },
6091 + {
6092 + "layer": 8,
6093 + "seed": 3,
6094 + "task_acc": 1.0,
6095 + "control_acc": 0.6458333333333334,
6096 + "selectivity": 0.35416666666666663
6097 + },
6098 + {
6099 + "layer": 8,
6100 + "seed": 4,
6101 + "task_acc": 1.0,
6102 + "control_acc": 0.1875,
6103 + "selectivity": 0.8125
6104 + },
6105 + {
6106 + "layer": 9,
6107 + "seed": 0,
6108 + "task_acc": 1.0,
6109 + "control_acc": 0.8541666666666666,
6110 + "selectivity": 0.14583333333333337
6111 + },
6112 + {
6113 + "layer": 9,
6114 + "seed": 1,
6115 + "task_acc": 1.0,
6116 + "control_acc": 0.8541666666666666,
6117 + "selectivity": 0.14583333333333337
6118 + },
6119 + {
6120 + "layer": 9,
6121 + "seed": 2,
6122 + "task_acc": 1.0,
6123 + "control_acc": 0.4583333333333333,
6124 + "selectivity": 0.5416666666666667
6125 + },
6126 + {
6127 + "layer": 9,
6128 + "seed": 3,
6129 + "task_acc": 1.0,
6130 + "control_acc": 0.7291666666666666,
6131 + "selectivity": 0.27083333333333337
6132 + },
6133 + {
6134 + "layer": 9,
6135 + "seed": 4,
6136 + "task_acc": 1.0,
6137 + "control_acc": 0.20833333333333334,
6138 + "selectivity": 0.7916666666666666
6139 + },
6140 + {
6141 + "layer": 10,
6142 + "seed": 0,
6143 + "task_acc": 1.0,
6144 + "control_acc": 0.7708333333333334,
6145 + "selectivity": 0.22916666666666663
6146 + },
6147 + {
6148 + "layer": 10,
6149 + "seed": 1,
6150 + "task_acc": 1.0,
6151 + "control_acc": 0.8125,
6152 + "selectivity": 0.1875
6153 + },
6154 + {
6155 + "layer": 10,
6156 + "seed": 2,
6157 + "task_acc": 1.0,
6158 + "control_acc": 0.9375,
6159 + "selectivity": 0.0625
6160 + },
6161 + {
6162 + "layer": 10,
6163 + "seed": 3,
6164 + "task_acc": 1.0,
6165 + "control_acc": 0.8125,
6166 + "selectivity": 0.1875
6167 + },
6168 + {
6169 + "layer": 10,
6170 + "seed": 4,
6171 + "task_acc": 1.0,
6172 + "control_acc": 0.22916666666666666,
6173 + "selectivity": 0.7708333333333334
6174 + },
6175 + {
6176 + "layer": 11,
6177 + "seed": 0,
6178 + "task_acc": 1.0,
6179 + "control_acc": 0.7916666666666666,
6180 + "selectivity": 0.20833333333333337
6181 + },
6182 + {
6183 + "layer": 11,
6184 + "seed": 1,
6185 + "task_acc": 1.0,
6186 + "control_acc": 0.7708333333333334,
6187 + "selectivity": 0.22916666666666663
6188 + },
6189 + {
6190 + "layer": 11,
6191 + "seed": 2,
6192 + "task_acc": 1.0,
6193 + "control_acc": 0.22916666666666666,
6194 + "selectivity": 0.7708333333333334
6195 + },
6196 + {
6197 + "layer": 11,
6198 + "seed": 3,
6199 + "task_acc": 1.0,
6200 + "control_acc": 0.16666666666666666,
6201 + "selectivity": 0.8333333333333334
6202 + },
6203 + {
6204 + "layer": 11,
6205 + "seed": 4,
6206 + "task_acc": 1.0,
6207 + "control_acc": 0.8333333333333334,
6208 + "selectivity": 0.16666666666666663
6209 + },
6210 + {
6211 + "layer": 12,
6212 + "seed": 0,
6213 + "task_acc": 1.0,
6214 + "control_acc": 0.6458333333333334,
6215 + "selectivity": 0.35416666666666663
6216 + },
6217 + {
6218 + "layer": 12,
6219 + "seed": 1,
6220 + "task_acc": 1.0,
6221 + "control_acc": 0.5833333333333334,
6222 + "selectivity": 0.41666666666666663
6223 + },
6224 + {
6225 + "layer": 12,
6226 + "seed": 2,
6227 + "task_acc": 1.0,
6228 + "control_acc": 0.2708333333333333,
6229 + "selectivity": 0.7291666666666667
6230 + },
6231 + {
6232 + "layer": 12,
6233 + "seed": 3,
6234 + "task_acc": 1.0,
6235 + "control_acc": 0.7083333333333334,
6236 + "selectivity": 0.29166666666666663
6237 + },
6238 + {
6239 + "layer": 12,
6240 + "seed": 4,
6241 + "task_acc": 1.0,
6242 + "control_acc": 0.8541666666666666,
6243 + "selectivity": 0.14583333333333337
6244 + },
6245 + {
6246 + "layer": 13,
6247 + "seed": 0,
6248 + "task_acc": 1.0,
6249 + "control_acc": 0.8541666666666666,
6250 + "selectivity": 0.14583333333333337
6251 + },
6252 + {
6253 + "layer": 13,
6254 + "seed": 1,
6255 + "task_acc": 1.0,
6256 + "control_acc": 0.75,
6257 + "selectivity": 0.25
6258 + },
6259 + {
6260 + "layer": 13,
6261 + "seed": 2,
6262 + "task_acc": 1.0,
6263 + "control_acc": 0.25,
6264 + "selectivity": 0.75
6265 + },
6266 + {
6267 + "layer": 13,
6268 + "seed": 3,
6269 + "task_acc": 1.0,
6270 + "control_acc": 0.6458333333333334,
6271 + "selectivity": 0.35416666666666663
6272 + },
6273 + {
6274 + "layer": 13,
6275 + "seed": 4,
6276 + "task_acc": 1.0,
6277 + "control_acc": 0.8541666666666666,
6278 + "selectivity": 0.14583333333333337
6279 + },
6280 + {
6281 + "layer": 14,
6282 + "seed": 0,
6283 + "task_acc": 1.0,
6284 + "control_acc": 0.7291666666666666,
6285 + "selectivity": 0.27083333333333337
6286 + },
6287 + {
6288 + "layer": 14,
6289 + "seed": 1,
6290 + "task_acc": 1.0,
6291 + "control_acc": 0.625,
6292 + "selectivity": 0.375
6293 + },
6294 + {
6295 + "layer": 14,
6296 + "seed": 2,
6297 + "task_acc": 1.0,
6298 + "control_acc": 0.3333333333333333,
6299 + "selectivity": 0.6666666666666667
6300 + },
6301 + {
6302 + "layer": 14,
6303 + "seed": 3,
6304 + "task_acc": 1.0,
6305 + "control_acc": 0.2708333333333333,
6306 + "selectivity": 0.7291666666666667
6307 + },
6308 + {
6309 + "layer": 14,
6310 + "seed": 4,
6311 + "task_acc": 1.0,
6312 + "control_acc": 0.3125,
6313 + "selectivity": 0.6875
6314 + },
6315 + {
6316 + "layer": 15,
6317 + "seed": 0,
6318 + "task_acc": 1.0,
6319 + "control_acc": 0.020833333333333332,
6320 + "selectivity": 0.9791666666666666
6321 + },
6322 + {
6323 + "layer": 15,
6324 + "seed": 1,
6325 + "task_acc": 1.0,
6326 + "control_acc": 0.7708333333333334,
6327 + "selectivity": 0.22916666666666663
6328 + },
6329 + {
6330 + "layer": 15,
6331 + "seed": 2,
6332 + "task_acc": 1.0,
6333 + "control_acc": 0.1875,
6334 + "selectivity": 0.8125
6335 + },
6336 + {
6337 + "layer": 15,
6338 + "seed": 3,
6339 + "task_acc": 1.0,
6340 + "control_acc": 0.8958333333333334,
6341 + "selectivity": 0.10416666666666663
6342 + },
6343 + {
6344 + "layer": 15,
6345 + "seed": 4,
6346 + "task_acc": 1.0,
6347 + "control_acc": 0.2708333333333333,
6348 + "selectivity": 0.7291666666666667
6349 + },
6350 + {
6351 + "layer": 16,
6352 + "seed": 0,
6353 + "task_acc": 1.0,
6354 + "control_acc": 0.7708333333333334,
6355 + "selectivity": 0.22916666666666663
6356 + },
6357 + {
6358 + "layer": 16,
6359 + "seed": 1,
6360 + "task_acc": 1.0,
6361 + "control_acc": 0.9166666666666666,
6362 + "selectivity": 0.08333333333333337
6363 + },
6364 + {
6365 + "layer": 16,
6366 + "seed": 2,
6367 + "task_acc": 1.0,
6368 + "control_acc": 0.3125,
6369 + "selectivity": 0.6875
6370 + },
6371 + {
6372 + "layer": 16,
6373 + "seed": 3,
6374 + "task_acc": 1.0,
6375 + "control_acc": 0.8541666666666666,
6376 + "selectivity": 0.14583333333333337
6377 + },
6378 + {
6379 + "layer": 16,
6380 + "seed": 4,
6381 + "task_acc": 1.0,
6382 + "control_acc": 0.25,
6383 + "selectivity": 0.75
6384 + },
6385 + {
6386 + "layer": 17,
6387 + "seed": 0,
6388 + "task_acc": 1.0,
6389 + "control_acc": 0.5,
6390 + "selectivity": 0.5
6391 + },
6392 + {
6393 + "layer": 17,
6394 + "seed": 1,
6395 + "task_acc": 1.0,
6396 + "control_acc": 0.75,
6397 + "selectivity": 0.25
6398 + },
6399 + {
6400 + "layer": 17,
6401 + "seed": 2,
6402 + "task_acc": 1.0,
6403 + "control_acc": 0.22916666666666666,
6404 + "selectivity": 0.7708333333333334
6405 + },
6406 + {
6407 + "layer": 17,
6408 + "seed": 3,
6409 + "task_acc": 1.0,
6410 + "control_acc": 0.7708333333333334,
6411 + "selectivity": 0.22916666666666663
6412 + },
6413 + {
6414 + "layer": 17,
6415 + "seed": 4,
6416 + "task_acc": 1.0,
6417 + "control_acc": 0.2708333333333333,
6418 + "selectivity": 0.7291666666666667
6419 + },
6420 + {
6421 + "layer": 18,
6422 + "seed": 0,
6423 + "task_acc": 1.0,
6424 + "control_acc": 0.8541666666666666,
6425 + "selectivity": 0.14583333333333337
6426 + },
6427 + {
6428 + "layer": 18,
6429 + "seed": 1,
6430 + "task_acc": 1.0,
6431 + "control_acc": 0.8333333333333334,
6432 + "selectivity": 0.16666666666666663
6433 + },
6434 + {
6435 + "layer": 18,
6436 + "seed": 2,
6437 + "task_acc": 1.0,
6438 + "control_acc": 0.3958333333333333,
6439 + "selectivity": 0.6041666666666667
6440 + },
6441 + {
6442 + "layer": 18,
6443 + "seed": 3,
6444 + "task_acc": 1.0,
6445 + "control_acc": 0.7916666666666666,
6446 + "selectivity": 0.20833333333333337
6447 + },
6448 + {
6449 + "layer": 18,
6450 + "seed": 4,
6451 + "task_acc": 1.0,
6452 + "control_acc": 0.25,
6453 + "selectivity": 0.75
6454 + },
6455 + {
6456 + "layer": 19,
6457 + "seed": 0,
6458 + "task_acc": 1.0,
6459 + "control_acc": 0.7916666666666666,
6460 + "selectivity": 0.20833333333333337
6461 + },
6462 + {
6463 + "layer": 19,
6464 + "seed": 1,
6465 + "task_acc": 1.0,
6466 + "control_acc": 0.8333333333333334,
6467 + "selectivity": 0.16666666666666663
6468 + },
6469 + {
6470 + "layer": 19,
6471 + "seed": 2,
6472 + "task_acc": 1.0,
6473 + "control_acc": 0.2708333333333333,
6474 + "selectivity": 0.7291666666666667
6475 + },
6476 + {
6477 + "layer": 19,
6478 + "seed": 3,
6479 + "task_acc": 1.0,
6480 + "control_acc": 0.041666666666666664,
6481 + "selectivity": 0.9583333333333334
6482 + },
6483 + {
6484 + "layer": 19,
6485 + "seed": 4,
6486 + "task_acc": 1.0,
6487 + "control_acc": 0.2916666666666667,
6488 + "selectivity": 0.7083333333333333
6489 + },
6490 + {
6491 + "layer": 20,
6492 + "seed": 0,
6493 + "task_acc": 1.0,
6494 + "control_acc": 0.9166666666666666,
6495 + "selectivity": 0.08333333333333337
6496 + },
6497 + {
6498 + "layer": 20,
6499 + "seed": 1,
6500 + "task_acc": 1.0,
6501 + "control_acc": 0.8333333333333334,
6502 + "selectivity": 0.16666666666666663
6503 + },
6504 + {
6505 + "layer": 20,
6506 + "seed": 2,
6507 + "task_acc": 1.0,
6508 + "control_acc": 0.16666666666666666,
6509 + "selectivity": 0.8333333333333334
6510 + },
6511 + {
6512 + "layer": 20,
6513 + "seed": 3,
6514 + "task_acc": 1.0,
6515 + "control_acc": 0.8125,
6516 + "selectivity": 0.1875
6517 + },
6518 + {
6519 + "layer": 20,
6520 + "seed": 4,
6521 + "task_acc": 1.0,
6522 + "control_acc": 0.3541666666666667,
6523 + "selectivity": 0.6458333333333333
6524 + },
6525 + {
6526 + "layer": 21,
6527 + "seed": 0,
6528 + "task_acc": 1.0,
6529 + "control_acc": 0.8125,
6530 + "selectivity": 0.1875
6531 + },
6532 + {
6533 + "layer": 21,
6534 + "seed": 1,
6535 + "task_acc": 1.0,
6536 + "control_acc": 0.8333333333333334,
6537 + "selectivity": 0.16666666666666663
6538 + },
6539 + {
6540 + "layer": 21,
6541 + "seed": 2,
6542 + "task_acc": 1.0,
6543 + "control_acc": 0.4166666666666667,
6544 + "selectivity": 0.5833333333333333
6545 + },
6546 + {
6547 + "layer": 21,
6548 + "seed": 3,
6549 + "task_acc": 1.0,
6550 + "control_acc": 0.041666666666666664,
6551 + "selectivity": 0.9583333333333334
6552 + },
6553 + {
6554 + "layer": 21,
6555 + "seed": 4,
6556 + "task_acc": 1.0,
6557 + "control_acc": 0.25,
6558 + "selectivity": 0.75
6559 + },
6560 + {
6561 + "layer": 22,
6562 + "seed": 0,
6563 + "task_acc": 1.0,
6564 + "control_acc": 0.8541666666666666,
6565 + "selectivity": 0.14583333333333337
6566 + },
6567 + {
6568 + "layer": 22,
6569 + "seed": 1,
6570 + "task_acc": 1.0,
6571 + "control_acc": 0.875,
6572 + "selectivity": 0.125
6573 + },
6574 + {
6575 + "layer": 22,
6576 + "seed": 2,
6577 + "task_acc": 1.0,
6578 + "control_acc": 0.375,
6579 + "selectivity": 0.625
6580 + },
6581 + {
6582 + "layer": 22,
6583 + "seed": 3,
6584 + "task_acc": 1.0,
6585 + "control_acc": 0.0625,
6586 + "selectivity": 0.9375
6587 + },
6588 + {
6589 + "layer": 22,
6590 + "seed": 4,
6591 + "task_acc": 1.0,
6592 + "control_acc": 0.3333333333333333,
6593 + "selectivity": 0.6666666666666667
6594 + },
6595 + {
6596 + "layer": 23,
6597 + "seed": 0,
6598 + "task_acc": 1.0,
6599 + "control_acc": 0.9166666666666666,
6600 + "selectivity": 0.08333333333333337
6601 + },
6602 + {
6603 + "layer": 23,
6604 + "seed": 1,
6605 + "task_acc": 1.0,
6606 + "control_acc": 0.7083333333333334,
6607 + "selectivity": 0.29166666666666663
6608 + },
6609 + {
6610 + "layer": 23,
6611 + "seed": 2,
6612 + "task_acc": 1.0,
6613 + "control_acc": 0.3958333333333333,
6614 + "selectivity": 0.6041666666666667
6615 + },
6616 + {
6617 + "layer": 23,
6618 + "seed": 3,
6619 + "task_acc": 1.0,
6620 + "control_acc": 0.8125,
6621 + "selectivity": 0.1875
6622 + },
6623 + {
6624 + "layer": 23,
6625 + "seed": 4,
6626 + "task_acc": 1.0,
6627 + "control_acc": 0.3333333333333333,
6628 + "selectivity": 0.6666666666666667
6629 + },
6630 + {
6631 + "layer": 24,
6632 + "seed": 0,
6633 + "task_acc": 1.0,
6634 + "control_acc": 0.7916666666666666,
6635 + "selectivity": 0.20833333333333337
6636 + },
6637 + {
6638 + "layer": 24,
6639 + "seed": 1,
6640 + "task_acc": 1.0,
6641 + "control_acc": 0.7916666666666666,
6642 + "selectivity": 0.20833333333333337
6643 + },
6644 + {
6645 + "layer": 24,
6646 + "seed": 2,
6647 + "task_acc": 1.0,
6648 + "control_acc": 0.3958333333333333,
6649 + "selectivity": 0.6041666666666667
6650 + },
6651 + {
6652 + "layer": 24,
6653 + "seed": 3,
6654 + "task_acc": 1.0,
6655 + "control_acc": 0.8333333333333334,
6656 + "selectivity": 0.16666666666666663
6657 + },
6658 + {
6659 + "layer": 24,
6660 + "seed": 4,
6661 + "task_acc": 1.0,
6662 + "control_acc": 0.2916666666666667,
6663 + "selectivity": 0.7083333333333333
6664 + },
6665 + {
6666 + "layer": 25,
6667 + "seed": 0,
6668 + "task_acc": 1.0,
6669 + "control_acc": 0.8541666666666666,
6670 + "selectivity": 0.14583333333333337
6671 + },
6672 + {
6673 + "layer": 25,
6674 + "seed": 1,
6675 + "task_acc": 1.0,
6676 + "control_acc": 0.8125,
6677 + "selectivity": 0.1875
6678 + },
6679 + {
6680 + "layer": 25,
6681 + "seed": 2,
6682 + "task_acc": 1.0,
6683 + "control_acc": 0.3958333333333333,
6684 + "selectivity": 0.6041666666666667
6685 + },
6686 + {
6687 + "layer": 25,
6688 + "seed": 3,
6689 + "task_acc": 1.0,
6690 + "control_acc": 0.041666666666666664,
6691 + "selectivity": 0.9583333333333334
6692 + },
6693 + {
6694 + "layer": 25,
6695 + "seed": 4,
6696 + "task_acc": 1.0,
6697 + "control_acc": 0.3541666666666667,
6698 + "selectivity": 0.6458333333333333
6699 + },
6700 + {
6701 + "layer": 26,
6702 + "seed": 0,
6703 + "task_acc": 1.0,
6704 + "control_acc": 0.8125,
6705 + "selectivity": 0.1875
6706 + },
6707 + {
6708 + "layer": 26,
6709 + "seed": 1,
6710 + "task_acc": 1.0,
6711 + "control_acc": 0.7291666666666666,
6712 + "selectivity": 0.27083333333333337
6713 + },
6714 + {
6715 + "layer": 26,
6716 + "seed": 2,
6717 + "task_acc": 1.0,
6718 + "control_acc": 0.1875,
6719 + "selectivity": 0.8125
6720 + },
6721 + {
6722 + "layer": 26,
6723 + "seed": 3,
6724 + "task_acc": 1.0,
6725 + "control_acc": 0.25,
6726 + "selectivity": 0.75
6727 + },
6728 + {
6729 + "layer": 26,
6730 + "seed": 4,
6731 + "task_acc": 1.0,
6732 + "control_acc": 0.3541666666666667,
6733 + "selectivity": 0.6458333333333333
6734 + },
6735 + {
6736 + "layer": 27,
6737 + "seed": 0,
6738 + "task_acc": 1.0,
6739 + "control_acc": 0.8958333333333334,
6740 + "selectivity": 0.10416666666666663
6741 + },
6742 + {
6743 + "layer": 27,
6744 + "seed": 1,
6745 + "task_acc": 1.0,
6746 + "control_acc": 0.875,
6747 + "selectivity": 0.125
6748 + },
6749 + {
6750 + "layer": 27,
6751 + "seed": 2,
6752 + "task_acc": 1.0,
6753 + "control_acc": 0.1875,
6754 + "selectivity": 0.8125
6755 + },
6756 + {
6757 + "layer": 27,
6758 + "seed": 3,
6759 + "task_acc": 1.0,
6760 + "control_acc": 0.2708333333333333,
6761 + "selectivity": 0.7291666666666667
6762 + },
6763 + {
6764 + "layer": 27,
6765 + "seed": 4,
6766 + "task_acc": 1.0,
6767 + "control_acc": 0.4166666666666667,
6768 + "selectivity": 0.5833333333333333
6769 + }
6770 + ]
6771 + },
6772 + "arith_B": {
6773 + "layers": [
6774 + {
6775 + "layer": 0,
6776 + "task_acc_mean": 1.0,
6777 + "task_acc_seed_sd": 0.0,
6778 + "selectivity_mean": 0.5083333333333333,
6779 + "selectivity_ci": [
6780 + 0.19999999999999998,
6781 + 0.8125
6782 + ],
6783 + "p_boot": 0.0001,
6784 + "fdr_significant": true
6785 + },
6786 + {
6787 + "layer": 1,
6788 + "task_acc_mean": 1.0,
6789 + "task_acc_seed_sd": 0.0,
6790 + "selectivity_mean": 0.3916666666666667,
6791 + "selectivity_ci": [
6792 + 0.04583333333333332,
6793 + 0.7375
6794 + ],
6795 + "p_boot": 0.0003,
6796 + "fdr_significant": true
6797 + },
6798 + {
6799 + "layer": 2,
6800 + "task_acc_mean": 1.0,
6801 + "task_acc_seed_sd": 0.0,
6802 + "selectivity_mean": 0.6625000000000001,
6803 + "selectivity_ci": [
6804 + 0.38333333333333336,
6805 + 0.8708333333333333
6806 + ],
6807 + "p_boot": 0.0001,
6808 + "fdr_significant": true
6809 + },
6810 + {
6811 + "layer": 3,
6812 + "task_acc_mean": 1.0,
6813 + "task_acc_seed_sd": 0.0,
6814 + "selectivity_mean": 0.7125000000000001,
6815 + "selectivity_ci": [
6816 + 0.4208333333333334,
6817 + 0.9
6818 + ],
6819 + "p_boot": 0.0001,
6820 + "fdr_significant": true
6821 + },
6822 + {
6823 + "layer": 4,
6824 + "task_acc_mean": 1.0,
6825 + "task_acc_seed_sd": 0.0,
6826 + "selectivity_mean": 0.5416666666666667,
6827 + "selectivity_ci": [
6828 + 0.19583333333333336,
6829 + 0.8875
6830 + ],
6831 + "p_boot": 0.0001,
6832 + "fdr_significant": true
6833 + },
6834 + {
6835 + "layer": 5,
6836 + "task_acc_mean": 1.0,
6837 + "task_acc_seed_sd": 0.0,
6838 + "selectivity_mean": 0.4,
6839 + "selectivity_ci": [
6840 + 0.029166666666666674,
6841 + 0.7708333333333333
6842 + ],
6843 + "p_boot": 0.0003,
6844 + "fdr_significant": true
6845 + },
6846 + {
6847 + "layer": 6,
6848 + "task_acc_mean": 1.0,
6849 + "task_acc_seed_sd": 0.0,
6850 + "selectivity_mean": 0.2375,
6851 + "selectivity_ci": [
6852 + 0.033333333333333326,
6853 + 0.5708333333333334
6854 + ],
6855 + "p_boot": 0.0004,
6856 + "fdr_significant": true
6857 + },
6858 + {
6859 + "layer": 7,
6860 + "task_acc_mean": 1.0,
6861 + "task_acc_seed_sd": 0.0,
6862 + "selectivity_mean": 0.4166666666666667,
6863 + "selectivity_ci": [
6864 + 0.11250000000000002,
6865 + 0.7291666666666667
6866 + ],
6867 + "p_boot": 0.0001,
6868 + "fdr_significant": true
6869 + },
6870 + {
6871 + "layer": 8,
6872 + "task_acc_mean": 1.0,
6873 + "task_acc_seed_sd": 0.0,
6874 + "selectivity_mean": 0.2208333333333333,
6875 + "selectivity_ci": [
6876 + 0.049999999999999975,
6877 + 0.5291666666666667
6878 + ],
6879 + "p_boot": 0.0001,
6880 + "fdr_significant": true
6881 + },
6882 + {
6883 + "layer": 9,
6884 + "task_acc_mean": 1.0,
6885 + "task_acc_seed_sd": 0.0,
6886 + "selectivity_mean": 0.5708333333333334,
6887 + "selectivity_ci": [
6888 + 0.2166666666666667,
6889 + 0.925
6890 + ],
6891 + "p_boot": 0.0001,
6892 + "fdr_significant": true
6893 + },
6894 + {
6895 + "layer": 10,
6896 + "task_acc_mean": 1.0,
6897 + "task_acc_seed_sd": 0.0,
6898 + "selectivity_mean": 0.6083333333333333,
6899 + "selectivity_ci": [
6900 + 0.2958333333333333,
6901 + 0.9083333333333332
6902 + ],
6903 + "p_boot": 0.0001,
6904 + "fdr_significant": true
6905 + },
6906 + {
6907 + "layer": 11,
6908 + "task_acc_mean": 1.0,
6909 + "task_acc_seed_sd": 0.0,
6910 + "selectivity_mean": 0.41666666666666663,
6911 + "selectivity_ci": [
6912 + 0.12499999999999997,
6913 + 0.7291666666666666
6914 + ],
6915 + "p_boot": 0.0001,
6916 + "fdr_significant": true
6917 + },
6918 + {
6919 + "layer": 12,
6920 + "task_acc_mean": 1.0,
6921 + "task_acc_seed_sd": 0.0,
6922 + "selectivity_mean": 0.4458333333333333,
6923 + "selectivity_ci": [
6924 + 0.15416666666666665,
6925 + 0.7499999999999999
6926 + ],
6927 + "p_boot": 0.0001,
6928 + "fdr_significant": true
6929 + },
6930 + {
6931 + "layer": 13,
6932 + "task_acc_mean": 1.0,
6933 + "task_acc_seed_sd": 0.0,
6934 + "selectivity_mean": 0.2916666666666667,
6935 + "selectivity_ci": [
6936 + 0.12916666666666668,
6937 + 0.5666666666666667
6938 + ],
6939 + "p_boot": 0.0001,
6940 + "fdr_significant": true
6941 + },
6942 + {
6943 + "layer": 14,
6944 + "task_acc_mean": 1.0,
6945 + "task_acc_seed_sd": 0.0,
6946 + "selectivity_mean": 0.4,
6947 + "selectivity_ci": [
6948 + 0.07916666666666668,
6949 + 0.7208333333333333
6950 + ],
6951 + "p_boot": 0.0001,
6952 + "fdr_significant": true
6953 + },
6954 + {
6955 + "layer": 15,
6956 + "task_acc_mean": 1.0,
6957 + "task_acc_seed_sd": 0.0,
6958 + "selectivity_mean": 0.2583333333333333,
6959 + "selectivity_ci": [
6960 + 0.09166666666666667,
6961 + 0.5791666666666667
6962 + ],
6963 + "p_boot": 0.0001,
6964 + "fdr_significant": true
6965 + },
6966 + {
6967 + "layer": 16,
6968 + "task_acc_mean": 1.0,
6969 + "task_acc_seed_sd": 0.0,
6970 + "selectivity_mean": 0.05833333333333335,
6971 + "selectivity_ci": [
6972 + 0.02083333333333335,
6973 + 0.09166666666666667
6974 + ],
6975 + "p_boot": 0.0002,
6976 + "fdr_significant": true
6977 + },
6978 + {
6979 + "layer": 17,
6980 + "task_acc_mean": 1.0,
6981 + "task_acc_seed_sd": 0.0,
6982 + "selectivity_mean": 0.3958333333333333,
6983 + "selectivity_ci": [
6984 + 0.054166666666666675,
6985 + 0.7375
6986 + ],
6987 + "p_boot": 0.0001,
6988 + "fdr_significant": true
6989 + },
6990 + {
6991 + "layer": 18,
6992 + "task_acc_mean": 1.0,
6993 + "task_acc_seed_sd": 0.0,
6994 + "selectivity_mean": 0.4,
6995 + "selectivity_ci": [
6996 + 0.07500000000000002,
6997 + 0.7291666666666667
6998 + ],
6999 + "p_boot": 0.0001,
7000 + "fdr_significant": true
7001 + },
7002 + {
7003 + "layer": 19,
7004 + "task_acc_mean": 1.0,
7005 + "task_acc_seed_sd": 0.0,
7006 + "selectivity_mean": 0.2541666666666666,
7007 + "selectivity_ci": [
7008 + 0.05,
7009 + 0.5875
7010 + ],
7011 + "p_boot": 0.0001,
7012 + "fdr_significant": true
7013 + },
7014 + {
7015 + "layer": 20,
7016 + "task_acc_mean": 1.0,
7017 + "task_acc_seed_sd": 0.0,
7018 + "selectivity_mean": 0.3791666666666667,
7019 + "selectivity_ci": [
7020 + 0.08333333333333333,
7021 + 0.6833333333333333
7022 + ],
7023 + "p_boot": 0.0001,
7024 + "fdr_significant": true
7025 + },
7026 + {
7027 + "layer": 21,
7028 + "task_acc_mean": 1.0,
7029 + "task_acc_seed_sd": 0.0,
7030 + "selectivity_mean": 0.3833333333333333,
7031 + "selectivity_ci": [
7032 + 0.12916666666666665,
7033 + 0.6375
7034 + ],
7035 + "p_boot": 0.0001,
7036 + "fdr_significant": true
7037 + },
7038 + {
7039 + "layer": 22,
7040 + "task_acc_mean": 1.0,
7041 + "task_acc_seed_sd": 0.0,
7042 + "selectivity_mean": 0.25,
7043 + "selectivity_ci": [
7044 + 0.05833333333333333,
7045 + 0.55
7046 + ],
7047 + "p_boot": 0.0005,
7048 + "fdr_significant": true
7049 + },
7050 + {
7051 + "layer": 23,
7052 + "task_acc_mean": 1.0,
7053 + "task_acc_seed_sd": 0.0,
7054 + "selectivity_mean": 0.3708333333333333,
7055 + "selectivity_ci": [
7056 + 0.12916666666666662,
7057 + 0.6166666666666666
7058 + ],
7059 + "p_boot": 0.0001,
7060 + "fdr_significant": true
7061 + },
7062 + {
7063 + "layer": 24,
7064 + "task_acc_mean": 1.0,
7065 + "task_acc_seed_sd": 0.0,
7066 + "selectivity_mean": 0.4083333333333333,
7067 + "selectivity_ci": [
7068 + 0.09166666666666663,
7069 + 0.7250000000000001
7070 + ],
7071 + "p_boot": 0.0001,
7072 + "fdr_significant": true
7073 + },
7074 + {
7075 + "layer": 25,
7076 + "task_acc_mean": 1.0,
7077 + "task_acc_seed_sd": 0.0,
7078 + "selectivity_mean": 0.5208333333333334,
7079 + "selectivity_ci": [
7080 + 0.2041666666666667,
7081 + 0.8333333333333334
7082 + ],
7083 + "p_boot": 0.0001,
7084 + "fdr_significant": true
7085 + },
7086 + {
7087 + "layer": 26,
7088 + "task_acc_mean": 1.0,
7089 + "task_acc_seed_sd": 0.0,
7090 + "selectivity_mean": 0.3625,
7091 + "selectivity_ci": [
7092 + 0.06666666666666665,
7093 + 0.6791666666666666
7094 + ],
7095 + "p_boot": 0.0001,
7096 + "fdr_significant": true
7097 + },
7098 + {
7099 + "layer": 27,
7100 + "task_acc_mean": 1.0,
7101 + "task_acc_seed_sd": 0.0,
7102 + "selectivity_mean": 0.5166666666666667,
7103 + "selectivity_ci": [
7104 + 0.22083333333333335,
7105 + 0.8125
7106 + ],
7107 + "p_boot": 0.0001,
7108 + "fdr_significant": true
7109 + }
7110 + ],
7111 + "replication_rate_topk": {
7112 + "k": 5,
7113 + "point": 1.0,
7114 + "ci": [
7115 + 1.0,
7116 + 1.0
7117 + ]
7118 + },
7119 + "n_fdr_significant": 28,
7120 + "cells": [
7121 + {
7122 + "layer": 0,
7123 + "seed": 0,
7124 + "task_acc": 1.0,
7125 + "control_acc": 0.9166666666666666,
7126 + "selectivity": 0.08333333333333337
7127 + },
7128 + {
7129 + "layer": 0,
7130 + "seed": 1,
7131 + "task_acc": 1.0,
7132 + "control_acc": 0.14583333333333334,
7133 + "selectivity": 0.8541666666666666
7134 + },
7135 + {
7136 + "layer": 0,
7137 + "seed": 2,
7138 + "task_acc": 1.0,
7139 + "control_acc": 0.20833333333333334,
7140 + "selectivity": 0.7916666666666666
7141 + },
7142 + {
7143 + "layer": 0,
7144 + "seed": 3,
7145 + "task_acc": 1.0,
7146 + "control_acc": 0.9583333333333334,
7147 + "selectivity": 0.04166666666666663
7148 + },
7149 + {
7150 + "layer": 0,
7151 + "seed": 4,
7152 + "task_acc": 1.0,
7153 + "control_acc": 0.22916666666666666,
7154 + "selectivity": 0.7708333333333334
7155 + },
7156 + {
7157 + "layer": 1,
7158 + "seed": 0,
7159 + "task_acc": 1.0,
7160 + "control_acc": 0.0625,
7161 + "selectivity": 0.9375
7162 + },
7163 + {
7164 + "layer": 1,
7165 + "seed": 1,
7166 + "task_acc": 1.0,
7167 + "control_acc": 0.8541666666666666,
7168 + "selectivity": 0.14583333333333337
7169 + },
7170 + {
7171 + "layer": 1,
7172 + "seed": 2,
7173 + "task_acc": 1.0,
7174 + "control_acc": 0.9583333333333334,
7175 + "selectivity": 0.04166666666666663
7176 + },
7177 + {
7178 + "layer": 1,
7179 + "seed": 3,
7180 + "task_acc": 1.0,
7181 + "control_acc": 1.0,
7182 + "selectivity": 0.0
7183 + },
7184 + {
7185 + "layer": 1,
7186 + "seed": 4,
7187 + "task_acc": 1.0,
7188 + "control_acc": 0.16666666666666666,
7189 + "selectivity": 0.8333333333333334
7190 + },
7191 + {
7192 + "layer": 2,
7193 + "seed": 0,
7194 + "task_acc": 1.0,
7195 + "control_acc": 0.0625,
7196 + "selectivity": 0.9375
7197 + },
7198 + {
7199 + "layer": 2,
7200 + "seed": 1,
7201 + "task_acc": 1.0,
7202 + "control_acc": 0.875,
7203 + "selectivity": 0.125
7204 + },
7205 + {
7206 + "layer": 2,
7207 + "seed": 2,
7208 + "task_acc": 1.0,
7209 + "control_acc": 0.1875,
7210 + "selectivity": 0.8125
7211 + },
7212 + {
7213 + "layer": 2,
7214 + "seed": 3,
7215 + "task_acc": 1.0,
7216 + "control_acc": 0.3958333333333333,
7217 + "selectivity": 0.6041666666666667
7218 + },
7219 + {
7220 + "layer": 2,
7221 + "seed": 4,
7222 + "task_acc": 1.0,
7223 + "control_acc": 0.16666666666666666,
7224 + "selectivity": 0.8333333333333334
7225 + },
7226 + {
7227 + "layer": 3,
7228 + "seed": 0,
7229 + "task_acc": 1.0,
7230 + "control_acc": 0.0625,
7231 + "selectivity": 0.9375
7232 + },
7233 + {
7234 + "layer": 3,
7235 + "seed": 1,
7236 + "task_acc": 1.0,
7237 + "control_acc": 0.8541666666666666,
7238 + "selectivity": 0.14583333333333337
7239 + },
7240 + {
7241 + "layer": 3,
7242 + "seed": 2,
7243 + "task_acc": 1.0,
7244 + "control_acc": 0.16666666666666666,
7245 + "selectivity": 0.8333333333333334
7246 + },
7247 + {
7248 + "layer": 3,
7249 + "seed": 3,
7250 + "task_acc": 1.0,
7251 + "control_acc": 0.25,
7252 + "selectivity": 0.75
7253 + },
7254 + {
7255 + "layer": 3,
7256 + "seed": 4,
7257 + "task_acc": 1.0,
7258 + "control_acc": 0.10416666666666667,
7259 + "selectivity": 0.8958333333333334
7260 + },
7261 + {
7262 + "layer": 4,
7263 + "seed": 0,
7264 + "task_acc": 1.0,
7265 + "control_acc": 0.041666666666666664,
7266 + "selectivity": 0.9583333333333334
7267 + },
7268 + {
7269 + "layer": 4,
7270 + "seed": 1,
7271 + "task_acc": 1.0,
7272 + "control_acc": 0.875,
7273 + "selectivity": 0.125
7274 + },
7275 + {
7276 + "layer": 4,
7277 + "seed": 2,
7278 + "task_acc": 1.0,
7279 + "control_acc": 0.9791666666666666,
7280 + "selectivity": 0.02083333333333337
7281 + },
7282 + {
7283 + "layer": 4,
7284 + "seed": 3,
7285 + "task_acc": 1.0,
7286 + "control_acc": 0.3125,
7287 + "selectivity": 0.6875
7288 + },
7289 + {
7290 + "layer": 4,
7291 + "seed": 4,
7292 + "task_acc": 1.0,
7293 + "control_acc": 0.08333333333333333,
7294 + "selectivity": 0.9166666666666666
7295 + },
7296 + {
7297 + "layer": 5,
7298 + "seed": 0,
7299 + "task_acc": 1.0,
7300 + "control_acc": 0.041666666666666664,
7301 + "selectivity": 0.9583333333333334
7302 + },
7303 + {
7304 + "layer": 5,
7305 + "seed": 1,
7306 + "task_acc": 1.0,
7307 + "control_acc": 0.8958333333333334,
7308 + "selectivity": 0.10416666666666663
7309 + },
7310 + {
7311 + "layer": 5,
7312 + "seed": 2,
7313 + "task_acc": 1.0,
7314 + "control_acc": 0.9791666666666666,
7315 + "selectivity": 0.02083333333333337
7316 + },
7317 + {
7318 + "layer": 5,
7319 + "seed": 3,
7320 + "task_acc": 1.0,
7321 + "control_acc": 1.0,
7322 + "selectivity": 0.0
7323 + },
7324 + {
7325 + "layer": 5,
7326 + "seed": 4,
7327 + "task_acc": 1.0,
7328 + "control_acc": 0.08333333333333333,
7329 + "selectivity": 0.9166666666666666
7330 + },
7331 + {
7332 + "layer": 6,
7333 + "seed": 0,
7334 + "task_acc": 1.0,
7335 + "control_acc": 0.9583333333333334,
7336 + "selectivity": 0.04166666666666663
7337 + },
7338 + {
7339 + "layer": 6,
7340 + "seed": 1,
7341 + "task_acc": 1.0,
7342 + "control_acc": 0.9166666666666666,
7343 + "selectivity": 0.08333333333333337
7344 + },
7345 + {
7346 + "layer": 6,
7347 + "seed": 2,
7348 + "task_acc": 1.0,
7349 + "control_acc": 0.10416666666666667,
7350 + "selectivity": 0.8958333333333334
7351 + },
7352 + {
7353 + "layer": 6,
7354 + "seed": 3,
7355 + "task_acc": 1.0,
7356 + "control_acc": 1.0,
7357 + "selectivity": 0.0
7358 + },
7359 + {
7360 + "layer": 6,
7361 + "seed": 4,
7362 + "task_acc": 1.0,
7363 + "control_acc": 0.8333333333333334,
7364 + "selectivity": 0.16666666666666663
7365 + },
7366 + {
7367 + "layer": 7,
7368 + "seed": 0,
7369 + "task_acc": 1.0,
7370 + "control_acc": 0.9166666666666666,
7371 + "selectivity": 0.08333333333333337
7372 + },
7373 + {
7374 + "layer": 7,
7375 + "seed": 1,
7376 + "task_acc": 1.0,
7377 + "control_acc": 0.875,
7378 + "selectivity": 0.125
7379 + },
7380 + {
7381 + "layer": 7,
7382 + "seed": 2,
7383 + "task_acc": 1.0,
7384 + "control_acc": 0.10416666666666667,
7385 + "selectivity": 0.8958333333333334
7386 + },
7387 + {
7388 + "layer": 7,
7389 + "seed": 3,
7390 + "task_acc": 1.0,
7391 + "control_acc": 0.16666666666666666,
7392 + "selectivity": 0.8333333333333334
7393 + },
7394 + {
7395 + "layer": 7,
7396 + "seed": 4,
7397 + "task_acc": 1.0,
7398 + "control_acc": 0.8541666666666666,
7399 + "selectivity": 0.14583333333333337
7400 + },
7401 + {
7402 + "layer": 8,
7403 + "seed": 0,
7404 + "task_acc": 1.0,
7405 + "control_acc": 0.9166666666666666,
7406 + "selectivity": 0.08333333333333337
7407 + },
7408 + {
7409 + "layer": 8,
7410 + "seed": 1,
7411 + "task_acc": 1.0,
7412 + "control_acc": 0.8958333333333334,
7413 + "selectivity": 0.10416666666666663
7414 + },
7415 + {
7416 + "layer": 8,
7417 + "seed": 2,
7418 + "task_acc": 1.0,
7419 + "control_acc": 0.9583333333333334,
7420 + "selectivity": 0.04166666666666663
7421 + },
7422 + {
7423 + "layer": 8,
7424 + "seed": 3,
7425 + "task_acc": 1.0,
7426 + "control_acc": 0.9583333333333334,
7427 + "selectivity": 0.04166666666666663
7428 + },
7429 + {
7430 + "layer": 8,
7431 + "seed": 4,
7432 + "task_acc": 1.0,
7433 + "control_acc": 0.16666666666666666,
7434 + "selectivity": 0.8333333333333334
7435 + },
7436 + {
7437 + "layer": 9,
7438 + "seed": 0,
7439 + "task_acc": 1.0,
7440 + "control_acc": 0.0625,
7441 + "selectivity": 0.9375
7442 + },
7443 + {
7444 + "layer": 9,
7445 + "seed": 1,
7446 + "task_acc": 1.0,
7447 + "control_acc": 0.0625,
7448 + "selectivity": 0.9375
7449 + },
7450 + {
7451 + "layer": 9,
7452 + "seed": 2,
7453 + "task_acc": 1.0,
7454 + "control_acc": 0.9166666666666666,
7455 + "selectivity": 0.08333333333333337
7456 + },
7457 + {
7458 + "layer": 9,
7459 + "seed": 3,
7460 + "task_acc": 1.0,
7461 + "control_acc": 0.9791666666666666,
7462 + "selectivity": 0.02083333333333337
7463 + },
7464 + {
7465 + "layer": 9,
7466 + "seed": 4,
7467 + "task_acc": 1.0,
7468 + "control_acc": 0.125,
7469 + "selectivity": 0.875
7470 + },
7471 + {
7472 + "layer": 10,
7473 + "seed": 0,
7474 + "task_acc": 1.0,
7475 + "control_acc": 0.75,
7476 + "selectivity": 0.25
7477 + },
7478 + {
7479 + "layer": 10,
7480 + "seed": 1,
7481 + "task_acc": 1.0,
7482 + "control_acc": 0.8958333333333334,
7483 + "selectivity": 0.10416666666666663
7484 + },
7485 + {
7486 + "layer": 10,
7487 + "seed": 2,
7488 + "task_acc": 1.0,
7489 + "control_acc": 0.0625,
7490 + "selectivity": 0.9375
7491 + },
7492 + {
7493 + "layer": 10,
7494 + "seed": 3,
7495 + "task_acc": 1.0,
7496 + "control_acc": 0.16666666666666666,
7497 + "selectivity": 0.8333333333333334
7498 + },
7499 + {
7500 + "layer": 10,
7501 + "seed": 4,
7502 + "task_acc": 1.0,
7503 + "control_acc": 0.08333333333333333,
7504 + "selectivity": 0.9166666666666666
7505 + },
7506 + {
7507 + "layer": 11,
7508 + "seed": 0,
7509 + "task_acc": 1.0,
7510 + "control_acc": 0.875,
7511 + "selectivity": 0.125
7512 + },
7513 + {
7514 + "layer": 11,
7515 + "seed": 1,
7516 + "task_acc": 1.0,
7517 + "control_acc": 0.8958333333333334,
7518 + "selectivity": 0.10416666666666663
7519 + },
7520 + {
7521 + "layer": 11,
7522 + "seed": 2,
7523 + "task_acc": 1.0,
7524 + "control_acc": 0.08333333333333333,
7525 + "selectivity": 0.9166666666666666
7526 + },
7527 + {
7528 + "layer": 11,
7529 + "seed": 3,
7530 + "task_acc": 1.0,
7531 + "control_acc": 0.22916666666666666,
7532 + "selectivity": 0.7708333333333334
7533 + },
7534 + {
7535 + "layer": 11,
7536 + "seed": 4,
7537 + "task_acc": 1.0,
7538 + "control_acc": 0.8333333333333334,
7539 + "selectivity": 0.16666666666666663
7540 + },
7541 + {
7542 + "layer": 12,
7543 + "seed": 0,
7544 + "task_acc": 1.0,
7545 + "control_acc": 0.8125,
7546 + "selectivity": 0.1875
7547 + },
7548 + {
7549 + "layer": 12,
7550 + "seed": 1,
7551 + "task_acc": 1.0,
7552 + "control_acc": 0.875,
7553 + "selectivity": 0.125
7554 + },
7555 + {
7556 + "layer": 12,
7557 + "seed": 2,
7558 + "task_acc": 1.0,
7559 + "control_acc": 0.08333333333333333,
7560 + "selectivity": 0.9166666666666666
7561 + },
7562 + {
7563 + "layer": 12,
7564 + "seed": 3,
7565 + "task_acc": 1.0,
7566 + "control_acc": 0.16666666666666666,
7567 + "selectivity": 0.8333333333333334
7568 + },
7569 + {
7570 + "layer": 12,
7571 + "seed": 4,
7572 + "task_acc": 1.0,
7573 + "control_acc": 0.8333333333333334,
7574 + "selectivity": 0.16666666666666663
7575 + },
7576 + {
7577 + "layer": 13,
7578 + "seed": 0,
7579 + "task_acc": 1.0,
7580 + "control_acc": 0.8541666666666666,
7581 + "selectivity": 0.14583333333333337
7582 + },
7583 + {
7584 + "layer": 13,
7585 + "seed": 1,
7586 + "task_acc": 1.0,
7587 + "control_acc": 0.8958333333333334,
7588 + "selectivity": 0.10416666666666663
7589 + },
7590 + {
7591 + "layer": 13,
7592 + "seed": 2,
7593 + "task_acc": 1.0,
7594 + "control_acc": 0.8541666666666666,
7595 + "selectivity": 0.14583333333333337
7596 + },
7597 + {
7598 + "layer": 13,
7599 + "seed": 3,
7600 + "task_acc": 1.0,
7601 + "control_acc": 0.16666666666666666,
7602 + "selectivity": 0.8333333333333334
7603 + },
7604 + {
7605 + "layer": 13,
7606 + "seed": 4,
7607 + "task_acc": 1.0,
7608 + "control_acc": 0.7708333333333334,
7609 + "selectivity": 0.22916666666666663
7610 + },
7611 + {
7612 + "layer": 14,
7613 + "seed": 0,
7614 + "task_acc": 1.0,
7615 + "control_acc": 0.125,
7616 + "selectivity": 0.875
7617 + },
7618 + {
7619 + "layer": 14,
7620 + "seed": 1,
7621 + "task_acc": 1.0,
7622 + "control_acc": 0.8541666666666666,
7623 + "selectivity": 0.14583333333333337
7624 + },
7625 + {
7626 + "layer": 14,
7627 + "seed": 2,
7628 + "task_acc": 1.0,
7629 + "control_acc": 0.9583333333333334,
7630 + "selectivity": 0.04166666666666663
7631 + },
7632 + {
7633 + "layer": 14,
7634 + "seed": 3,
7635 + "task_acc": 1.0,
7636 + "control_acc": 0.14583333333333334,
7637 + "selectivity": 0.8541666666666666
7638 + },
7639 + {
7640 + "layer": 14,
7641 + "seed": 4,
7642 + "task_acc": 1.0,
7643 + "control_acc": 0.9166666666666666,
7644 + "selectivity": 0.08333333333333337
7645 + },
7646 + {
7647 + "layer": 15,
7648 + "seed": 0,
7649 + "task_acc": 1.0,
7650 + "control_acc": 0.8958333333333334,
7651 + "selectivity": 0.10416666666666663
7652 + },
7653 + {
7654 + "layer": 15,
7655 + "seed": 1,
7656 + "task_acc": 1.0,
7657 + "control_acc": 0.9166666666666666,
7658 + "selectivity": 0.08333333333333337
7659 + },
7660 + {
7661 + "layer": 15,
7662 + "seed": 2,
7663 + "task_acc": 1.0,
7664 + "control_acc": 0.10416666666666667,
7665 + "selectivity": 0.8958333333333334
7666 + },
7667 + {
7668 + "layer": 15,
7669 + "seed": 3,
7670 + "task_acc": 1.0,
7671 + "control_acc": 0.8958333333333334,
7672 + "selectivity": 0.10416666666666663
7673 + },
7674 + {
7675 + "layer": 15,
7676 + "seed": 4,
7677 + "task_acc": 1.0,
7678 + "control_acc": 0.8958333333333334,
7679 + "selectivity": 0.10416666666666663
7680 + },
7681 + {
7682 + "layer": 16,
7683 + "seed": 0,
7684 + "task_acc": 1.0,
7685 + "control_acc": 0.9166666666666666,
7686 + "selectivity": 0.08333333333333337
7687 + },
7688 + {
7689 + "layer": 16,
7690 + "seed": 1,
7691 + "task_acc": 1.0,
7692 + "control_acc": 0.9166666666666666,
7693 + "selectivity": 0.08333333333333337
7694 + },
7695 + {
7696 + "layer": 16,
7697 + "seed": 2,
7698 + "task_acc": 1.0,
7699 + "control_acc": 0.9791666666666666,
7700 + "selectivity": 0.02083333333333337
7701 + },
7702 + {
7703 + "layer": 16,
7704 + "seed": 3,
7705 + "task_acc": 1.0,
7706 + "control_acc": 1.0,
7707 + "selectivity": 0.0
7708 + },
7709 + {
7710 + "layer": 16,
7711 + "seed": 4,
7712 + "task_acc": 1.0,
7713 + "control_acc": 0.8958333333333334,
7714 + "selectivity": 0.10416666666666663
7715 + },
7716 + {
7717 + "layer": 17,
7718 + "seed": 0,
7719 + "task_acc": 1.0,
7720 + "control_acc": 0.8958333333333334,
7721 + "selectivity": 0.10416666666666663
7722 + },
7723 + {
7724 + "layer": 17,
7725 + "seed": 1,
7726 + "task_acc": 1.0,
7727 + "control_acc": 0.9166666666666666,
7728 + "selectivity": 0.08333333333333337
7729 + },
7730 + {
7731 + "layer": 17,
7732 + "seed": 2,
7733 + "task_acc": 1.0,
7734 + "control_acc": 1.0,
7735 + "selectivity": 0.0
7736 + },
7737 + {
7738 + "layer": 17,
7739 + "seed": 3,
7740 + "task_acc": 1.0,
7741 + "control_acc": 0.125,
7742 + "selectivity": 0.875
7743 + },
7744 + {
7745 + "layer": 17,
7746 + "seed": 4,
7747 + "task_acc": 1.0,
7748 + "control_acc": 0.08333333333333333,
7749 + "selectivity": 0.9166666666666666
7750 + },
7751 + {
7752 + "layer": 18,
7753 + "seed": 0,
7754 + "task_acc": 1.0,
7755 + "control_acc": 0.9166666666666666,
7756 + "selectivity": 0.08333333333333337
7757 + },
7758 + {
7759 + "layer": 18,
7760 + "seed": 1,
7761 + "task_acc": 1.0,
7762 + "control_acc": 0.9166666666666666,
7763 + "selectivity": 0.08333333333333337
7764 + },
7765 + {
7766 + "layer": 18,
7767 + "seed": 2,
7768 + "task_acc": 1.0,
7769 + "control_acc": 0.9375,
7770 + "selectivity": 0.0625
7771 + },
7772 + {
7773 + "layer": 18,
7774 + "seed": 3,
7775 + "task_acc": 1.0,
7776 + "control_acc": 0.125,
7777 + "selectivity": 0.875
7778 + },
7779 + {
7780 + "layer": 18,
7781 + "seed": 4,
7782 + "task_acc": 1.0,
7783 + "control_acc": 0.10416666666666667,
7784 + "selectivity": 0.8958333333333334
7785 + },
7786 + {
7787 + "layer": 19,
7788 + "seed": 0,
7789 + "task_acc": 1.0,
7790 + "control_acc": 0.9791666666666666,
7791 + "selectivity": 0.02083333333333337
7792 + },
7793 + {
7794 + "layer": 19,
7795 + "seed": 1,
7796 + "task_acc": 1.0,
7797 + "control_acc": 0.08333333333333333,
7798 + "selectivity": 0.9166666666666666
7799 + },
7800 + {
7801 + "layer": 19,
7802 + "seed": 2,
7803 + "task_acc": 1.0,
7804 + "control_acc": 0.8333333333333334,
7805 + "selectivity": 0.16666666666666663
7806 + },
7807 + {
7808 + "layer": 19,
7809 + "seed": 3,
7810 + "task_acc": 1.0,
7811 + "control_acc": 0.9583333333333334,
7812 + "selectivity": 0.04166666666666663
7813 + },
7814 + {
7815 + "layer": 19,
7816 + "seed": 4,
7817 + "task_acc": 1.0,
7818 + "control_acc": 0.875,
7819 + "selectivity": 0.125
7820 + },
7821 + {
7822 + "layer": 20,
7823 + "seed": 0,
7824 + "task_acc": 1.0,
7825 + "control_acc": 0.8333333333333334,
7826 + "selectivity": 0.16666666666666663
7827 + },
7828 + {
7829 + "layer": 20,
7830 + "seed": 1,
7831 + "task_acc": 1.0,
7832 + "control_acc": 0.9375,
7833 + "selectivity": 0.0625
7834 + },
7835 + {
7836 + "layer": 20,
7837 + "seed": 2,
7838 + "task_acc": 1.0,
7839 + "control_acc": 0.2708333333333333,
7840 + "selectivity": 0.7291666666666667
7841 + },
7842 + {
7843 + "layer": 20,
7844 + "seed": 3,
7845 + "task_acc": 1.0,
7846 + "control_acc": 0.9375,
7847 + "selectivity": 0.0625
7848 + },
7849 + {
7850 + "layer": 20,
7851 + "seed": 4,
7852 + "task_acc": 1.0,
7853 + "control_acc": 0.125,
7854 + "selectivity": 0.875
7855 + },
7856 + {
7857 + "layer": 21,
7858 + "seed": 0,
7859 + "task_acc": 1.0,
7860 + "control_acc": 0.8125,
7861 + "selectivity": 0.1875
7862 + },
7863 + {
7864 + "layer": 21,
7865 + "seed": 1,
7866 + "task_acc": 1.0,
7867 + "control_acc": 0.875,
7868 + "selectivity": 0.125
7869 + },
7870 + {
7871 + "layer": 21,
7872 + "seed": 2,
7873 + "task_acc": 1.0,
7874 + "control_acc": 0.25,
7875 + "selectivity": 0.75
7876 + },
7877 + {
7878 + "layer": 21,
7879 + "seed": 3,
7880 + "task_acc": 1.0,
7881 + "control_acc": 0.25,
7882 + "selectivity": 0.75
7883 + },
7884 + {
7885 + "layer": 21,
7886 + "seed": 4,
7887 + "task_acc": 1.0,
7888 + "control_acc": 0.8958333333333334,
7889 + "selectivity": 0.10416666666666663
7890 + },
7891 + {
7892 + "layer": 22,
7893 + "seed": 0,
7894 + "task_acc": 1.0,
7895 + "control_acc": 0.875,
7896 + "selectivity": 0.125
7897 + },
7898 + {
7899 + "layer": 22,
7900 + "seed": 1,
7901 + "task_acc": 1.0,
7902 + "control_acc": 0.875,
7903 + "selectivity": 0.125
7904 + },
7905 + {
7906 + "layer": 22,
7907 + "seed": 2,
7908 + "task_acc": 1.0,
7909 + "control_acc": 0.16666666666666666,
7910 + "selectivity": 0.8333333333333334
7911 + },
7912 + {
7913 + "layer": 22,
7914 + "seed": 3,
7915 + "task_acc": 1.0,
7916 + "control_acc": 1.0,
7917 + "selectivity": 0.0
7918 + },
7919 + {
7920 + "layer": 22,
7921 + "seed": 4,
7922 + "task_acc": 1.0,
7923 + "control_acc": 0.8333333333333334,
7924 + "selectivity": 0.16666666666666663
7925 + },
7926 + {
7927 + "layer": 23,
7928 + "seed": 0,
7929 + "task_acc": 1.0,
7930 + "control_acc": 0.875,
7931 + "selectivity": 0.125
7932 + },
7933 + {
7934 + "layer": 23,
7935 + "seed": 1,
7936 + "task_acc": 1.0,
7937 + "control_acc": 0.8958333333333334,
7938 + "selectivity": 0.10416666666666663
7939 + },
7940 + {
7941 + "layer": 23,
7942 + "seed": 2,
7943 + "task_acc": 1.0,
7944 + "control_acc": 0.2916666666666667,
7945 + "selectivity": 0.7083333333333333
7946 + },
7947 + {
7948 + "layer": 23,
7949 + "seed": 3,
7950 + "task_acc": 1.0,
7951 + "control_acc": 0.25,
7952 + "selectivity": 0.75
7953 + },
7954 + {
7955 + "layer": 23,
7956 + "seed": 4,
7957 + "task_acc": 1.0,
7958 + "control_acc": 0.8333333333333334,
7959 + "selectivity": 0.16666666666666663
7960 + },
7961 + {
7962 + "layer": 24,
7963 + "seed": 0,
7964 + "task_acc": 1.0,
7965 + "control_acc": 0.8333333333333334,
7966 + "selectivity": 0.16666666666666663
7967 + },
7968 + {
7969 + "layer": 24,
7970 + "seed": 1,
7971 + "task_acc": 1.0,
7972 + "control_acc": 0.8958333333333334,
7973 + "selectivity": 0.10416666666666663
7974 + },
7975 + {
7976 + "layer": 24,
7977 + "seed": 2,
7978 + "task_acc": 1.0,
7979 + "control_acc": 0.16666666666666666,
7980 + "selectivity": 0.8333333333333334
7981 + },
7982 + {
7983 + "layer": 24,
7984 + "seed": 3,
7985 + "task_acc": 1.0,
7986 + "control_acc": 0.9583333333333334,
7987 + "selectivity": 0.04166666666666663
7988 + },
7989 + {
7990 + "layer": 24,
7991 + "seed": 4,
7992 + "task_acc": 1.0,
7993 + "control_acc": 0.10416666666666667,
7994 + "selectivity": 0.8958333333333334
7995 + },
7996 + {
7997 + "layer": 25,
7998 + "seed": 0,
7999 + "task_acc": 1.0,
8000 + "control_acc": 0.9791666666666666,
8001 + "selectivity": 0.02083333333333337
8002 + },
8003 + {
8004 + "layer": 25,
8005 + "seed": 1,
8006 + "task_acc": 1.0,
8007 + "control_acc": 0.875,
8008 + "selectivity": 0.125
8009 + },
8010 + {
8011 + "layer": 25,
8012 + "seed": 2,
8013 + "task_acc": 1.0,
8014 + "control_acc": 0.125,
8015 + "selectivity": 0.875
8016 + },
8017 + {
8018 + "layer": 25,
8019 + "seed": 3,
8020 + "task_acc": 1.0,
8021 + "control_acc": 0.25,
8022 + "selectivity": 0.75
8023 + },
8024 + {
8025 + "layer": 25,
8026 + "seed": 4,
8027 + "task_acc": 1.0,
8028 + "control_acc": 0.16666666666666666,
8029 + "selectivity": 0.8333333333333334
8030 + },
8031 + {
8032 + "layer": 26,
8033 + "seed": 0,
8034 + "task_acc": 1.0,
8035 + "control_acc": 0.8958333333333334,
8036 + "selectivity": 0.10416666666666663
8037 + },
8038 + {
8039 + "layer": 26,
8040 + "seed": 1,
8041 + "task_acc": 1.0,
8042 + "control_acc": 0.875,
8043 + "selectivity": 0.125
8044 + },
8045 + {
8046 + "layer": 26,
8047 + "seed": 2,
8048 + "task_acc": 1.0,
8049 + "control_acc": 0.2708333333333333,
8050 + "selectivity": 0.7291666666666667
8051 + },
8052 + {
8053 + "layer": 26,
8054 + "seed": 3,
8055 + "task_acc": 1.0,
8056 + "control_acc": 1.0,
8057 + "selectivity": 0.0
8058 + },
8059 + {
8060 + "layer": 26,
8061 + "seed": 4,
8062 + "task_acc": 1.0,
8063 + "control_acc": 0.14583333333333334,
8064 + "selectivity": 0.8541666666666666
8065 + },
8066 + {
8067 + "layer": 27,
8068 + "seed": 0,
8069 + "task_acc": 1.0,
8070 + "control_acc": 0.9375,
8071 + "selectivity": 0.0625
8072 + },
8073 + {
8074 + "layer": 27,
8075 + "seed": 1,
8076 + "task_acc": 1.0,
8077 + "control_acc": 0.8541666666666666,
8078 + "selectivity": 0.14583333333333337
8079 + },
8080 + {
8081 + "layer": 27,
8082 + "seed": 2,
8083 + "task_acc": 1.0,
8084 + "control_acc": 0.20833333333333334,
8085 + "selectivity": 0.7916666666666666
8086 + },
8087 + {
8088 + "layer": 27,
8089 + "seed": 3,
8090 + "task_acc": 1.0,
8091 + "control_acc": 0.3125,
8092 + "selectivity": 0.6875
8093 + },
8094 + {
8095 + "layer": 27,
8096 + "seed": 4,
8097 + "task_acc": 1.0,
8098 + "control_acc": 0.10416666666666667,
8099 + "selectivity": 0.8958333333333334
8100 + }
8101 + ]
8102 + }
8103 + },
8104 + "twin": {
8105 + "lang_id_A": {
8106 + "layers": [
8107 + {
8108 + "layer": 0,
8109 + "task_acc_mean": 1.0,
8110 + "task_acc_seed_sd": 0.0,
8111 + "selectivity_mean": 0.4458333333333333,
8112 + "selectivity_ci": [
8113 + 0.1791666666666667,
8114 + 0.7249999999999999
8115 + ],
8116 + "p_boot": 0.0001,
8117 + "fdr_significant": true
8118 + },
8119 + {
8120 + "layer": 1,
8121 + "task_acc_mean": 1.0,
8122 + "task_acc_seed_sd": 0.0,
8123 + "selectivity_mean": 0.5291666666666667,
8124 + "selectivity_ci": [
8125 + 0.2375,
8126 + 0.8208333333333332
8127 + ],
8128 + "p_boot": 0.0001,
8129 + "fdr_significant": true
8130 + },
8131 + {
8132 + "layer": 2,
8133 + "task_acc_mean": 1.0,
8134 + "task_acc_seed_sd": 0.0,
8135 + "selectivity_mean": 0.2625,
8136 + "selectivity_ci": [
8137 + 0.09583333333333333,
8138 + 0.5541666666666668
8139 + ],
8140 + "p_boot": 0.0001,
8141 + "fdr_significant": true
8142 + },
8143 + {
8144 + "layer": 3,
8145 + "task_acc_mean": 1.0,
8146 + "task_acc_seed_sd": 0.0,
8147 + "selectivity_mean": 0.4000000000000001,
8148 + "selectivity_ci": [
8149 + 0.0916666666666667,
8150 + 0.7166666666666668
8151 + ],
8152 + "p_boot": 0.0001,
8153 + "fdr_significant": true
8154 + },
8155 + {
8156 + "layer": 4,
8157 + "task_acc_mean": 1.0,
8158 + "task_acc_seed_sd": 0.0,
8159 + "selectivity_mean": 0.7833333333333334,
8160 + "selectivity_ci": [
8161 + 0.4416666666666667,
8162 + 0.975
8163 + ],
8164 + "p_boot": 0.0001,
8165 + "fdr_significant": true
8166 + },
8167 + {
8168 + "layer": 5,
8169 + "task_acc_mean": 1.0,
8170 + "task_acc_seed_sd": 0.0,
8171 + "selectivity_mean": 0.8083333333333333,
8172 + "selectivity_ci": [
8173 + 0.5375,
8174 + 0.9625
8175 + ],
8176 + "p_boot": 0.0001,
8177 + "fdr_significant": true
8178 + },
8179 + {
8180 + "layer": 6,
8181 + "task_acc_mean": 1.0,
8182 + "task_acc_seed_sd": 0.0,
8183 + "selectivity_mean": 0.7375,
8184 + "selectivity_ci": [
8185 + 0.4666666666666667,
8186 + 0.925
8187 + ],
8188 + "p_boot": 0.0001,
8189 + "fdr_significant": true
8190 + },
8191 + {
8192 + "layer": 7,
8193 + "task_acc_mean": 1.0,
8194 + "task_acc_seed_sd": 0.0,
8195 + "selectivity_mean": 0.2041666666666667,
8196 + "selectivity_ci": [
8197 + 0.09583333333333337,
8198 + 0.325
8199 + ],
8200 + "p_boot": 0.0001,
8201 + "fdr_significant": true
8202 + },
8203 + {
8204 + "layer": 8,
8205 + "task_acc_mean": 1.0,
8206 + "task_acc_seed_sd": 0.0,
8207 + "selectivity_mean": 0.6416666666666667,
8208 + "selectivity_ci": [
8209 + 0.2875000000000001,
8210 + 0.9791666666666666
8211 + ],
8212 + "p_boot": 0.0001,
8213 + "fdr_significant": true
8214 + },
8215 + {
8216 + "layer": 9,
8217 + "task_acc_mean": 1.0,
8218 + "task_acc_seed_sd": 0.0,
8219 + "selectivity_mean": 0.6166666666666667,
8220 + "selectivity_ci": [
8221 + 0.29166666666666663,
8222 + 0.9416666666666668
8223 + ],
8224 + "p_boot": 0.0001,
8225 + "fdr_significant": true
8226 + },
8227 + {
8228 + "layer": 10,
8229 + "task_acc_mean": 1.0,
8230 + "task_acc_seed_sd": 0.0,
8231 + "selectivity_mean": 0.35416666666666663,
8232 + "selectivity_ci": [
8233 + 0.1125,
8234 + 0.6583333333333332
8235 + ],
8236 + "p_boot": 0.0001,
8237 + "fdr_significant": true
8238 + },
8239 + {
8240 + "layer": 11,
8241 + "task_acc_mean": 1.0,
8242 + "task_acc_seed_sd": 0.0,
8243 + "selectivity_mean": 0.5791666666666666,
8244 + "selectivity_ci": [
8245 + 0.2291666666666666,
8246 + 0.9083333333333332
8247 + ],
8248 + "p_boot": 0.0001,
8249 + "fdr_significant": true
8250 + },
8251 + {
8252 + "layer": 12,
8253 + "task_acc_mean": 1.0,
8254 + "task_acc_seed_sd": 0.0,
8255 + "selectivity_mean": 0.5958333333333333,
8256 + "selectivity_ci": [
8257 + 0.24999999999999992,
8258 + 0.9375
8259 + ],
8260 + "p_boot": 0.0001,
8261 + "fdr_significant": true
8262 + },
8263 + {
8264 + "layer": 13,
8265 + "task_acc_mean": 1.0,
8266 + "task_acc_seed_sd": 0.0,
8267 + "selectivity_mean": 0.3041666666666666,
8268 + "selectivity_ci": [
8269 + 0.08749999999999998,
8270 + 0.6041666666666666
8271 + ],
8272 + "p_boot": 0.0001,
8273 + "fdr_significant": true
8274 + },
8275 + {
8276 + "layer": 14,
8277 + "task_acc_mean": 1.0,
8278 + "task_acc_seed_sd": 0.0,
8279 + "selectivity_mean": 0.22916666666666669,
8280 + "selectivity_ci": [
8281 + 0.0375,
8282 + 0.5583333333333333
8283 + ],
8284 + "p_boot": 0.0001,
8285 + "fdr_significant": true
8286 + },
8287 + {
8288 + "layer": 15,
8289 + "task_acc_mean": 1.0,
8290 + "task_acc_seed_sd": 0.0,
8291 + "selectivity_mean": 0.2416666666666666,
8292 + "selectivity_ci": [
8293 + 0.05833333333333333,
8294 + 0.5291666666666666
8295 + ],
8296 + "p_boot": 0.0001,
8297 + "fdr_significant": true
8298 + },
8299 + {
8300 + "layer": 16,
8301 + "task_acc_mean": 1.0,
8302 + "task_acc_seed_sd": 0.0,
8303 + "selectivity_mean": 0.4416666666666666,
8304 + "selectivity_ci": [
8305 + 0.13333333333333333,
8306 + 0.7791666666666666
8307 + ],
8308 + "p_boot": 0.0001,
8309 + "fdr_significant": true
8310 + },
8311 + {
8312 + "layer": 17,
8313 + "task_acc_mean": 1.0,
8314 + "task_acc_seed_sd": 0.0,
8315 + "selectivity_mean": 0.2625,
8316 + "selectivity_ci": [
8317 + 0.0375,
8318 + 0.6416666666666666
8319 + ],
8320 + "p_boot": 0.0004,
8321 + "fdr_significant": true
8322 + },
8323 + {
8324 + "layer": 18,
8325 + "task_acc_mean": 1.0,
8326 + "task_acc_seed_sd": 0.0,
8327 + "selectivity_mean": 0.5916666666666666,
8328 + "selectivity_ci": [
8329 + 0.2833333333333334,
8330 + 0.9
8331 + ],
8332 + "p_boot": 0.0001,
8333 + "fdr_significant": true
8334 + },
8335 + {
8336 + "layer": 19,
8337 + "task_acc_mean": 1.0,
8338 + "task_acc_seed_sd": 0.0,
8339 + "selectivity_mean": 0.4916666666666667,
8340 + "selectivity_ci": [
8341 + 0.15,
8342 + 0.8333333333333334
8343 + ],
8344 + "p_boot": 0.0001,
8345 + "fdr_significant": true
8346 + },
8347 + {
8348 + "layer": 20,
8349 + "task_acc_mean": 1.0,
8350 + "task_acc_seed_sd": 0.0,
8351 + "selectivity_mean": 0.2333333333333333,
8352 + "selectivity_ci": [
8353 + 0.02083333333333335,
8354 + 0.6083333333333333
8355 + ],
8356 + "p_boot": 0.0003,
8357 + "fdr_significant": true
8358 + },
8359 + {
8360 + "layer": 21,
8361 + "task_acc_mean": 1.0,
8362 + "task_acc_seed_sd": 0.0,
8363 + "selectivity_mean": 0.4291666666666667,
8364 + "selectivity_ci": [
8365 + 0.1375,
8366 + 0.7208333333333334
8367 + ],
8368 + "p_boot": 0.0001,
8369 + "fdr_significant": true
8370 + },
8371 + {
8372 + "layer": 22,
8373 + "task_acc_mean": 1.0,
8374 + "task_acc_seed_sd": 0.0,
8375 + "selectivity_mean": 0.4625,
8376 + "selectivity_ci": [
8377 + 0.15416666666666665,
8378 + 0.8041666666666666
8379 + ],
8380 + "p_boot": 0.0001,
8381 + "fdr_significant": true
8382 + },
8383 + {
8384 + "layer": 23,
8385 + "task_acc_mean": 1.0,
8386 + "task_acc_seed_sd": 0.0,
8387 + "selectivity_mean": 0.4083333333333334,
8388 + "selectivity_ci": [
8389 + 0.054166666666666696,
8390 + 0.7625
8391 + ],
8392 + "p_boot": 0.0001,
8393 + "fdr_significant": true
8394 + },
8395 + {
8396 + "layer": 24,
8397 + "task_acc_mean": 1.0,
8398 + "task_acc_seed_sd": 0.0,
8399 + "selectivity_mean": 0.22916666666666669,
8400 + "selectivity_ci": [
8401 + 0.02083333333333335,
8402 + 0.5625
8403 + ],
8404 + "p_boot": 0.0004,
8405 + "fdr_significant": true
8406 + },
8407 + {
8408 + "layer": 25,
8409 + "task_acc_mean": 1.0,
8410 + "task_acc_seed_sd": 0.0,
8411 + "selectivity_mean": 0.6291666666666667,
8412 + "selectivity_ci": [
8413 + 0.2958333333333333,
8414 + 0.9583333333333334
8415 + ],
8416 + "p_boot": 0.0001,
8417 + "fdr_significant": true
8418 + },
8419 + {
8420 + "layer": 26,
8421 + "task_acc_mean": 1.0,
8422 + "task_acc_seed_sd": 0.0,
8423 + "selectivity_mean": 0.4083333333333333,
8424 + "selectivity_ci": [
8425 + 0.09583333333333333,
8426 + 0.7541666666666667
8427 + ],
8428 + "p_boot": 0.0001,
8429 + "fdr_significant": true
8430 + },
8431 + {
8432 + "layer": 27,
8433 + "task_acc_mean": 1.0,
8434 + "task_acc_seed_sd": 0.0,
8435 + "selectivity_mean": 0.4625,
8436 + "selectivity_ci": [
8437 + 0.175,
8438 + 0.7916666666666667
8439 + ],
8440 + "p_boot": 0.0001,
8441 + "fdr_significant": true
8442 + }
8443 + ],
8444 + "replication_rate_topk": {
8445 + "k": 5,
8446 + "point": 1.0,
8447 + "ci": [
8448 + 1.0,
8449 + 1.0
8450 + ]
8451 + },
8452 + "n_fdr_significant": 28,
8453 + "cells": [
8454 + {
8455 + "layer": 0,
8456 + "seed": 0,
8457 + "task_acc": 1.0,
8458 + "control_acc": 0.7916666666666666,
8459 + "selectivity": 0.20833333333333337
8460 + },
8461 + {
8462 + "layer": 0,
8463 + "seed": 1,
8464 + "task_acc": 1.0,
8465 + "control_acc": 0.7083333333333334,
8466 + "selectivity": 0.29166666666666663
8467 + },
8468 + {
8469 + "layer": 0,
8470 + "seed": 2,
8471 + "task_acc": 1.0,
8472 + "control_acc": 0.14583333333333334,
8473 + "selectivity": 0.8541666666666666
8474 + },
8475 + {
8476 + "layer": 0,
8477 + "seed": 3,
8478 + "task_acc": 1.0,
8479 + "control_acc": 0.9375,
8480 + "selectivity": 0.0625
8481 + },
8482 + {
8483 + "layer": 0,
8484 + "seed": 4,
8485 + "task_acc": 1.0,
8486 + "control_acc": 0.1875,
8487 + "selectivity": 0.8125
8488 + },
8489 + {
8490 + "layer": 1,
8491 + "seed": 0,
8492 + "task_acc": 1.0,
8493 + "control_acc": 0.875,
8494 + "selectivity": 0.125
8495 + },
8496 + {
8497 + "layer": 1,
8498 + "seed": 1,
8499 + "task_acc": 1.0,
8500 + "control_acc": 0.14583333333333334,
8501 + "selectivity": 0.8541666666666666
8502 + },
8503 + {
8504 + "layer": 1,
8505 + "seed": 2,
8506 + "task_acc": 1.0,
8507 + "control_acc": 0.22916666666666666,
8508 + "selectivity": 0.7708333333333334
8509 + },
8510 + {
8511 + "layer": 1,
8512 + "seed": 3,
8513 + "task_acc": 1.0,
8514 + "control_acc": 0.9166666666666666,
8515 + "selectivity": 0.08333333333333337
8516 + },
8517 + {
8518 + "layer": 1,
8519 + "seed": 4,
8520 + "task_acc": 1.0,
8521 + "control_acc": 0.1875,
8522 + "selectivity": 0.8125
8523 + },
8524 + {
8525 + "layer": 2,
8526 + "seed": 0,
8527 + "task_acc": 1.0,
8528 + "control_acc": 0.8125,
8529 + "selectivity": 0.1875
8530 + },
8531 + {
8532 + "layer": 2,
8533 + "seed": 1,
8534 + "task_acc": 1.0,
8535 + "control_acc": 0.8958333333333334,
8536 + "selectivity": 0.10416666666666663
8537 + },
8538 + {
8539 + "layer": 2,
8540 + "seed": 2,
8541 + "task_acc": 1.0,
8542 + "control_acc": 0.16666666666666666,
8543 + "selectivity": 0.8333333333333334
8544 + },
8545 + {
8546 + "layer": 2,
8547 + "seed": 3,
8548 + "task_acc": 1.0,
8549 + "control_acc": 0.9166666666666666,
8550 + "selectivity": 0.08333333333333337
8551 + },
8552 + {
8553 + "layer": 2,
8554 + "seed": 4,
8555 + "task_acc": 1.0,
8556 + "control_acc": 0.8958333333333334,
8557 + "selectivity": 0.10416666666666663
8558 + },
8559 + {
8560 + "layer": 3,
8561 + "seed": 0,
8562 + "task_acc": 1.0,
8563 + "control_acc": 0.875,
8564 + "selectivity": 0.125
8565 + },
8566 + {
8567 + "layer": 3,
8568 + "seed": 1,
8569 + "task_acc": 1.0,
8570 + "control_acc": 0.1875,
8571 + "selectivity": 0.8125
8572 + },
8573 + {
8574 + "layer": 3,
8575 + "seed": 2,
8576 + "task_acc": 1.0,
8577 + "control_acc": 0.10416666666666667,
8578 + "selectivity": 0.8958333333333334
8579 + },
8580 + {
8581 + "layer": 3,
8582 + "seed": 3,
8583 + "task_acc": 1.0,
8584 + "control_acc": 0.9166666666666666,
8585 + "selectivity": 0.08333333333333337
8586 + },
8587 + {
8588 + "layer": 3,
8589 + "seed": 4,
8590 + "task_acc": 1.0,
8591 + "control_acc": 0.9166666666666666,
8592 + "selectivity": 0.08333333333333337
8593 + },
8594 + {
8595 + "layer": 4,
8596 + "seed": 0,
8597 + "task_acc": 1.0,
8598 + "control_acc": 0.0,
8599 + "selectivity": 1.0
8600 + },
8601 + {
8602 + "layer": 4,
8603 + "seed": 1,
8604 + "task_acc": 1.0,
8605 + "control_acc": 0.041666666666666664,
8606 + "selectivity": 0.9583333333333334
8607 + },
8608 + {
8609 + "layer": 4,
8610 + "seed": 2,
8611 + "task_acc": 1.0,
8612 + "control_acc": 0.125,
8613 + "selectivity": 0.875
8614 + },
8615 + {
8616 + "layer": 4,
8617 + "seed": 3,
8618 + "task_acc": 1.0,
8619 + "control_acc": 0.041666666666666664,
8620 + "selectivity": 0.9583333333333334
8621 + },
8622 + {
8623 + "layer": 4,
8624 + "seed": 4,
8625 + "task_acc": 1.0,
8626 + "control_acc": 0.875,
8627 + "selectivity": 0.125
8628 + },
8629 + {
8630 + "layer": 5,
8631 + "seed": 0,
8632 + "task_acc": 1.0,
8633 + "control_acc": 0.020833333333333332,
8634 + "selectivity": 0.9791666666666666
8635 + },
8636 + {
8637 + "layer": 5,
8638 + "seed": 1,
8639 + "task_acc": 1.0,
8640 + "control_acc": 0.041666666666666664,
8641 + "selectivity": 0.9583333333333334
8642 + },
8643 + {
8644 + "layer": 5,
8645 + "seed": 2,
8646 + "task_acc": 1.0,
8647 + "control_acc": 0.7291666666666666,
8648 + "selectivity": 0.27083333333333337
8649 + },
8650 + {
8651 + "layer": 5,
8652 + "seed": 3,
8653 + "task_acc": 1.0,
8654 + "control_acc": 0.08333333333333333,
8655 + "selectivity": 0.9166666666666666
8656 + },
8657 + {
8658 + "layer": 5,
8659 + "seed": 4,
8660 + "task_acc": 1.0,
8661 + "control_acc": 0.08333333333333333,
8662 + "selectivity": 0.9166666666666666
8663 + },
8664 + {
8665 + "layer": 6,
8666 + "seed": 0,
8667 + "task_acc": 1.0,
8668 + "control_acc": 0.0,
8669 + "selectivity": 1.0
8670 + },
8671 + {
8672 + "layer": 6,
8673 + "seed": 1,
8674 + "task_acc": 1.0,
8675 + "control_acc": 0.14583333333333334,
8676 + "selectivity": 0.8541666666666666
8677 + },
8678 + {
8679 + "layer": 6,
8680 + "seed": 2,
8681 + "task_acc": 1.0,
8682 + "control_acc": 0.7916666666666666,
8683 + "selectivity": 0.20833333333333337
8684 + },
8685 + {
8686 + "layer": 6,
8687 + "seed": 3,
8688 + "task_acc": 1.0,
8689 + "control_acc": 0.22916666666666666,
8690 + "selectivity": 0.7708333333333334
8691 + },
8692 + {
8693 + "layer": 6,
8694 + "seed": 4,
8695 + "task_acc": 1.0,
8696 + "control_acc": 0.14583333333333334,
8697 + "selectivity": 0.8541666666666666
8698 + },
8699 + {
8700 + "layer": 7,
8701 + "seed": 0,
8702 + "task_acc": 1.0,
8703 + "control_acc": 0.8541666666666666,
8704 + "selectivity": 0.14583333333333337
8705 + },
8706 + {
8707 + "layer": 7,
8708 + "seed": 1,
8709 + "task_acc": 1.0,
8710 + "control_acc": 0.5833333333333334,
8711 + "selectivity": 0.41666666666666663
8712 + },
8713 + {
8714 + "layer": 7,
8715 + "seed": 2,
8716 + "task_acc": 1.0,
8717 + "control_acc": 0.7916666666666666,
8718 + "selectivity": 0.20833333333333337
8719 + },
8720 + {
8721 + "layer": 7,
8722 + "seed": 3,
8723 + "task_acc": 1.0,
8724 + "control_acc": 0.9791666666666666,
8725 + "selectivity": 0.02083333333333337
8726 + },
8727 + {
8728 + "layer": 7,
8729 + "seed": 4,
8730 + "task_acc": 1.0,
8731 + "control_acc": 0.7708333333333334,
8732 + "selectivity": 0.22916666666666663
8733 + },
8734 + {
8735 + "layer": 8,
8736 + "seed": 0,
8737 + "task_acc": 1.0,
8738 + "control_acc": 0.0,
8739 + "selectivity": 1.0
8740 + },
8741 + {
8742 + "layer": 8,
8743 + "seed": 1,
8744 + "task_acc": 1.0,
8745 + "control_acc": 0.0625,
8746 + "selectivity": 0.9375
8747 + },
8748 + {
8749 + "layer": 8,
8750 + "seed": 2,
8751 + "task_acc": 1.0,
8752 + "control_acc": 0.7916666666666666,
8753 + "selectivity": 0.20833333333333337
8754 + },
8755 + {
8756 + "layer": 8,
8757 + "seed": 3,
8758 + "task_acc": 1.0,
8759 + "control_acc": 0.020833333333333332,
8760 + "selectivity": 0.9791666666666666
8761 + },
8762 + {
8763 + "layer": 8,
8764 + "seed": 4,
8765 + "task_acc": 1.0,
8766 + "control_acc": 0.9166666666666666,
8767 + "selectivity": 0.08333333333333337
8768 + },
8769 + {
8770 + "layer": 9,
8771 + "seed": 0,
8772 + "task_acc": 1.0,
8773 + "control_acc": 0.0,
8774 + "selectivity": 1.0
8775 + },
8776 + {
8777 + "layer": 9,
8778 + "seed": 1,
8779 + "task_acc": 1.0,
8780 + "control_acc": 0.0625,
8781 + "selectivity": 0.9375
8782 + },
8783 + {
8784 + "layer": 9,
8785 + "seed": 2,
8786 + "task_acc": 1.0,
8787 + "control_acc": 0.7916666666666666,
8788 + "selectivity": 0.20833333333333337
8789 + },
8790 + {
8791 + "layer": 9,
8792 + "seed": 3,
8793 + "task_acc": 1.0,
8794 + "control_acc": 0.16666666666666666,
8795 + "selectivity": 0.8333333333333334
8796 + },
8797 + {
8798 + "layer": 9,
8799 + "seed": 4,
8800 + "task_acc": 1.0,
8801 + "control_acc": 0.8958333333333334,
8802 + "selectivity": 0.10416666666666663
8803 + },
8804 + {
8805 + "layer": 10,
8806 + "seed": 0,
8807 + "task_acc": 1.0,
8808 + "control_acc": 0.8125,
8809 + "selectivity": 0.1875
8810 + },
8811 + {
8812 + "layer": 10,
8813 + "seed": 1,
8814 + "task_acc": 1.0,
8815 + "control_acc": 0.6458333333333334,
8816 + "selectivity": 0.35416666666666663
8817 + },
8818 + {
8819 + "layer": 10,
8820 + "seed": 2,
8821 + "task_acc": 1.0,
8822 + "control_acc": 0.6875,
8823 + "selectivity": 0.3125
8824 + },
8825 + {
8826 + "layer": 10,
8827 + "seed": 3,
8828 + "task_acc": 1.0,
8829 + "control_acc": 1.0,
8830 + "selectivity": 0.0
8831 + },
8832 + {
8833 + "layer": 10,
8834 + "seed": 4,
8835 + "task_acc": 1.0,
8836 + "control_acc": 0.08333333333333333,
8837 + "selectivity": 0.9166666666666666
8838 + },
8839 + {
8840 + "layer": 11,
8841 + "seed": 0,
8842 + "task_acc": 1.0,
8843 + "control_acc": 0.0,
8844 + "selectivity": 1.0
8845 + },
8846 + {
8847 + "layer": 11,
8848 + "seed": 1,
8849 + "task_acc": 1.0,
8850 + "control_acc": 0.14583333333333334,
8851 + "selectivity": 0.8541666666666666
8852 + },
8853 + {
8854 + "layer": 11,
8855 + "seed": 2,
8856 + "task_acc": 1.0,
8857 + "control_acc": 0.8333333333333334,
8858 + "selectivity": 0.16666666666666663
8859 + },
8860 + {
8861 + "layer": 11,
8862 + "seed": 3,
8863 + "task_acc": 1.0,
8864 + "control_acc": 0.9583333333333334,
8865 + "selectivity": 0.04166666666666663
8866 + },
8867 + {
8868 + "layer": 11,
8869 + "seed": 4,
8870 + "task_acc": 1.0,
8871 + "control_acc": 0.16666666666666666,
8872 + "selectivity": 0.8333333333333334
8873 + },
8874 + {
8875 + "layer": 12,
8876 + "seed": 0,
8877 + "task_acc": 1.0,
8878 + "control_acc": 0.041666666666666664,
8879 + "selectivity": 0.9583333333333334
8880 + },
8881 + {
8882 + "layer": 12,
8883 + "seed": 1,
8884 + "task_acc": 1.0,
8885 + "control_acc": 0.041666666666666664,
8886 + "selectivity": 0.9583333333333334
8887 + },
8888 + {
8889 + "layer": 12,
8890 + "seed": 2,
8891 + "task_acc": 1.0,
8892 + "control_acc": 0.8333333333333334,
8893 + "selectivity": 0.16666666666666663
8894 + },
8895 + {
8896 + "layer": 12,
8897 + "seed": 3,
8898 + "task_acc": 1.0,
8899 + "control_acc": 0.14583333333333334,
8900 + "selectivity": 0.8541666666666666
8901 + },
8902 + {
8903 + "layer": 12,
8904 + "seed": 4,
8905 + "task_acc": 1.0,
8906 + "control_acc": 0.9583333333333334,
8907 + "selectivity": 0.04166666666666663
8908 + },
8909 + {
8910 + "layer": 13,
8911 + "seed": 0,
8912 + "task_acc": 1.0,
8913 + "control_acc": 0.8958333333333334,
8914 + "selectivity": 0.10416666666666663
8915 + },
8916 + {
8917 + "layer": 13,
8918 + "seed": 1,
8919 + "task_acc": 1.0,
8920 + "control_acc": 0.6458333333333334,
8921 + "selectivity": 0.35416666666666663
8922 + },
8923 + {
8924 + "layer": 13,
8925 + "seed": 2,
8926 + "task_acc": 1.0,
8927 + "control_acc": 0.8541666666666666,
8928 + "selectivity": 0.14583333333333337
8929 + },
8930 + {
8931 + "layer": 13,
8932 + "seed": 3,
8933 + "task_acc": 1.0,
8934 + "control_acc": 0.125,
8935 + "selectivity": 0.875
8936 + },
8937 + {
8938 + "layer": 13,
8939 + "seed": 4,
8940 + "task_acc": 1.0,
8941 + "control_acc": 0.9583333333333334,
8942 + "selectivity": 0.04166666666666663
8943 + },
8944 + {
8945 + "layer": 14,
8946 + "seed": 0,
8947 + "task_acc": 1.0,
8948 + "control_acc": 0.9791666666666666,
8949 + "selectivity": 0.02083333333333337
8950 + },
8951 + {
8952 + "layer": 14,
8953 + "seed": 1,
8954 + "task_acc": 1.0,
8955 + "control_acc": 0.9583333333333334,
8956 + "selectivity": 0.04166666666666663
8957 + },
8958 + {
8959 + "layer": 14,
8960 + "seed": 2,
8961 + "task_acc": 1.0,
8962 + "control_acc": 0.8541666666666666,
8963 + "selectivity": 0.14583333333333337
8964 + },
8965 + {
8966 + "layer": 14,
8967 + "seed": 3,
8968 + "task_acc": 1.0,
8969 + "control_acc": 0.125,
8970 + "selectivity": 0.875
8971 + },
8972 + {
8973 + "layer": 14,
8974 + "seed": 4,
8975 + "task_acc": 1.0,
8976 + "control_acc": 0.9375,
8977 + "selectivity": 0.0625
8978 + },
8979 + {
8980 + "layer": 15,
8981 + "seed": 0,
8982 + "task_acc": 1.0,
8983 + "control_acc": 0.9166666666666666,
8984 + "selectivity": 0.08333333333333337
8985 + },
8986 + {
8987 + "layer": 15,
8988 + "seed": 1,
8989 + "task_acc": 1.0,
8990 + "control_acc": 0.9375,
8991 + "selectivity": 0.0625
8992 + },
8993 + {
8994 + "layer": 15,
8995 + "seed": 2,
8996 + "task_acc": 1.0,
8997 + "control_acc": 0.7708333333333334,
8998 + "selectivity": 0.22916666666666663
8999 + },
9000 + {
9001 + "layer": 15,
9002 + "seed": 3,
9003 + "task_acc": 1.0,
9004 + "control_acc": 0.20833333333333334,
9005 + "selectivity": 0.7916666666666666
9006 + },
9007 + {
9008 + "layer": 15,
9009 + "seed": 4,
9010 + "task_acc": 1.0,
9011 + "control_acc": 0.9583333333333334,
9012 + "selectivity": 0.04166666666666663
9013 + },
9014 + {
9015 + "layer": 16,
9016 + "seed": 0,
9017 + "task_acc": 1.0,
9018 + "control_acc": 0.020833333333333332,
9019 + "selectivity": 0.9791666666666666
9020 + },
9021 + {
9022 + "layer": 16,
9023 + "seed": 1,
9024 + "task_acc": 1.0,
9025 + "control_acc": 0.8333333333333334,
9026 + "selectivity": 0.16666666666666663
9027 + },
9028 + {
9029 + "layer": 16,
9030 + "seed": 2,
9031 + "task_acc": 1.0,
9032 + "control_acc": 0.7916666666666666,
9033 + "selectivity": 0.20833333333333337
9034 + },
9035 + {
9036 + "layer": 16,
9037 + "seed": 3,
9038 + "task_acc": 1.0,
9039 + "control_acc": 0.20833333333333334,
9040 + "selectivity": 0.7916666666666666
9041 + },
9042 + {
9043 + "layer": 16,
9044 + "seed": 4,
9045 + "task_acc": 1.0,
9046 + "control_acc": 0.9375,
9047 + "selectivity": 0.0625
9048 + },
9049 + {
9050 + "layer": 17,
9051 + "seed": 0,
9052 + "task_acc": 1.0,
9053 + "control_acc": 0.020833333333333332,
9054 + "selectivity": 0.9791666666666666
9055 + },
9056 + {
9057 + "layer": 17,
9058 + "seed": 1,
9059 + "task_acc": 1.0,
9060 + "control_acc": 0.9375,
9061 + "selectivity": 0.0625
9062 + },
9063 + {
9064 + "layer": 17,
9065 + "seed": 2,
9066 + "task_acc": 1.0,
9067 + "control_acc": 0.7916666666666666,
9068 + "selectivity": 0.20833333333333337
9069 + },
9070 + {
9071 + "layer": 17,
9072 + "seed": 3,
9073 + "task_acc": 1.0,
9074 + "control_acc": 1.0,
9075 + "selectivity": 0.0
9076 + },
9077 + {
9078 + "layer": 17,
9079 + "seed": 4,
9080 + "task_acc": 1.0,
9081 + "control_acc": 0.9375,
9082 + "selectivity": 0.0625
9083 + },
9084 + {
9085 + "layer": 18,
9086 + "seed": 0,
9087 + "task_acc": 1.0,
9088 + "control_acc": 0.0,
9089 + "selectivity": 1.0
9090 + },
9091 + {
9092 + "layer": 18,
9093 + "seed": 1,
9094 + "task_acc": 1.0,
9095 + "control_acc": 0.8333333333333334,
9096 + "selectivity": 0.16666666666666663
9097 + },
9098 + {
9099 + "layer": 18,
9100 + "seed": 2,
9101 + "task_acc": 1.0,
9102 + "control_acc": 0.8541666666666666,
9103 + "selectivity": 0.14583333333333337
9104 + },
9105 + {
9106 + "layer": 18,
9107 + "seed": 3,
9108 + "task_acc": 1.0,
9109 + "control_acc": 0.20833333333333334,
9110 + "selectivity": 0.7916666666666666
9111 + },
9112 + {
9113 + "layer": 18,
9114 + "seed": 4,
9115 + "task_acc": 1.0,
9116 + "control_acc": 0.14583333333333334,
9117 + "selectivity": 0.8541666666666666
9118 + },
9119 + {
9120 + "layer": 19,
9121 + "seed": 0,
9122 + "task_acc": 1.0,
9123 + "control_acc": 0.0,
9124 + "selectivity": 1.0
9125 + },
9126 + {
9127 + "layer": 19,
9128 + "seed": 1,
9129 + "task_acc": 1.0,
9130 + "control_acc": 0.041666666666666664,
9131 + "selectivity": 0.9583333333333334
9132 + },
9133 + {
9134 + "layer": 19,
9135 + "seed": 2,
9136 + "task_acc": 1.0,
9137 + "control_acc": 0.7708333333333334,
9138 + "selectivity": 0.22916666666666663
9139 + },
9140 + {
9141 + "layer": 19,
9142 + "seed": 3,
9143 + "task_acc": 1.0,
9144 + "control_acc": 0.9791666666666666,
9145 + "selectivity": 0.02083333333333337
9146 + },
9147 + {
9148 + "layer": 19,
9149 + "seed": 4,
9150 + "task_acc": 1.0,
9151 + "control_acc": 0.75,
9152 + "selectivity": 0.25
9153 + },
9154 + {
9155 + "layer": 20,
9156 + "seed": 0,
9157 + "task_acc": 1.0,
9158 + "control_acc": 0.9791666666666666,
9159 + "selectivity": 0.02083333333333337
9160 + },
9161 + {
9162 + "layer": 20,
9163 + "seed": 1,
9164 + "task_acc": 1.0,
9165 + "control_acc": 0.020833333333333332,
9166 + "selectivity": 0.9791666666666666
9167 + },
9168 + {
9169 + "layer": 20,
9170 + "seed": 2,
9171 + "task_acc": 1.0,
9172 + "control_acc": 0.8958333333333334,
9173 + "selectivity": 0.10416666666666663
9174 + },
9175 + {
9176 + "layer": 20,
9177 + "seed": 3,
9178 + "task_acc": 1.0,
9179 + "control_acc": 1.0,
9180 + "selectivity": 0.0
9181 + },
9182 + {
9183 + "layer": 20,
9184 + "seed": 4,
9185 + "task_acc": 1.0,
9186 + "control_acc": 0.9375,
9187 + "selectivity": 0.0625
9188 + },
9189 + {
9190 + "layer": 21,
9191 + "seed": 0,
9192 + "task_acc": 1.0,
9193 + "control_acc": 0.16666666666666666,
9194 + "selectivity": 0.8333333333333334
9195 + },
9196 + {
9197 + "layer": 21,
9198 + "seed": 1,
9199 + "task_acc": 1.0,
9200 + "control_acc": 0.7291666666666666,
9201 + "selectivity": 0.27083333333333337
9202 + },
9203 + {
9204 + "layer": 21,
9205 + "seed": 2,
9206 + "task_acc": 1.0,
9207 + "control_acc": 0.8541666666666666,
9208 + "selectivity": 0.14583333333333337
9209 + },
9210 + {
9211 + "layer": 21,
9212 + "seed": 3,
9213 + "task_acc": 1.0,
9214 + "control_acc": 0.16666666666666666,
9215 + "selectivity": 0.8333333333333334
9216 + },
9217 + {
9218 + "layer": 21,
9219 + "seed": 4,
9220 + "task_acc": 1.0,
9221 + "control_acc": 0.9375,
9222 + "selectivity": 0.0625
9223 + },
9224 + {
9225 + "layer": 22,
9226 + "seed": 0,
9227 + "task_acc": 1.0,
9228 + "control_acc": 0.0,
9229 + "selectivity": 1.0
9230 + },
9231 + {
9232 + "layer": 22,
9233 + "seed": 1,
9234 + "task_acc": 1.0,
9235 + "control_acc": 0.8958333333333334,
9236 + "selectivity": 0.10416666666666663
9237 + },
9238 + {
9239 + "layer": 22,
9240 + "seed": 2,
9241 + "task_acc": 1.0,
9242 + "control_acc": 0.8125,
9243 + "selectivity": 0.1875
9244 + },
9245 + {
9246 + "layer": 22,
9247 + "seed": 3,
9248 + "task_acc": 1.0,
9249 + "control_acc": 0.16666666666666666,
9250 + "selectivity": 0.8333333333333334
9251 + },
9252 + {
9253 + "layer": 22,
9254 + "seed": 4,
9255 + "task_acc": 1.0,
9256 + "control_acc": 0.8125,
9257 + "selectivity": 0.1875
9258 + },
9259 + {
9260 + "layer": 23,
9261 + "seed": 0,
9262 + "task_acc": 1.0,
9263 + "control_acc": 0.0625,
9264 + "selectivity": 0.9375
9265 + },
9266 + {
9267 + "layer": 23,
9268 + "seed": 1,
9269 + "task_acc": 1.0,
9270 + "control_acc": 0.9791666666666666,
9271 + "selectivity": 0.02083333333333337
9272 + },
9273 + {
9274 + "layer": 23,
9275 + "seed": 2,
9276 + "task_acc": 1.0,
9277 + "control_acc": 0.8125,
9278 + "selectivity": 0.1875
9279 + },
9280 + {
9281 + "layer": 23,
9282 + "seed": 3,
9283 + "task_acc": 1.0,
9284 + "control_acc": 0.125,
9285 + "selectivity": 0.875
9286 + },
9287 + {
9288 + "layer": 23,
9289 + "seed": 4,
9290 + "task_acc": 1.0,
9291 + "control_acc": 0.9791666666666666,
9292 + "selectivity": 0.02083333333333337
9293 + },
9294 + {
9295 + "layer": 24,
9296 + "seed": 0,
9297 + "task_acc": 1.0,
9298 + "control_acc": 0.9791666666666666,
9299 + "selectivity": 0.02083333333333337
9300 + },
9301 + {
9302 + "layer": 24,
9303 + "seed": 1,
9304 + "task_acc": 1.0,
9305 + "control_acc": 0.9375,
9306 + "selectivity": 0.0625
9307 + },
9308 + {
9309 + "layer": 24,
9310 + "seed": 2,
9311 + "task_acc": 1.0,
9312 + "control_acc": 0.8125,
9313 + "selectivity": 0.1875
9314 + },
9315 + {
9316 + "layer": 24,
9317 + "seed": 3,
9318 + "task_acc": 1.0,
9319 + "control_acc": 0.125,
9320 + "selectivity": 0.875
9321 + },
9322 + {
9323 + "layer": 24,
9324 + "seed": 4,
9325 + "task_acc": 1.0,
9326 + "control_acc": 1.0,
9327 + "selectivity": 0.0
9328 + },
9329 + {
9330 + "layer": 25,
9331 + "seed": 0,
9332 + "task_acc": 1.0,
9333 + "control_acc": 0.0,
9334 + "selectivity": 1.0
9335 + },
9336 + {
9337 + "layer": 25,
9338 + "seed": 1,
9339 + "task_acc": 1.0,
9340 + "control_acc": 0.8958333333333334,
9341 + "selectivity": 0.10416666666666663
9342 + },
9343 + {
9344 + "layer": 25,
9345 + "seed": 2,
9346 + "task_acc": 1.0,
9347 + "control_acc": 0.7916666666666666,
9348 + "selectivity": 0.20833333333333337
9349 + },
9350 + {
9351 + "layer": 25,
9352 + "seed": 3,
9353 + "task_acc": 1.0,
9354 + "control_acc": 0.125,
9355 + "selectivity": 0.875
9356 + },
9357 + {
9358 + "layer": 25,
9359 + "seed": 4,
9360 + "task_acc": 1.0,
9361 + "control_acc": 0.041666666666666664,
9362 + "selectivity": 0.9583333333333334
9363 + },
9364 + {
9365 + "layer": 26,
9366 + "seed": 0,
9367 + "task_acc": 1.0,
9368 + "control_acc": 0.041666666666666664,
9369 + "selectivity": 0.9583333333333334
9370 + },
9371 + {
9372 + "layer": 26,
9373 + "seed": 1,
9374 + "task_acc": 1.0,
9375 + "control_acc": 0.8541666666666666,
9376 + "selectivity": 0.14583333333333337
9377 + },
9378 + {
9379 + "layer": 26,
9380 + "seed": 2,
9381 + "task_acc": 1.0,
9382 + "control_acc": 0.875,
9383 + "selectivity": 0.125
9384 + },
9385 + {
9386 + "layer": 26,
9387 + "seed": 3,
9388 + "task_acc": 1.0,
9389 + "control_acc": 0.22916666666666666,
9390 + "selectivity": 0.7708333333333334
9391 + },
9392 + {
9393 + "layer": 26,
9394 + "seed": 4,
9395 + "task_acc": 1.0,
9396 + "control_acc": 0.9583333333333334,
9397 + "selectivity": 0.04166666666666663
9398 + },
9399 + {
9400 + "layer": 27,
9401 + "seed": 0,
9402 + "task_acc": 1.0,
9403 + "control_acc": 0.0,
9404 + "selectivity": 1.0
9405 + },
9406 + {
9407 + "layer": 27,
9408 + "seed": 1,
9409 + "task_acc": 1.0,
9410 + "control_acc": 0.75,
9411 + "selectivity": 0.25
9412 + },
9413 + {
9414 + "layer": 27,
9415 + "seed": 2,
9416 + "task_acc": 1.0,
9417 + "control_acc": 0.7916666666666666,
9418 + "selectivity": 0.20833333333333337
9419 + },
9420 + {
9421 + "layer": 27,
9422 + "seed": 3,
9423 + "task_acc": 1.0,
9424 + "control_acc": 0.25,
9425 + "selectivity": 0.75
9426 + },
9427 + {
9428 + "layer": 27,
9429 + "seed": 4,
9430 + "task_acc": 1.0,
9431 + "control_acc": 0.8958333333333334,
9432 + "selectivity": 0.10416666666666663
9433 + }
9434 + ]
9435 + },
9436 + "code_prose_A": {
9437 + "layers": [
9438 + {
9439 + "layer": 0,
9440 + "task_acc_mean": 1.0,
9441 + "task_acc_seed_sd": 0.0,
9442 + "selectivity_mean": 0.575,
9443 + "selectivity_ci": [
9444 + 0.3125,
9445 + 0.8041666666666666
9446 + ],
9447 + "p_boot": 0.0001,
9448 + "fdr_significant": true
9449 + },
9450 + {
9451 + "layer": 1,
9452 + "task_acc_mean": 1.0,
9453 + "task_acc_seed_sd": 0.0,
9454 + "selectivity_mean": 0.7,
9455 + "selectivity_ci": [
9456 + 0.3708333333333333,
9457 + 0.9166666666666666
9458 + ],
9459 + "p_boot": 0.0001,
9460 + "fdr_significant": true
9461 + },
9462 + {
9463 + "layer": 2,
9464 + "task_acc_mean": 1.0,
9465 + "task_acc_seed_sd": 0.0,
9466 + "selectivity_mean": 0.4333333333333334,
9467 + "selectivity_ci": [
9468 + 0.13333333333333333,
9469 + 0.7375
9470 + ],
9471 + "p_boot": 0.0001,
9472 + "fdr_significant": true
9473 + },
9474 + {
9475 + "layer": 3,
9476 + "task_acc_mean": 1.0,
9477 + "task_acc_seed_sd": 0.0,
9478 + "selectivity_mean": 0.4375,
9479 + "selectivity_ci": [
9480 + 0.14583333333333331,
9481 + 0.7625
9482 + ],
9483 + "p_boot": 0.0001,
9484 + "fdr_significant": true
9485 + },
9486 + {
9487 + "layer": 4,
9488 + "task_acc_mean": 1.0,
9489 + "task_acc_seed_sd": 0.0,
9490 + "selectivity_mean": 0.4791666666666667,
9491 + "selectivity_ci": [
9492 + 0.2166666666666667,
9493 + 0.7416666666666667
9494 + ],
9495 + "p_boot": 0.0001,
9496 + "fdr_significant": true
9497 + },
9498 + {
9499 + "layer": 5,
9500 + "task_acc_mean": 1.0,
9501 + "task_acc_seed_sd": 0.0,
9502 + "selectivity_mean": 0.2958333333333333,
9503 + "selectivity_ci": [
9504 + 0.07083333333333333,
9505 + 0.5874999999999999
9506 + ],
9507 + "p_boot": 0.0001,
9508 + "fdr_significant": true
9509 + },
9510 + {
9511 + "layer": 6,
9512 + "task_acc_mean": 1.0,
9513 + "task_acc_seed_sd": 0.0,
9514 + "selectivity_mean": 0.7291666666666667,
9515 + "selectivity_ci": [
9516 + 0.4875,
9517 + 0.8791666666666668
9518 + ],
9519 + "p_boot": 0.0001,
9520 + "fdr_significant": true
9521 + },
9522 + {
9523 + "layer": 7,
9524 + "task_acc_mean": 1.0,
9525 + "task_acc_seed_sd": 0.0,
9526 + "selectivity_mean": 0.6833333333333333,
9527 + "selectivity_ci": [
9528 + 0.3333333333333333,
9529 + 0.9083333333333334
9530 + ],
9531 + "p_boot": 0.0002,
9532 + "fdr_significant": true
9533 + },
9534 + {
9535 + "layer": 8,
9536 + "task_acc_mean": 1.0,
9537 + "task_acc_seed_sd": 0.0,
9538 + "selectivity_mean": 0.5874999999999999,
9539 + "selectivity_ci": [
9540 + 0.2958333333333333,
9541 + 0.875
9542 + ],
9543 + "p_boot": 0.0001,
9544 + "fdr_significant": true
9545 + },
9546 + {
9547 + "layer": 9,
9548 + "task_acc_mean": 1.0,
9549 + "task_acc_seed_sd": 0.0,
9550 + "selectivity_mean": 0.8791666666666667,
9551 + "selectivity_ci": [
9552 + 0.8166666666666667,
9553 + 0.9416666666666667
9554 + ],
9555 + "p_boot": 0.0001,
9556 + "fdr_significant": true
9557 + },
9558 + {
9559 + "layer": 10,
9560 + "task_acc_mean": 1.0,
9561 + "task_acc_seed_sd": 0.0,
9562 + "selectivity_mean": 0.6291666666666667,
9563 + "selectivity_ci": [
9564 + 0.3374999999999999,
9565 + 0.9208333333333332
9566 + ],
9567 + "p_boot": 0.0001,
9568 + "fdr_significant": true
9569 + },
9570 + {
9571 + "layer": 11,
9572 + "task_acc_mean": 1.0,
9573 + "task_acc_seed_sd": 0.0,
9574 + "selectivity_mean": 0.4666666666666667,
9575 + "selectivity_ci": [
9576 + 0.19999999999999998,
9577 + 0.7333333333333333
9578 + ],
9579 + "p_boot": 0.0001,
9580 + "fdr_significant": true
9581 + },
9582 + {
9583 + "layer": 12,
9584 + "task_acc_mean": 1.0,
9585 + "task_acc_seed_sd": 0.0,
9586 + "selectivity_mean": 0.5708333333333333,
9587 + "selectivity_ci": [
9588 + 0.2166666666666667,
9589 + 0.8875
9590 + ],
9591 + "p_boot": 0.0001,
9592 + "fdr_significant": true
9593 + },
9594 + {
9595 + "layer": 13,
9596 + "task_acc_mean": 1.0,
9597 + "task_acc_seed_sd": 0.0,
9598 + "selectivity_mean": 0.5666666666666667,
9599 + "selectivity_ci": [
9600 + 0.2625,
9601 + 0.8708333333333332
9602 + ],
9603 + "p_boot": 0.0001,
9604 + "fdr_significant": true
9605 + },
9606 + {
9607 + "layer": 14,
9608 + "task_acc_mean": 1.0,
9609 + "task_acc_seed_sd": 0.0,
9610 + "selectivity_mean": 0.7166666666666666,
9611 + "selectivity_ci": [
9612 + 0.45833333333333337,
9613 + 0.8791666666666667
9614 + ],
9615 + "p_boot": 0.0001,
9616 + "fdr_significant": true
9617 + },
9618 + {
9619 + "layer": 15,
9620 + "task_acc_mean": 1.0,
9621 + "task_acc_seed_sd": 0.0,
9622 + "selectivity_mean": 0.36250000000000004,
9623 + "selectivity_ci": [
9624 + 0.02083333333333335,
9625 + 0.75
9626 + ],
9627 + "p_boot": 0.0002,
9628 + "fdr_significant": true
9629 + },
9630 + {
9631 + "layer": 16,
9632 + "task_acc_mean": 1.0,
9633 + "task_acc_seed_sd": 0.0,
9634 + "selectivity_mean": 0.6458333333333333,
9635 + "selectivity_ci": [
9636 + 0.3416666666666667,
9637 + 0.9166666666666666
9638 + ],
9639 + "p_boot": 0.0001,
9640 + "fdr_significant": true
9641 + },
9642 + {
9643 + "layer": 17,
9644 + "task_acc_mean": 1.0,
9645 + "task_acc_seed_sd": 0.0,
9646 + "selectivity_mean": 0.4666666666666666,
9647 + "selectivity_ci": [
9648 + 0.15416666666666665,
9649 + 0.7791666666666666
9650 + ],
9651 + "p_boot": 0.0001,
9652 + "fdr_significant": true
9653 + },
9654 + {
9655 + "layer": 18,
9656 + "task_acc_mean": 1.0,
9657 + "task_acc_seed_sd": 0.0,
9658 + "selectivity_mean": 0.7875,
9659 + "selectivity_ci": [
9660 + 0.4916666666666667,
9661 + 0.9583333333333334
9662 + ],
9663 + "p_boot": 0.0001,
9664 + "fdr_significant": true
9665 + },
9666 + {
9667 + "layer": 19,
9668 + "task_acc_mean": 1.0,
9669 + "task_acc_seed_sd": 0.0,
9670 + "selectivity_mean": 0.6,
9671 + "selectivity_ci": [
9672 + 0.2958333333333333,
9673 + 0.9041666666666666
9674 + ],
9675 + "p_boot": 0.0001,
9676 + "fdr_significant": true
9677 + },
9678 + {
9679 + "layer": 20,
9680 + "task_acc_mean": 1.0,
9681 + "task_acc_seed_sd": 0.0,
9682 + "selectivity_mean": 0.4291666666666667,
9683 + "selectivity_ci": [
9684 + 0.1375,
9685 + 0.725
9686 + ],
9687 + "p_boot": 0.0002,
9688 + "fdr_significant": true
9689 + },
9690 + {
9691 + "layer": 21,
9692 + "task_acc_mean": 1.0,
9693 + "task_acc_seed_sd": 0.0,
9694 + "selectivity_mean": 0.6375,
9695 + "selectivity_ci": [
9696 + 0.3208333333333334,
9697 + 0.8708333333333332
9698 + ],
9699 + "p_boot": 0.0001,
9700 + "fdr_significant": true
9701 + },
9702 + {
9703 + "layer": 22,
9704 + "task_acc_mean": 1.0,
9705 + "task_acc_seed_sd": 0.0,
9706 + "selectivity_mean": 0.4125,
9707 + "selectivity_ci": [
9708 + 0.029166666666666674,
9709 + 0.7958333333333333
9710 + ],
9711 + "p_boot": 0.0104,
9712 + "fdr_significant": true
9713 + },
9714 + {
9715 + "layer": 23,
9716 + "task_acc_mean": 1.0,
9717 + "task_acc_seed_sd": 0.0,
9718 + "selectivity_mean": 0.6083333333333334,
9719 + "selectivity_ci": [
9720 + 0.2541666666666667,
9721 + 0.9291666666666668
9722 + ],
9723 + "p_boot": 0.0004,
9724 + "fdr_significant": true
9725 + },
9726 + {
9727 + "layer": 24,
9728 + "task_acc_mean": 1.0,
9729 + "task_acc_seed_sd": 0.0,
9730 + "selectivity_mean": 0.5708333333333333,
9731 + "selectivity_ci": [
9732 + 0.2708333333333333,
9733 + 0.8625
9734 + ],
9735 + "p_boot": 0.0001,
9736 + "fdr_significant": true
9737 + },
9738 + {
9739 + "layer": 25,
9740 + "task_acc_mean": 1.0,
9741 + "task_acc_seed_sd": 0.0,
9742 + "selectivity_mean": 0.8833333333333332,
9743 + "selectivity_ci": [
9744 + 0.7791666666666666,
9745 + 0.9625
9746 + ],
9747 + "p_boot": 0.0001,
9748 + "fdr_significant": true
9749 + },
9750 + {
9751 + "layer": 26,
9752 + "task_acc_mean": 1.0,
9753 + "task_acc_seed_sd": 0.0,
9754 + "selectivity_mean": 0.4708333333333333,
9755 + "selectivity_ci": [
9756 + 0.19166666666666665,
9757 + 0.8
9758 + ],
9759 + "p_boot": 0.0001,
9760 + "fdr_significant": true
9761 + },
9762 + {
9763 + "layer": 27,
9764 + "task_acc_mean": 1.0,
9765 + "task_acc_seed_sd": 0.0,
9766 + "selectivity_mean": 0.7333333333333334,
9767 + "selectivity_ci": [
9768 + 0.425,
9769 + 0.9041666666666666
9770 + ],
9771 + "p_boot": 0.0001,
9772 + "fdr_significant": true
9773 + }
9774 + ],
9775 + "replication_rate_topk": {
9776 + "k": 5,
9777 + "point": 1.0,
9778 + "ci": [
9779 + 1.0,
9780 + 1.0
9781 + ]
9782 + },
9783 + "n_fdr_significant": 28,
9784 + "cells": [
9785 + {
9786 + "layer": 0,
9787 + "seed": 0,
9788 + "task_acc": 1.0,
9789 + "control_acc": 0.3541666666666667,
9790 + "selectivity": 0.6458333333333333
9791 + },
9792 + {
9793 + "layer": 0,
9794 + "seed": 1,
9795 + "task_acc": 1.0,
9796 + "control_acc": 0.10416666666666667,
9797 + "selectivity": 0.8958333333333334
9798 + },
9799 + {
9800 + "layer": 0,
9801 + "seed": 2,
9802 + "task_acc": 1.0,
9803 + "control_acc": 0.5208333333333334,
9804 + "selectivity": 0.47916666666666663
9805 + },
9806 + {
9807 + "layer": 0,
9808 + "seed": 3,
9809 + "task_acc": 1.0,
9810 + "control_acc": 0.20833333333333334,
9811 + "selectivity": 0.7916666666666666
9812 + },
9813 + {
9814 + "layer": 0,
9815 + "seed": 4,
9816 + "task_acc": 1.0,
9817 + "control_acc": 0.9375,
9818 + "selectivity": 0.0625
9819 + },
9820 + {
9821 + "layer": 1,
9822 + "seed": 0,
9823 + "task_acc": 1.0,
9824 + "control_acc": 0.9375,
9825 + "selectivity": 0.0625
9826 + },
9827 + {
9828 + "layer": 1,
9829 + "seed": 1,
9830 + "task_acc": 1.0,
9831 + "control_acc": 0.125,
9832 + "selectivity": 0.875
9833 + },
9834 + {
9835 + "layer": 1,
9836 + "seed": 2,
9837 + "task_acc": 1.0,
9838 + "control_acc": 0.0,
9839 + "selectivity": 1.0
9840 + },
9841 + {
9842 + "layer": 1,
9843 + "seed": 3,
9844 + "task_acc": 1.0,
9845 + "control_acc": 0.22916666666666666,
9846 + "selectivity": 0.7708333333333334
9847 + },
9848 + {
9849 + "layer": 1,
9850 + "seed": 4,
9851 + "task_acc": 1.0,
9852 + "control_acc": 0.20833333333333334,
9853 + "selectivity": 0.7916666666666666
9854 + },
9855 + {
9856 + "layer": 2,
9857 + "seed": 0,
9858 + "task_acc": 1.0,
9859 + "control_acc": 0.9375,
9860 + "selectivity": 0.0625
9861 + },
9862 + {
9863 + "layer": 2,
9864 + "seed": 1,
9865 + "task_acc": 1.0,
9866 + "control_acc": 0.10416666666666667,
9867 + "selectivity": 0.8958333333333334
9868 + },
9869 + {
9870 + "layer": 2,
9871 + "seed": 2,
9872 + "task_acc": 1.0,
9873 + "control_acc": 0.8333333333333334,
9874 + "selectivity": 0.16666666666666663
9875 + },
9876 + {
9877 + "layer": 2,
9878 + "seed": 3,
9879 + "task_acc": 1.0,
9880 + "control_acc": 0.16666666666666666,
9881 + "selectivity": 0.8333333333333334
9882 + },
9883 + {
9884 + "layer": 2,
9885 + "seed": 4,
9886 + "task_acc": 1.0,
9887 + "control_acc": 0.7916666666666666,
9888 + "selectivity": 0.20833333333333337
9889 + },
9890 + {
9891 + "layer": 3,
9892 + "seed": 0,
9893 + "task_acc": 1.0,
9894 + "control_acc": 0.2708333333333333,
9895 + "selectivity": 0.7291666666666667
9896 + },
9897 + {
9898 + "layer": 3,
9899 + "seed": 1,
9900 + "task_acc": 1.0,
9901 + "control_acc": 0.7708333333333334,
9902 + "selectivity": 0.22916666666666663
9903 + },
9904 + {
9905 + "layer": 3,
9906 + "seed": 2,
9907 + "task_acc": 1.0,
9908 + "control_acc": 0.020833333333333332,
9909 + "selectivity": 0.9791666666666666
9910 + },
9911 + {
9912 + "layer": 3,
9913 + "seed": 3,
9914 + "task_acc": 1.0,
9915 + "control_acc": 0.8541666666666666,
9916 + "selectivity": 0.14583333333333337
9917 + },
9918 + {
9919 + "layer": 3,
9920 + "seed": 4,
9921 + "task_acc": 1.0,
9922 + "control_acc": 0.8958333333333334,
9923 + "selectivity": 0.10416666666666663
9924 + },
9925 + {
9926 + "layer": 4,
9927 + "seed": 0,
9928 + "task_acc": 1.0,
9929 + "control_acc": 0.6666666666666666,
9930 + "selectivity": 0.33333333333333337
9931 + },
9932 + {
9933 + "layer": 4,
9934 + "seed": 1,
9935 + "task_acc": 1.0,
9936 + "control_acc": 0.75,
9937 + "selectivity": 0.25
9938 + },
9939 + {
9940 + "layer": 4,
9941 + "seed": 2,
9942 + "task_acc": 1.0,
9943 + "control_acc": 0.125,
9944 + "selectivity": 0.875
9945 + },
9946 + {
9947 + "layer": 4,
9948 + "seed": 3,
9949 + "task_acc": 1.0,
9950 + "control_acc": 0.875,
9951 + "selectivity": 0.125
9952 + },
9953 + {
9954 + "layer": 4,
9955 + "seed": 4,
9956 + "task_acc": 1.0,
9957 + "control_acc": 0.1875,
9958 + "selectivity": 0.8125
9959 + },
9960 + {
9961 + "layer": 5,
9962 + "seed": 0,
9963 + "task_acc": 1.0,
9964 + "control_acc": 0.5416666666666666,
9965 + "selectivity": 0.45833333333333337
9966 + },
9967 + {
9968 + "layer": 5,
9969 + "seed": 1,
9970 + "task_acc": 1.0,
9971 + "control_acc": 0.8958333333333334,
9972 + "selectivity": 0.10416666666666663
9973 + },
9974 + {
9975 + "layer": 5,
9976 + "seed": 2,
9977 + "task_acc": 1.0,
9978 + "control_acc": 0.9791666666666666,
9979 + "selectivity": 0.02083333333333337
9980 + },
9981 + {
9982 + "layer": 5,
9983 + "seed": 3,
9984 + "task_acc": 1.0,
9985 + "control_acc": 0.8958333333333334,
9986 + "selectivity": 0.10416666666666663
9987 + },
9988 + {
9989 + "layer": 5,
9990 + "seed": 4,
9991 + "task_acc": 1.0,
9992 + "control_acc": 0.20833333333333334,
9993 + "selectivity": 0.7916666666666666
9994 + },
9995 + {
9996 + "layer": 6,
9997 + "seed": 0,
9998 + "task_acc": 1.0,
9999 + "control_acc": 0.75,
10000 + "selectivity": 0.25
10001 + },
10002 + {
10003 + "layer": 6,
10004 + "seed": 1,
10005 + "task_acc": 1.0,
10006 + "control_acc": 0.10416666666666667,
10007 + "selectivity": 0.8958333333333334
10008 + },
10009 + {
10010 + "layer": 6,
10011 + "seed": 2,
10012 + "task_acc": 1.0,
10013 + "control_acc": 0.1875,
10014 + "selectivity": 0.8125
10015 + },
10016 + {
10017 + "layer": 6,
10018 + "seed": 3,
10019 + "task_acc": 1.0,
10020 + "control_acc": 0.10416666666666667,
10021 + "selectivity": 0.8958333333333334
10022 + },
10023 + {
10024 + "layer": 6,
10025 + "seed": 4,
10026 + "task_acc": 1.0,
10027 + "control_acc": 0.20833333333333334,
10028 + "selectivity": 0.7916666666666666
10029 + },
10030 + {
10031 + "layer": 7,
10032 + "seed": 0,
10033 + "task_acc": 1.0,
10034 + "control_acc": 1.0,
10035 + "selectivity": 0.0
10036 + },
10037 + {
10038 + "layer": 7,
10039 + "seed": 1,
10040 + "task_acc": 1.0,
10041 + "control_acc": 0.041666666666666664,
10042 + "selectivity": 0.9583333333333334
10043 + },
10044 + {
10045 + "layer": 7,
10046 + "seed": 2,
10047 + "task_acc": 1.0,
10048 + "control_acc": 0.08333333333333333,
10049 + "selectivity": 0.9166666666666666
10050 + },
10051 + {
10052 + "layer": 7,
10053 + "seed": 3,
10054 + "task_acc": 1.0,
10055 + "control_acc": 0.25,
10056 + "selectivity": 0.75
10057 + },
10058 + {
10059 + "layer": 7,
10060 + "seed": 4,
10061 + "task_acc": 1.0,
10062 + "control_acc": 0.20833333333333334,
10063 + "selectivity": 0.7916666666666666
10064 + },
10065 + {
10066 + "layer": 8,
10067 + "seed": 0,
10068 + "task_acc": 1.0,
10069 + "control_acc": 0.8333333333333334,
10070 + "selectivity": 0.16666666666666663
10071 + },
10072 + {
10073 + "layer": 8,
10074 + "seed": 1,
10075 + "task_acc": 1.0,
10076 + "control_acc": 0.8125,
10077 + "selectivity": 0.1875
10078 + },
10079 + {
10080 + "layer": 8,
10081 + "seed": 2,
10082 + "task_acc": 1.0,
10083 + "control_acc": 0.0,
10084 + "selectivity": 1.0
10085 + },
10086 + {
10087 + "layer": 8,
10088 + "seed": 3,
10089 + "task_acc": 1.0,
10090 + "control_acc": 0.20833333333333334,
10091 + "selectivity": 0.7916666666666666
10092 + },
10093 + {
10094 + "layer": 8,
10095 + "seed": 4,
10096 + "task_acc": 1.0,
10097 + "control_acc": 0.20833333333333334,
10098 + "selectivity": 0.7916666666666666
10099 + },
10100 + {
10101 + "layer": 9,
10102 + "seed": 0,
10103 + "task_acc": 1.0,
10104 + "control_acc": 0.125,
10105 + "selectivity": 0.875
10106 + },
10107 + {
10108 + "layer": 9,
10109 + "seed": 1,
10110 + "task_acc": 1.0,
10111 + "control_acc": 0.0625,
10112 + "selectivity": 0.9375
10113 + },
10114 + {
10115 + "layer": 9,
10116 + "seed": 2,
10117 + "task_acc": 1.0,
10118 + "control_acc": 0.020833333333333332,
10119 + "selectivity": 0.9791666666666666
10120 + },
10121 + {
10122 + "layer": 9,
10123 + "seed": 3,
10124 + "task_acc": 1.0,
10125 + "control_acc": 0.1875,
10126 + "selectivity": 0.8125
10127 + },
10128 + {
10129 + "layer": 9,
10130 + "seed": 4,
10131 + "task_acc": 1.0,
10132 + "control_acc": 0.20833333333333334,
10133 + "selectivity": 0.7916666666666666
10134 + },
10135 + {
10136 + "layer": 10,
10137 + "seed": 0,
10138 + "task_acc": 1.0,
10139 + "control_acc": 0.7083333333333334,
10140 + "selectivity": 0.29166666666666663
10141 + },
10142 + {
10143 + "layer": 10,
10144 + "seed": 1,
10145 + "task_acc": 1.0,
10146 + "control_acc": 0.8333333333333334,
10147 + "selectivity": 0.16666666666666663
10148 + },
10149 + {
10150 + "layer": 10,
10151 + "seed": 2,
10152 + "task_acc": 1.0,
10153 + "control_acc": 0.0,
10154 + "selectivity": 1.0
10155 + },
10156 + {
10157 + "layer": 10,
10158 + "seed": 3,
10159 + "task_acc": 1.0,
10160 + "control_acc": 0.08333333333333333,
10161 + "selectivity": 0.9166666666666666
10162 + },
10163 + {
10164 + "layer": 10,
10165 + "seed": 4,
10166 + "task_acc": 1.0,
10167 + "control_acc": 0.22916666666666666,
10168 + "selectivity": 0.7708333333333334
10169 + },
10170 + {
10171 + "layer": 11,
10172 + "seed": 0,
10173 + "task_acc": 1.0,
10174 + "control_acc": 0.7083333333333334,
10175 + "selectivity": 0.29166666666666663
10176 + },
10177 + {
10178 + "layer": 11,
10179 + "seed": 1,
10180 + "task_acc": 1.0,
10181 + "control_acc": 0.8125,
10182 + "selectivity": 0.1875
10183 + },
10184 + {
10185 + "layer": 11,
10186 + "seed": 2,
10187 + "task_acc": 1.0,
10188 + "control_acc": 0.14583333333333334,
10189 + "selectivity": 0.8541666666666666
10190 + },
10191 + {
10192 + "layer": 11,
10193 + "seed": 3,
10194 + "task_acc": 1.0,
10195 + "control_acc": 0.8333333333333334,
10196 + "selectivity": 0.16666666666666663
10197 + },
10198 + {
10199 + "layer": 11,
10200 + "seed": 4,
10201 + "task_acc": 1.0,
10202 + "control_acc": 0.16666666666666666,
10203 + "selectivity": 0.8333333333333334
10204 + },
10205 + {
10206 + "layer": 12,
10207 + "seed": 0,
10208 + "task_acc": 1.0,
10209 + "control_acc": 1.0,
10210 + "selectivity": 0.0
10211 + },
10212 + {
10213 + "layer": 12,
10214 + "seed": 1,
10215 + "task_acc": 1.0,
10216 + "control_acc": 0.14583333333333334,
10217 + "selectivity": 0.8541666666666666
10218 + },
10219 + {
10220 + "layer": 12,
10221 + "seed": 2,
10222 + "task_acc": 1.0,
10223 + "control_acc": 0.08333333333333333,
10224 + "selectivity": 0.9166666666666666
10225 + },
10226 + {
10227 + "layer": 12,
10228 + "seed": 3,
10229 + "task_acc": 1.0,
10230 + "control_acc": 0.7916666666666666,
10231 + "selectivity": 0.20833333333333337
10232 + },
10233 + {
10234 + "layer": 12,
10235 + "seed": 4,
10236 + "task_acc": 1.0,
10237 + "control_acc": 0.125,
10238 + "selectivity": 0.875
10239 + },
10240 + {
10241 + "layer": 13,
10242 + "seed": 0,
10243 + "task_acc": 1.0,
10244 + "control_acc": 0.8541666666666666,
10245 + "selectivity": 0.14583333333333337
10246 + },
10247 + {
10248 + "layer": 13,
10249 + "seed": 1,
10250 + "task_acc": 1.0,
10251 + "control_acc": 0.875,
10252 + "selectivity": 0.125
10253 + },
10254 + {
10255 + "layer": 13,
10256 + "seed": 2,
10257 + "task_acc": 1.0,
10258 + "control_acc": 0.0625,
10259 + "selectivity": 0.9375
10260 + },
10261 + {
10262 + "layer": 13,
10263 + "seed": 3,
10264 + "task_acc": 1.0,
10265 + "control_acc": 0.14583333333333334,
10266 + "selectivity": 0.8541666666666666
10267 + },
10268 + {
10269 + "layer": 13,
10270 + "seed": 4,
10271 + "task_acc": 1.0,
10272 + "control_acc": 0.22916666666666666,
10273 + "selectivity": 0.7708333333333334
10274 + },
10275 + {
10276 + "layer": 14,
10277 + "seed": 0,
10278 + "task_acc": 1.0,
10279 + "control_acc": 0.14583333333333334,
10280 + "selectivity": 0.8541666666666666
10281 + },
10282 + {
10283 + "layer": 14,
10284 + "seed": 1,
10285 + "task_acc": 1.0,
10286 + "control_acc": 0.7916666666666666,
10287 + "selectivity": 0.20833333333333337
10288 + },
10289 + {
10290 + "layer": 14,
10291 + "seed": 2,
10292 + "task_acc": 1.0,
10293 + "control_acc": 0.08333333333333333,
10294 + "selectivity": 0.9166666666666666
10295 + },
10296 + {
10297 + "layer": 14,
10298 + "seed": 3,
10299 + "task_acc": 1.0,
10300 + "control_acc": 0.14583333333333334,
10301 + "selectivity": 0.8541666666666666
10302 + },
10303 + {
10304 + "layer": 14,
10305 + "seed": 4,
10306 + "task_acc": 1.0,
10307 + "control_acc": 0.25,
10308 + "selectivity": 0.75
10309 + },
10310 + {
10311 + "layer": 15,
10312 + "seed": 0,
10313 + "task_acc": 1.0,
10314 + "control_acc": 1.0,
10315 + "selectivity": 0.0
10316 + },
10317 + {
10318 + "layer": 15,
10319 + "seed": 1,
10320 + "task_acc": 1.0,
10321 + "control_acc": 0.9791666666666666,
10322 + "selectivity": 0.02083333333333337
10323 + },
10324 + {
10325 + "layer": 15,
10326 + "seed": 2,
10327 + "task_acc": 1.0,
10328 + "control_acc": 0.0,
10329 + "selectivity": 1.0
10330 + },
10331 + {
10332 + "layer": 15,
10333 + "seed": 3,
10334 + "task_acc": 1.0,
10335 + "control_acc": 0.9375,
10336 + "selectivity": 0.0625
10337 + },
10338 + {
10339 + "layer": 15,
10340 + "seed": 4,
10341 + "task_acc": 1.0,
10342 + "control_acc": 0.2708333333333333,
10343 + "selectivity": 0.7291666666666667
10344 + },
10345 + {
10346 + "layer": 16,
10347 + "seed": 0,
10348 + "task_acc": 1.0,
10349 + "control_acc": 0.5208333333333334,
10350 + "selectivity": 0.47916666666666663
10351 + },
10352 + {
10353 + "layer": 16,
10354 + "seed": 1,
10355 + "task_acc": 1.0,
10356 + "control_acc": 0.020833333333333332,
10357 + "selectivity": 0.9791666666666666
10358 + },
10359 + {
10360 + "layer": 16,
10361 + "seed": 2,
10362 + "task_acc": 1.0,
10363 + "control_acc": 0.0625,
10364 + "selectivity": 0.9375
10365 + },
10366 + {
10367 + "layer": 16,
10368 + "seed": 3,
10369 + "task_acc": 1.0,
10370 + "control_acc": 0.9166666666666666,
10371 + "selectivity": 0.08333333333333337
10372 + },
10373 + {
10374 + "layer": 16,
10375 + "seed": 4,
10376 + "task_acc": 1.0,
10377 + "control_acc": 0.25,
10378 + "selectivity": 0.75
10379 + },
10380 + {
10381 + "layer": 17,
10382 + "seed": 0,
10383 + "task_acc": 1.0,
10384 + "control_acc": 0.4791666666666667,
10385 + "selectivity": 0.5208333333333333
10386 + },
10387 + {
10388 + "layer": 17,
10389 + "seed": 1,
10390 + "task_acc": 1.0,
10391 + "control_acc": 0.0625,
10392 + "selectivity": 0.9375
10393 + },
10394 + {
10395 + "layer": 17,
10396 + "seed": 2,
10397 + "task_acc": 1.0,
10398 + "control_acc": 0.9791666666666666,
10399 + "selectivity": 0.02083333333333337
10400 + },
10401 + {
10402 + "layer": 17,
10403 + "seed": 3,
10404 + "task_acc": 1.0,
10405 + "control_acc": 0.8958333333333334,
10406 + "selectivity": 0.10416666666666663
10407 + },
10408 + {
10409 + "layer": 17,
10410 + "seed": 4,
10411 + "task_acc": 1.0,
10412 + "control_acc": 0.25,
10413 + "selectivity": 0.75
10414 + },
10415 + {
10416 + "layer": 18,
10417 + "seed": 0,
10418 + "task_acc": 1.0,
10419 + "control_acc": 0.0625,
10420 + "selectivity": 0.9375
10421 + },
10422 + {
10423 + "layer": 18,
10424 + "seed": 1,
10425 + "task_acc": 1.0,
10426 + "control_acc": 0.0,
10427 + "selectivity": 1.0
10428 + },
10429 + {
10430 + "layer": 18,
10431 + "seed": 2,
10432 + "task_acc": 1.0,
10433 + "control_acc": 0.08333333333333333,
10434 + "selectivity": 0.9166666666666666
10435 + },
10436 + {
10437 + "layer": 18,
10438 + "seed": 3,
10439 + "task_acc": 1.0,
10440 + "control_acc": 0.7916666666666666,
10441 + "selectivity": 0.20833333333333337
10442 + },
10443 + {
10444 + "layer": 18,
10445 + "seed": 4,
10446 + "task_acc": 1.0,
10447 + "control_acc": 0.125,
10448 + "selectivity": 0.875
10449 + },
10450 + {
10451 + "layer": 19,
10452 + "seed": 0,
10453 + "task_acc": 1.0,
10454 + "control_acc": 0.041666666666666664,
10455 + "selectivity": 0.9583333333333334
10456 + },
10457 + {
10458 + "layer": 19,
10459 + "seed": 1,
10460 + "task_acc": 1.0,
10461 + "control_acc": 0.75,
10462 + "selectivity": 0.25
10463 + },
10464 + {
10465 + "layer": 19,
10466 + "seed": 2,
10467 + "task_acc": 1.0,
10468 + "control_acc": 0.08333333333333333,
10469 + "selectivity": 0.9166666666666666
10470 + },
10471 + {
10472 + "layer": 19,
10473 + "seed": 3,
10474 + "task_acc": 1.0,
10475 + "control_acc": 0.8958333333333334,
10476 + "selectivity": 0.10416666666666663
10477 + },
10478 + {
10479 + "layer": 19,
10480 + "seed": 4,
10481 + "task_acc": 1.0,
10482 + "control_acc": 0.22916666666666666,
10483 + "selectivity": 0.7708333333333334
10484 + },
10485 + {
10486 + "layer": 20,
10487 + "seed": 0,
10488 + "task_acc": 1.0,
10489 + "control_acc": 1.0,
10490 + "selectivity": 0.0
10491 + },
10492 + {
10493 + "layer": 20,
10494 + "seed": 1,
10495 + "task_acc": 1.0,
10496 + "control_acc": 0.7916666666666666,
10497 + "selectivity": 0.20833333333333337
10498 + },
10499 + {
10500 + "layer": 20,
10501 + "seed": 2,
10502 + "task_acc": 1.0,
10503 + "control_acc": 0.20833333333333334,
10504 + "selectivity": 0.7916666666666666
10505 + },
10506 + {
10507 + "layer": 20,
10508 + "seed": 3,
10509 + "task_acc": 1.0,
10510 + "control_acc": 0.7291666666666666,
10511 + "selectivity": 0.27083333333333337
10512 + },
10513 + {
10514 + "layer": 20,
10515 + "seed": 4,
10516 + "task_acc": 1.0,
10517 + "control_acc": 0.125,
10518 + "selectivity": 0.875
10519 + },
10520 + {
10521 + "layer": 21,
10522 + "seed": 0,
10523 + "task_acc": 1.0,
10524 + "control_acc": 0.9791666666666666,
10525 + "selectivity": 0.02083333333333337
10526 + },
10527 + {
10528 + "layer": 21,
10529 + "seed": 1,
10530 + "task_acc": 1.0,
10531 + "control_acc": 0.0625,
10532 + "selectivity": 0.9375
10533 + },
10534 + {
10535 + "layer": 21,
10536 + "seed": 2,
10537 + "task_acc": 1.0,
10538 + "control_acc": 0.22916666666666666,
10539 + "selectivity": 0.7708333333333334
10540 + },
10541 + {
10542 + "layer": 21,
10543 + "seed": 3,
10544 + "task_acc": 1.0,
10545 + "control_acc": 0.14583333333333334,
10546 + "selectivity": 0.8541666666666666
10547 + },
10548 + {
10549 + "layer": 21,
10550 + "seed": 4,
10551 + "task_acc": 1.0,
10552 + "control_acc": 0.3958333333333333,
10553 + "selectivity": 0.6041666666666667
10554 + },
10555 + {
10556 + "layer": 22,
10557 + "seed": 0,
10558 + "task_acc": 1.0,
10559 + "control_acc": 1.0,
10560 + "selectivity": 0.0
10561 + },
10562 + {
10563 + "layer": 22,
10564 + "seed": 1,
10565 + "task_acc": 1.0,
10566 + "control_acc": 0.8541666666666666,
10567 + "selectivity": 0.14583333333333337
10568 + },
10569 + {
10570 + "layer": 22,
10571 + "seed": 2,
10572 + "task_acc": 1.0,
10573 + "control_acc": 0.0,
10574 + "selectivity": 1.0
10575 + },
10576 + {
10577 + "layer": 22,
10578 + "seed": 3,
10579 + "task_acc": 1.0,
10580 + "control_acc": 1.0,
10581 + "selectivity": 0.0
10582 + },
10583 + {
10584 + "layer": 22,
10585 + "seed": 4,
10586 + "task_acc": 1.0,
10587 + "control_acc": 0.08333333333333333,
10588 + "selectivity": 0.9166666666666666
10589 + },
10590 + {
10591 + "layer": 23,
10592 + "seed": 0,
10593 + "task_acc": 1.0,
10594 + "control_acc": 0.041666666666666664,
10595 + "selectivity": 0.9583333333333334
10596 + },
10597 + {
10598 + "layer": 23,
10599 + "seed": 1,
10600 + "task_acc": 1.0,
10601 + "control_acc": 0.0,
10602 + "selectivity": 1.0
10603 + },
10604 + {
10605 + "layer": 23,
10606 + "seed": 2,
10607 + "task_acc": 1.0,
10608 + "control_acc": 1.0,
10609 + "selectivity": 0.0
10610 + },
10611 + {
10612 + "layer": 23,
10613 + "seed": 3,
10614 + "task_acc": 1.0,
10615 + "control_acc": 0.6875,
10616 + "selectivity": 0.3125
10617 + },
10618 + {
10619 + "layer": 23,
10620 + "seed": 4,
10621 + "task_acc": 1.0,
10622 + "control_acc": 0.22916666666666666,
10623 + "selectivity": 0.7708333333333334
10624 + },
10625 + {
10626 + "layer": 24,
10627 + "seed": 0,
10628 + "task_acc": 1.0,
10629 + "control_acc": 0.8125,
10630 + "selectivity": 0.1875
10631 + },
10632 + {
10633 + "layer": 24,
10634 + "seed": 1,
10635 + "task_acc": 1.0,
10636 + "control_acc": 0.8958333333333334,
10637 + "selectivity": 0.10416666666666663
10638 + },
10639 + {
10640 + "layer": 24,
10641 + "seed": 2,
10642 + "task_acc": 1.0,
10643 + "control_acc": 0.10416666666666667,
10644 + "selectivity": 0.8958333333333334
10645 + },
10646 + {
10647 + "layer": 24,
10648 + "seed": 3,
10649 + "task_acc": 1.0,
10650 + "control_acc": 0.14583333333333334,
10651 + "selectivity": 0.8541666666666666
10652 + },
10653 + {
10654 + "layer": 24,
10655 + "seed": 4,
10656 + "task_acc": 1.0,
10657 + "control_acc": 0.1875,
10658 + "selectivity": 0.8125
10659 + },
10660 + {
10661 + "layer": 25,
10662 + "seed": 0,
10663 + "task_acc": 1.0,
10664 + "control_acc": 0.3125,
10665 + "selectivity": 0.6875
10666 + },
10667 + {
10668 + "layer": 25,
10669 + "seed": 1,
10670 + "task_acc": 1.0,
10671 + "control_acc": 0.0,
10672 + "selectivity": 1.0
10673 + },
10674 + {
10675 + "layer": 25,
10676 + "seed": 2,
10677 + "task_acc": 1.0,
10678 + "control_acc": 0.08333333333333333,
10679 + "selectivity": 0.9166666666666666
10680 + },
10681 + {
10682 + "layer": 25,
10683 + "seed": 3,
10684 + "task_acc": 1.0,
10685 + "control_acc": 0.0625,
10686 + "selectivity": 0.9375
10687 + },
10688 + {
10689 + "layer": 25,
10690 + "seed": 4,
10691 + "task_acc": 1.0,
10692 + "control_acc": 0.125,
10693 + "selectivity": 0.875
10694 + },
10695 + {
10696 + "layer": 26,
10697 + "seed": 0,
10698 + "task_acc": 1.0,
10699 + "control_acc": 0.75,
10700 + "selectivity": 0.25
10701 + },
10702 + {
10703 + "layer": 26,
10704 + "seed": 1,
10705 + "task_acc": 1.0,
10706 + "control_acc": 0.75,
10707 + "selectivity": 0.25
10708 + },
10709 + {
10710 + "layer": 26,
10711 + "seed": 2,
10712 + "task_acc": 1.0,
10713 + "control_acc": 0.0,
10714 + "selectivity": 1.0
10715 + },
10716 + {
10717 + "layer": 26,
10718 + "seed": 3,
10719 + "task_acc": 1.0,
10720 + "control_acc": 0.8958333333333334,
10721 + "selectivity": 0.10416666666666663
10722 + },
10723 + {
10724 + "layer": 26,
10725 + "seed": 4,
10726 + "task_acc": 1.0,
10727 + "control_acc": 0.25,
10728 + "selectivity": 0.75
10729 + },
10730 + {
10731 + "layer": 27,
10732 + "seed": 0,
10733 + "task_acc": 1.0,
10734 + "control_acc": 0.10416666666666667,
10735 + "selectivity": 0.8958333333333334
10736 + },
10737 + {
10738 + "layer": 27,
10739 + "seed": 1,
10740 + "task_acc": 1.0,
10741 + "control_acc": 0.875,
10742 + "selectivity": 0.125
10743 + },
10744 + {
10745 + "layer": 27,
10746 + "seed": 2,
10747 + "task_acc": 1.0,
10748 + "control_acc": 0.10416666666666667,
10749 + "selectivity": 0.8958333333333334
10750 + },
10751 + {
10752 + "layer": 27,
10753 + "seed": 3,
10754 + "task_acc": 1.0,
10755 + "control_acc": 0.08333333333333333,
10756 + "selectivity": 0.9166666666666666
10757 + },
10758 + {
10759 + "layer": 27,
10760 + "seed": 4,
10761 + "task_acc": 1.0,
10762 + "control_acc": 0.16666666666666666,
10763 + "selectivity": 0.8333333333333334
10764 + }
10765 + ]
10766 + },
10767 + "arith_A": {
10768 + "layers": [
10769 + {
10770 + "layer": 0,
10771 + "task_acc_mean": 1.0,
10772 + "task_acc_seed_sd": 0.0,
10773 + "selectivity_mean": 0.3125,
10774 + "selectivity_ci": [
10775 + 0.12083333333333338,
10776 + 0.5791666666666667
10777 + ],
10778 + "p_boot": 0.0001,
10779 + "fdr_significant": true
10780 + },
10781 + {
10782 + "layer": 1,
10783 + "task_acc_mean": 1.0,
10784 + "task_acc_seed_sd": 0.0,
10785 + "selectivity_mean": 0.5583333333333333,
10786 + "selectivity_ci": [
10787 + 0.25,
10788 + 0.8625
10789 + ],
10790 + "p_boot": 0.0001,
10791 + "fdr_significant": true
10792 + },
10793 + {
10794 + "layer": 2,
10795 + "task_acc_mean": 1.0,
10796 + "task_acc_seed_sd": 0.0,
10797 + "selectivity_mean": 0.4875,
10798 + "selectivity_ci": [
10799 + 0.25416666666666665,
10800 + 0.7208333333333334
10801 + ],
10802 + "p_boot": 0.0001,
10803 + "fdr_significant": true
10804 + },
10805 + {
10806 + "layer": 3,
10807 + "task_acc_mean": 1.0,
10808 + "task_acc_seed_sd": 0.0,
10809 + "selectivity_mean": 0.5375,
10810 + "selectivity_ci": [
10811 + 0.225,
10812 + 0.85
10813 + ],
10814 + "p_boot": 0.0001,
10815 + "fdr_significant": true
10816 + },
10817 + {
10818 + "layer": 4,
10819 + "task_acc_mean": 1.0,
10820 + "task_acc_seed_sd": 0.0,
10821 + "selectivity_mean": 0.3291666666666667,
10822 + "selectivity_ci": [
10823 + 0.07083333333333333,
10824 + 0.6041666666666667
10825 + ],
10826 + "p_boot": 0.0001,
10827 + "fdr_significant": true
10828 + },
10829 + {
10830 + "layer": 5,
10831 + "task_acc_mean": 1.0,
10832 + "task_acc_seed_sd": 0.0,
10833 + "selectivity_mean": 0.5125,
10834 + "selectivity_ci": [
10835 + 0.225,
10836 + 0.8
10837 + ],
10838 + "p_boot": 0.0001,
10839 + "fdr_significant": true
10840 + },
10841 + {
10842 + "layer": 6,
10843 + "task_acc_mean": 1.0,
10844 + "task_acc_seed_sd": 0.0,
10845 + "selectivity_mean": 0.3458333333333333,
10846 + "selectivity_ci": [
10847 + 0.07083333333333335,
10848 + 0.6208333333333332
10849 + ],
10850 + "p_boot": 0.0001,
10851 + "fdr_significant": true
10852 + },
10853 + {
10854 + "layer": 7,
10855 + "task_acc_mean": 1.0,
10856 + "task_acc_seed_sd": 0.0,
10857 + "selectivity_mean": 0.3791666666666667,
10858 + "selectivity_ci": [
10859 + 0.06666666666666668,
10860 + 0.7000000000000001
10861 + ],
10862 + "p_boot": 0.0001,
10863 + "fdr_significant": true
10864 + },
10865 + {
10866 + "layer": 8,
10867 + "task_acc_mean": 1.0,
10868 + "task_acc_seed_sd": 0.0,
10869 + "selectivity_mean": 0.3166666666666667,
10870 + "selectivity_ci": [
10871 + 0.05833333333333337,
10872 + 0.5791666666666666
10873 + ],
10874 + "p_boot": 0.0001,
10875 + "fdr_significant": true
10876 + },
10877 + {
10878 + "layer": 9,
10879 + "task_acc_mean": 1.0,
10880 + "task_acc_seed_sd": 0.0,
10881 + "selectivity_mean": 0.3875,
10882 + "selectivity_ci": [
10883 + 0.09166666666666665,
10884 + 0.7249999999999999
10885 + ],
10886 + "p_boot": 0.0001,
10887 + "fdr_significant": true
10888 + },
10889 + {
10890 + "layer": 10,
10891 + "task_acc_mean": 1.0,
10892 + "task_acc_seed_sd": 0.0,
10893 + "selectivity_mean": 0.3291666666666667,
10894 + "selectivity_ci": [
10895 + 0.04166666666666667,
10896 + 0.6166666666666667
10897 + ],
10898 + "p_boot": 0.0004,
10899 + "fdr_significant": true
10900 + },
10901 + {
10902 + "layer": 11,
10903 + "task_acc_mean": 1.0,
10904 + "task_acc_seed_sd": 0.0,
10905 + "selectivity_mean": 0.5041666666666667,
10906 + "selectivity_ci": [
10907 + 0.2,
10908 + 0.8083333333333333
10909 + ],
10910 + "p_boot": 0.0001,
10911 + "fdr_significant": true
10912 + },
10913 + {
10914 + "layer": 12,
10915 + "task_acc_mean": 1.0,
10916 + "task_acc_seed_sd": 0.0,
10917 + "selectivity_mean": 0.19166666666666665,
10918 + "selectivity_ci": [
10919 + 0.04583333333333335,
10920 + 0.45
10921 + ],
10922 + "p_boot": 0.0001,
10923 + "fdr_significant": true
10924 + },
10925 + {
10926 + "layer": 13,
10927 + "task_acc_mean": 1.0,
10928 + "task_acc_seed_sd": 0.0,
10929 + "selectivity_mean": 0.18750000000000003,
10930 + "selectivity_ci": [
10931 + 0.029166666666666674,
10932 + 0.4625
10933 + ],
10934 + "p_boot": 0.0005,
10935 + "fdr_significant": true
10936 + },
10937 + {
10938 + "layer": 14,
10939 + "task_acc_mean": 1.0,
10940 + "task_acc_seed_sd": 0.0,
10941 + "selectivity_mean": 0.22916666666666666,
10942 + "selectivity_ci": [
10943 + 0.03749999999999998,
10944 + 0.5083333333333333
10945 + ],
10946 + "p_boot": 0.0006,
10947 + "fdr_significant": true
10948 + },
10949 + {
10950 + "layer": 15,
10951 + "task_acc_mean": 1.0,
10952 + "task_acc_seed_sd": 0.0,
10953 + "selectivity_mean": 0.2833333333333334,
10954 + "selectivity_ci": [
10955 + 0.02083333333333335,
10956 + 0.5625000000000001
10957 + ],
10958 + "p_boot": 0.0003,
10959 + "fdr_significant": true
10960 + },
10961 + {
10962 + "layer": 16,
10963 + "task_acc_mean": 1.0,
10964 + "task_acc_seed_sd": 0.0,
10965 + "selectivity_mean": 0.4833333333333334,
10966 + "selectivity_ci": [
10967 + 0.1875,
10968 + 0.7791666666666668
10969 + ],
10970 + "p_boot": 0.0001,
10971 + "fdr_significant": true
10972 + },
10973 + {
10974 + "layer": 17,
10975 + "task_acc_mean": 1.0,
10976 + "task_acc_seed_sd": 0.0,
10977 + "selectivity_mean": 0.5,
10978 + "selectivity_ci": [
10979 + 0.1875,
10980 + 0.8125
10981 + ],
10982 + "p_boot": 0.0001,
10983 + "fdr_significant": true
10984 + },
10985 + {
10986 + "layer": 18,
10987 + "task_acc_mean": 1.0,
10988 + "task_acc_seed_sd": 0.0,
10989 + "selectivity_mean": 0.5083333333333334,
10990 + "selectivity_ci": [
10991 + 0.19583333333333336,
10992 + 0.8208333333333334
10993 + ],
10994 + "p_boot": 0.0001,
10995 + "fdr_significant": true
10996 + },
10997 + {
10998 + "layer": 19,
10999 + "task_acc_mean": 1.0,
11000 + "task_acc_seed_sd": 0.0,
11001 + "selectivity_mean": 0.3666666666666667,
11002 + "selectivity_ci": [
11003 + 0.0458333333333333,
11004 + 0.7291666666666667
11005 + ],
11006 + "p_boot": 0.0001,
11007 + "fdr_significant": true
11008 + },
11009 + {
11010 + "layer": 20,
11011 + "task_acc_mean": 1.0,
11012 + "task_acc_seed_sd": 0.0,
11013 + "selectivity_mean": 0.5333333333333334,
11014 + "selectivity_ci": [
11015 + 0.1833333333333333,
11016 + 0.8833333333333332
11017 + ],
11018 + "p_boot": 0.0001,
11019 + "fdr_significant": true
11020 + },
11021 + {
11022 + "layer": 21,
11023 + "task_acc_mean": 1.0,
11024 + "task_acc_seed_sd": 0.0,
11025 + "selectivity_mean": 0.5208333333333334,
11026 + "selectivity_ci": [
11027 + 0.2041666666666667,
11028 + 0.8375
11029 + ],
11030 + "p_boot": 0.0001,
11031 + "fdr_significant": true
11032 + },
11033 + {
11034 + "layer": 22,
11035 + "task_acc_mean": 1.0,
11036 + "task_acc_seed_sd": 0.0,
11037 + "selectivity_mean": 0.525,
11038 + "selectivity_ci": [
11039 + 0.2041666666666667,
11040 + 0.8458333333333334
11041 + ],
11042 + "p_boot": 0.0001,
11043 + "fdr_significant": true
11044 + },
11045 + {
11046 + "layer": 23,
11047 + "task_acc_mean": 1.0,
11048 + "task_acc_seed_sd": 0.0,
11049 + "selectivity_mean": 0.3375,
11050 + "selectivity_ci": [
11051 + 0.04166666666666665,
11052 + 0.6333333333333334
11053 + ],
11054 + "p_boot": 0.0003,
11055 + "fdr_significant": true
11056 + },
11057 + {
11058 + "layer": 24,
11059 + "task_acc_mean": 1.0,
11060 + "task_acc_seed_sd": 0.0,
11061 + "selectivity_mean": 0.09583333333333333,
11062 + "selectivity_ci": [
11063 + 0.06249999999999998,
11064 + 0.13333333333333333
11065 + ],
11066 + "p_boot": 0.0001,
11067 + "fdr_significant": true
11068 + },
11069 + {
11070 + "layer": 25,
11071 + "task_acc_mean": 1.0,
11072 + "task_acc_seed_sd": 0.0,
11073 + "selectivity_mean": 0.25,
11074 + "selectivity_ci": [
11075 + 0.10000000000000002,
11076 + 0.4666666666666668
11077 + ],
11078 + "p_boot": 0.0001,
11079 + "fdr_significant": true
11080 + },
11081 + {
11082 + "layer": 26,
11083 + "task_acc_mean": 1.0,
11084 + "task_acc_seed_sd": 0.0,
11085 + "selectivity_mean": 0.35,
11086 + "selectivity_ci": [
11087 + 0.02083333333333335,
11088 + 0.7208333333333334
11089 + ],
11090 + "p_boot": 0.0005,
11091 + "fdr_significant": true
11092 + },
11093 + {
11094 + "layer": 27,
11095 + "task_acc_mean": 1.0,
11096 + "task_acc_seed_sd": 0.0,
11097 + "selectivity_mean": 0.2583333333333333,
11098 + "selectivity_ci": [
11099 + 0.06249999999999998,
11100 + 0.5833333333333334
11101 + ],
11102 + "p_boot": 0.0001,
11103 + "fdr_significant": true
11104 + }
11105 + ],
11106 + "replication_rate_topk": {
11107 + "k": 5,
11108 + "point": 1.0,
11109 + "ci": [
11110 + 1.0,
11111 + 1.0
11112 + ]
11113 + },
11114 + "n_fdr_significant": 28,
11115 + "cells": [
11116 + {
11117 + "layer": 0,
11118 + "seed": 0,
11119 + "task_acc": 1.0,
11120 + "control_acc": 0.8541666666666666,
11121 + "selectivity": 0.14583333333333337
11122 + },
11123 + {
11124 + "layer": 0,
11125 + "seed": 1,
11126 + "task_acc": 1.0,
11127 + "control_acc": 0.8541666666666666,
11128 + "selectivity": 0.14583333333333337
11129 + },
11130 + {
11131 + "layer": 0,
11132 + "seed": 2,
11133 + "task_acc": 1.0,
11134 + "control_acc": 0.9166666666666666,
11135 + "selectivity": 0.08333333333333337
11136 + },
11137 + {
11138 + "layer": 0,
11139 + "seed": 3,
11140 + "task_acc": 1.0,
11141 + "control_acc": 0.625,
11142 + "selectivity": 0.375
11143 + },
11144 + {
11145 + "layer": 0,
11146 + "seed": 4,
11147 + "task_acc": 1.0,
11148 + "control_acc": 0.1875,
11149 + "selectivity": 0.8125
11150 + },
11151 + {
11152 + "layer": 1,
11153 + "seed": 0,
11154 + "task_acc": 1.0,
11155 + "control_acc": 0.8541666666666666,
11156 + "selectivity": 0.14583333333333337
11157 + },
11158 + {
11159 + "layer": 1,
11160 + "seed": 1,
11161 + "task_acc": 1.0,
11162 + "control_acc": 0.8958333333333334,
11163 + "selectivity": 0.10416666666666663
11164 + },
11165 + {
11166 + "layer": 1,
11167 + "seed": 2,
11168 + "task_acc": 1.0,
11169 + "control_acc": 0.22916666666666666,
11170 + "selectivity": 0.7708333333333334
11171 + },
11172 + {
11173 + "layer": 1,
11174 + "seed": 3,
11175 + "task_acc": 1.0,
11176 + "control_acc": 0.020833333333333332,
11177 + "selectivity": 0.9791666666666666
11178 + },
11179 + {
11180 + "layer": 1,
11181 + "seed": 4,
11182 + "task_acc": 1.0,
11183 + "control_acc": 0.20833333333333334,
11184 + "selectivity": 0.7916666666666666
11185 + },
11186 + {
11187 + "layer": 2,
11188 + "seed": 0,
11189 + "task_acc": 1.0,
11190 + "control_acc": 0.7291666666666666,
11191 + "selectivity": 0.27083333333333337
11192 + },
11193 + {
11194 + "layer": 2,
11195 + "seed": 1,
11196 + "task_acc": 1.0,
11197 + "control_acc": 0.8958333333333334,
11198 + "selectivity": 0.10416666666666663
11199 + },
11200 + {
11201 + "layer": 2,
11202 + "seed": 2,
11203 + "task_acc": 1.0,
11204 + "control_acc": 0.4791666666666667,
11205 + "selectivity": 0.5208333333333333
11206 + },
11207 + {
11208 + "layer": 2,
11209 + "seed": 3,
11210 + "task_acc": 1.0,
11211 + "control_acc": 0.2916666666666667,
11212 + "selectivity": 0.7083333333333333
11213 + },
11214 + {
11215 + "layer": 2,
11216 + "seed": 4,
11217 + "task_acc": 1.0,
11218 + "control_acc": 0.16666666666666666,
11219 + "selectivity": 0.8333333333333334
11220 + },
11221 + {
11222 + "layer": 3,
11223 + "seed": 0,
11224 + "task_acc": 1.0,
11225 + "control_acc": 0.9375,
11226 + "selectivity": 0.0625
11227 + },
11228 + {
11229 + "layer": 3,
11230 + "seed": 1,
11231 + "task_acc": 1.0,
11232 + "control_acc": 0.875,
11233 + "selectivity": 0.125
11234 + },
11235 + {
11236 + "layer": 3,
11237 + "seed": 2,
11238 + "task_acc": 1.0,
11239 + "control_acc": 0.25,
11240 + "selectivity": 0.75
11241 + },
11242 + {
11243 + "layer": 3,
11244 + "seed": 3,
11245 + "task_acc": 1.0,
11246 + "control_acc": 0.041666666666666664,
11247 + "selectivity": 0.9583333333333334
11248 + },
11249 + {
11250 + "layer": 3,
11251 + "seed": 4,
11252 + "task_acc": 1.0,
11253 + "control_acc": 0.20833333333333334,
11254 + "selectivity": 0.7916666666666666
11255 + },
11256 + {
11257 + "layer": 4,
11258 + "seed": 0,
11259 + "task_acc": 1.0,
11260 + "control_acc": 0.8958333333333334,
11261 + "selectivity": 0.10416666666666663
11262 + },
11263 + {
11264 + "layer": 4,
11265 + "seed": 1,
11266 + "task_acc": 1.0,
11267 + "control_acc": 0.9791666666666666,
11268 + "selectivity": 0.02083333333333337
11269 + },
11270 + {
11271 + "layer": 4,
11272 + "seed": 2,
11273 + "task_acc": 1.0,
11274 + "control_acc": 0.25,
11275 + "selectivity": 0.75
11276 + },
11277 + {
11278 + "layer": 4,
11279 + "seed": 3,
11280 + "task_acc": 1.0,
11281 + "control_acc": 0.8958333333333334,
11282 + "selectivity": 0.10416666666666663
11283 + },
11284 + {
11285 + "layer": 4,
11286 + "seed": 4,
11287 + "task_acc": 1.0,
11288 + "control_acc": 0.3333333333333333,
11289 + "selectivity": 0.6666666666666667
11290 + },
11291 + {
11292 + "layer": 5,
11293 + "seed": 0,
11294 + "task_acc": 1.0,
11295 + "control_acc": 0.9166666666666666,
11296 + "selectivity": 0.08333333333333337
11297 + },
11298 + {
11299 + "layer": 5,
11300 + "seed": 1,
11301 + "task_acc": 1.0,
11302 + "control_acc": 0.10416666666666667,
11303 + "selectivity": 0.8958333333333334
11304 + },
11305 + {
11306 + "layer": 5,
11307 + "seed": 2,
11308 + "task_acc": 1.0,
11309 + "control_acc": 0.2916666666666667,
11310 + "selectivity": 0.7083333333333333
11311 + },
11312 + {
11313 + "layer": 5,
11314 + "seed": 3,
11315 + "task_acc": 1.0,
11316 + "control_acc": 0.875,
11317 + "selectivity": 0.125
11318 + },
11319 + {
11320 + "layer": 5,
11321 + "seed": 4,
11322 + "task_acc": 1.0,
11323 + "control_acc": 0.25,
11324 + "selectivity": 0.75
11325 + },
11326 + {
11327 + "layer": 6,
11328 + "seed": 0,
11329 + "task_acc": 1.0,
11330 + "control_acc": 0.8125,
11331 + "selectivity": 0.1875
11332 + },
11333 + {
11334 + "layer": 6,
11335 + "seed": 1,
11336 + "task_acc": 1.0,
11337 + "control_acc": 0.9791666666666666,
11338 + "selectivity": 0.02083333333333337
11339 + },
11340 + {
11341 + "layer": 6,
11342 + "seed": 2,
11343 + "task_acc": 1.0,
11344 + "control_acc": 0.2916666666666667,
11345 + "selectivity": 0.7083333333333333
11346 + },
11347 + {
11348 + "layer": 6,
11349 + "seed": 3,
11350 + "task_acc": 1.0,
11351 + "control_acc": 0.9375,
11352 + "selectivity": 0.0625
11353 + },
11354 + {
11355 + "layer": 6,
11356 + "seed": 4,
11357 + "task_acc": 1.0,
11358 + "control_acc": 0.25,
11359 + "selectivity": 0.75
11360 + },
11361 + {
11362 + "layer": 7,
11363 + "seed": 0,
11364 + "task_acc": 1.0,
11365 + "control_acc": 0.8333333333333334,
11366 + "selectivity": 0.16666666666666663
11367 + },
11368 + {
11369 + "layer": 7,
11370 + "seed": 1,
11371 + "task_acc": 1.0,
11372 + "control_acc": 0.9791666666666666,
11373 + "selectivity": 0.02083333333333337
11374 + },
11375 + {
11376 + "layer": 7,
11377 + "seed": 2,
11378 + "task_acc": 1.0,
11379 + "control_acc": 0.10416666666666667,
11380 + "selectivity": 0.8958333333333334
11381 + },
11382 + {
11383 + "layer": 7,
11384 + "seed": 3,
11385 + "task_acc": 1.0,
11386 + "control_acc": 0.9375,
11387 + "selectivity": 0.0625
11388 + },
11389 + {
11390 + "layer": 7,
11391 + "seed": 4,
11392 + "task_acc": 1.0,
11393 + "control_acc": 0.25,
11394 + "selectivity": 0.75
11395 + },
11396 + {
11397 + "layer": 8,
11398 + "seed": 0,
11399 + "task_acc": 1.0,
11400 + "control_acc": 0.9166666666666666,
11401 + "selectivity": 0.08333333333333337
11402 + },
11403 + {
11404 + "layer": 8,
11405 + "seed": 1,
11406 + "task_acc": 1.0,
11407 + "control_acc": 0.9791666666666666,
11408 + "selectivity": 0.02083333333333337
11409 + },
11410 + {
11411 + "layer": 8,
11412 + "seed": 2,
11413 + "task_acc": 1.0,
11414 + "control_acc": 0.2916666666666667,
11415 + "selectivity": 0.7083333333333333
11416 + },
11417 + {
11418 + "layer": 8,
11419 + "seed": 3,
11420 + "task_acc": 1.0,
11421 + "control_acc": 0.9166666666666666,
11422 + "selectivity": 0.08333333333333337
11423 + },
11424 + {
11425 + "layer": 8,
11426 + "seed": 4,
11427 + "task_acc": 1.0,
11428 + "control_acc": 0.3125,
11429 + "selectivity": 0.6875
11430 + },
11431 + {
11432 + "layer": 9,
11433 + "seed": 0,
11434 + "task_acc": 1.0,
11435 + "control_acc": 0.9375,
11436 + "selectivity": 0.0625
11437 + },
11438 + {
11439 + "layer": 9,
11440 + "seed": 1,
11441 + "task_acc": 1.0,
11442 + "control_acc": 0.0625,
11443 + "selectivity": 0.9375
11444 + },
11445 + {
11446 + "layer": 9,
11447 + "seed": 2,
11448 + "task_acc": 1.0,
11449 + "control_acc": 0.2916666666666667,
11450 + "selectivity": 0.7083333333333333
11451 + },
11452 + {
11453 + "layer": 9,
11454 + "seed": 3,
11455 + "task_acc": 1.0,
11456 + "control_acc": 0.875,
11457 + "selectivity": 0.125
11458 + },
11459 + {
11460 + "layer": 9,
11461 + "seed": 4,
11462 + "task_acc": 1.0,
11463 + "control_acc": 0.8958333333333334,
11464 + "selectivity": 0.10416666666666663
11465 + },
11466 + {
11467 + "layer": 10,
11468 + "seed": 0,
11469 + "task_acc": 1.0,
11470 + "control_acc": 0.9166666666666666,
11471 + "selectivity": 0.08333333333333337
11472 + },
11473 + {
11474 + "layer": 10,
11475 + "seed": 1,
11476 + "task_acc": 1.0,
11477 + "control_acc": 1.0,
11478 + "selectivity": 0.0
11479 + },
11480 + {
11481 + "layer": 10,
11482 + "seed": 2,
11483 + "task_acc": 1.0,
11484 + "control_acc": 0.25,
11485 + "selectivity": 0.75
11486 + },
11487 + {
11488 + "layer": 10,
11489 + "seed": 3,
11490 + "task_acc": 1.0,
11491 + "control_acc": 0.9375,
11492 + "selectivity": 0.0625
11493 + },
11494 + {
11495 + "layer": 10,
11496 + "seed": 4,
11497 + "task_acc": 1.0,
11498 + "control_acc": 0.25,
11499 + "selectivity": 0.75
11500 + },
11501 + {
11502 + "layer": 11,
11503 + "seed": 0,
11504 + "task_acc": 1.0,
11505 + "control_acc": 0.9375,
11506 + "selectivity": 0.0625
11507 + },
11508 + {
11509 + "layer": 11,
11510 + "seed": 1,
11511 + "task_acc": 1.0,
11512 + "control_acc": 0.10416666666666667,
11513 + "selectivity": 0.8958333333333334
11514 + },
11515 + {
11516 + "layer": 11,
11517 + "seed": 2,
11518 + "task_acc": 1.0,
11519 + "control_acc": 0.25,
11520 + "selectivity": 0.75
11521 + },
11522 + {
11523 + "layer": 11,
11524 + "seed": 3,
11525 + "task_acc": 1.0,
11526 + "control_acc": 0.9375,
11527 + "selectivity": 0.0625
11528 + },
11529 + {
11530 + "layer": 11,
11531 + "seed": 4,
11532 + "task_acc": 1.0,
11533 + "control_acc": 0.25,
11534 + "selectivity": 0.75
11535 + },
11536 + {
11537 + "layer": 12,
11538 + "seed": 0,
11539 + "task_acc": 1.0,
11540 + "control_acc": 0.9375,
11541 + "selectivity": 0.0625
11542 + },
11543 + {
11544 + "layer": 12,
11545 + "seed": 1,
11546 + "task_acc": 1.0,
11547 + "control_acc": 0.8958333333333334,
11548 + "selectivity": 0.10416666666666663
11549 + },
11550 + {
11551 + "layer": 12,
11552 + "seed": 2,
11553 + "task_acc": 1.0,
11554 + "control_acc": 0.2916666666666667,
11555 + "selectivity": 0.7083333333333333
11556 + },
11557 + {
11558 + "layer": 12,
11559 + "seed": 3,
11560 + "task_acc": 1.0,
11561 + "control_acc": 0.9375,
11562 + "selectivity": 0.0625
11563 + },
11564 + {
11565 + "layer": 12,
11566 + "seed": 4,
11567 + "task_acc": 1.0,
11568 + "control_acc": 0.9791666666666666,
11569 + "selectivity": 0.02083333333333337
11570 + },
11571 + {
11572 + "layer": 13,
11573 + "seed": 0,
11574 + "task_acc": 1.0,
11575 + "control_acc": 0.9166666666666666,
11576 + "selectivity": 0.08333333333333337
11577 + },
11578 + {
11579 + "layer": 13,
11580 + "seed": 1,
11581 + "task_acc": 1.0,
11582 + "control_acc": 1.0,
11583 + "selectivity": 0.0
11584 + },
11585 + {
11586 + "layer": 13,
11587 + "seed": 2,
11588 + "task_acc": 1.0,
11589 + "control_acc": 0.2708333333333333,
11590 + "selectivity": 0.7291666666666667
11591 + },
11592 + {
11593 + "layer": 13,
11594 + "seed": 3,
11595 + "task_acc": 1.0,
11596 + "control_acc": 0.9375,
11597 + "selectivity": 0.0625
11598 + },
11599 + {
11600 + "layer": 13,
11601 + "seed": 4,
11602 + "task_acc": 1.0,
11603 + "control_acc": 0.9375,
11604 + "selectivity": 0.0625
11605 + },
11606 + {
11607 + "layer": 14,
11608 + "seed": 0,
11609 + "task_acc": 1.0,
11610 + "control_acc": 0.8958333333333334,
11611 + "selectivity": 0.10416666666666663
11612 + },
11613 + {
11614 + "layer": 14,
11615 + "seed": 1,
11616 + "task_acc": 1.0,
11617 + "control_acc": 1.0,
11618 + "selectivity": 0.0
11619 + },
11620 + {
11621 + "layer": 14,
11622 + "seed": 2,
11623 + "task_acc": 1.0,
11624 + "control_acc": 0.7708333333333334,
11625 + "selectivity": 0.22916666666666663
11626 + },
11627 + {
11628 + "layer": 14,
11629 + "seed": 3,
11630 + "task_acc": 1.0,
11631 + "control_acc": 0.9583333333333334,
11632 + "selectivity": 0.04166666666666663
11633 + },
11634 + {
11635 + "layer": 14,
11636 + "seed": 4,
11637 + "task_acc": 1.0,
11638 + "control_acc": 0.22916666666666666,
11639 + "selectivity": 0.7708333333333334
11640 + },
11641 + {
11642 + "layer": 15,
11643 + "seed": 0,
11644 + "task_acc": 1.0,
11645 + "control_acc": 0.9375,
11646 + "selectivity": 0.0625
11647 + },
11648 + {
11649 + "layer": 15,
11650 + "seed": 1,
11651 + "task_acc": 1.0,
11652 + "control_acc": 1.0,
11653 + "selectivity": 0.0
11654 + },
11655 + {
11656 + "layer": 15,
11657 + "seed": 2,
11658 + "task_acc": 1.0,
11659 + "control_acc": 0.2708333333333333,
11660 + "selectivity": 0.7291666666666667
11661 + },
11662 + {
11663 + "layer": 15,
11664 + "seed": 3,
11665 + "task_acc": 1.0,
11666 + "control_acc": 0.9791666666666666,
11667 + "selectivity": 0.02083333333333337
11668 + },
11669 + {
11670 + "layer": 15,
11671 + "seed": 4,
11672 + "task_acc": 1.0,
11673 + "control_acc": 0.3958333333333333,
11674 + "selectivity": 0.6041666666666667
11675 + },
11676 + {
11677 + "layer": 16,
11678 + "seed": 0,
11679 + "task_acc": 1.0,
11680 + "control_acc": 0.9375,
11681 + "selectivity": 0.0625
11682 + },
11683 + {
11684 + "layer": 16,
11685 + "seed": 1,
11686 + "task_acc": 1.0,
11687 + "control_acc": 0.041666666666666664,
11688 + "selectivity": 0.9583333333333334
11689 + },
11690 + {
11691 + "layer": 16,
11692 + "seed": 2,
11693 + "task_acc": 1.0,
11694 + "control_acc": 0.3333333333333333,
11695 + "selectivity": 0.6666666666666667
11696 + },
11697 + {
11698 + "layer": 16,
11699 + "seed": 3,
11700 + "task_acc": 1.0,
11701 + "control_acc": 0.9166666666666666,
11702 + "selectivity": 0.08333333333333337
11703 + },
11704 + {
11705 + "layer": 16,
11706 + "seed": 4,
11707 + "task_acc": 1.0,
11708 + "control_acc": 0.3541666666666667,
11709 + "selectivity": 0.6458333333333333
11710 + },
11711 + {
11712 + "layer": 17,
11713 + "seed": 0,
11714 + "task_acc": 1.0,
11715 + "control_acc": 0.9375,
11716 + "selectivity": 0.0625
11717 + },
11718 + {
11719 + "layer": 17,
11720 + "seed": 1,
11721 + "task_acc": 1.0,
11722 + "control_acc": 0.14583333333333334,
11723 + "selectivity": 0.8541666666666666
11724 + },
11725 + {
11726 + "layer": 17,
11727 + "seed": 2,
11728 + "task_acc": 1.0,
11729 + "control_acc": 0.2708333333333333,
11730 + "selectivity": 0.7291666666666667
11731 + },
11732 + {
11733 + "layer": 17,
11734 + "seed": 3,
11735 + "task_acc": 1.0,
11736 + "control_acc": 0.9583333333333334,
11737 + "selectivity": 0.04166666666666663
11738 + },
11739 + {
11740 + "layer": 17,
11741 + "seed": 4,
11742 + "task_acc": 1.0,
11743 + "control_acc": 0.1875,
11744 + "selectivity": 0.8125
11745 + },
11746 + {
11747 + "layer": 18,
11748 + "seed": 0,
11749 + "task_acc": 1.0,
11750 + "control_acc": 0.9375,
11751 + "selectivity": 0.0625
11752 + },
11753 + {
11754 + "layer": 18,
11755 + "seed": 1,
11756 + "task_acc": 1.0,
11757 + "control_acc": 0.0625,
11758 + "selectivity": 0.9375
11759 + },
11760 + {
11761 + "layer": 18,
11762 + "seed": 2,
11763 + "task_acc": 1.0,
11764 + "control_acc": 0.2708333333333333,
11765 + "selectivity": 0.7291666666666667
11766 + },
11767 + {
11768 + "layer": 18,
11769 + "seed": 3,
11770 + "task_acc": 1.0,
11771 + "control_acc": 0.9375,
11772 + "selectivity": 0.0625
11773 + },
11774 + {
11775 + "layer": 18,
11776 + "seed": 4,
11777 + "task_acc": 1.0,
11778 + "control_acc": 0.25,
11779 + "selectivity": 0.75
11780 + },
11781 + {
11782 + "layer": 19,
11783 + "seed": 0,
11784 + "task_acc": 1.0,
11785 + "control_acc": 0.9375,
11786 + "selectivity": 0.0625
11787 + },
11788 + {
11789 + "layer": 19,
11790 + "seed": 1,
11791 + "task_acc": 1.0,
11792 + "control_acc": 0.041666666666666664,
11793 + "selectivity": 0.9583333333333334
11794 + },
11795 + {
11796 + "layer": 19,
11797 + "seed": 2,
11798 + "task_acc": 1.0,
11799 + "control_acc": 0.2708333333333333,
11800 + "selectivity": 0.7291666666666667
11801 + },
11802 + {
11803 + "layer": 19,
11804 + "seed": 3,
11805 + "task_acc": 1.0,
11806 + "control_acc": 0.9583333333333334,
11807 + "selectivity": 0.04166666666666663
11808 + },
11809 + {
11810 + "layer": 19,
11811 + "seed": 4,
11812 + "task_acc": 1.0,
11813 + "control_acc": 0.9583333333333334,
11814 + "selectivity": 0.04166666666666663
11815 + },
11816 + {
11817 + "layer": 20,
11818 + "seed": 0,
11819 + "task_acc": 1.0,
11820 + "control_acc": 0.9375,
11821 + "selectivity": 0.0625
11822 + },
11823 + {
11824 + "layer": 20,
11825 + "seed": 1,
11826 + "task_acc": 1.0,
11827 + "control_acc": 0.041666666666666664,
11828 + "selectivity": 0.9583333333333334
11829 + },
11830 + {
11831 + "layer": 20,
11832 + "seed": 2,
11833 + "task_acc": 1.0,
11834 + "control_acc": 0.2916666666666667,
11835 + "selectivity": 0.7083333333333333
11836 + },
11837 + {
11838 + "layer": 20,
11839 + "seed": 3,
11840 + "task_acc": 1.0,
11841 + "control_acc": 0.9583333333333334,
11842 + "selectivity": 0.04166666666666663
11843 + },
11844 + {
11845 + "layer": 20,
11846 + "seed": 4,
11847 + "task_acc": 1.0,
11848 + "control_acc": 0.10416666666666667,
11849 + "selectivity": 0.8958333333333334
11850 + },
11851 + {
11852 + "layer": 21,
11853 + "seed": 0,
11854 + "task_acc": 1.0,
11855 + "control_acc": 0.9375,
11856 + "selectivity": 0.0625
11857 + },
11858 + {
11859 + "layer": 21,
11860 + "seed": 1,
11861 + "task_acc": 1.0,
11862 + "control_acc": 0.0625,
11863 + "selectivity": 0.9375
11864 + },
11865 + {
11866 + "layer": 21,
11867 + "seed": 2,
11868 + "task_acc": 1.0,
11869 + "control_acc": 0.22916666666666666,
11870 + "selectivity": 0.7708333333333334
11871 + },
11872 + {
11873 + "layer": 21,
11874 + "seed": 3,
11875 + "task_acc": 1.0,
11876 + "control_acc": 0.9375,
11877 + "selectivity": 0.0625
11878 + },
11879 + {
11880 + "layer": 21,
11881 + "seed": 4,
11882 + "task_acc": 1.0,
11883 + "control_acc": 0.22916666666666666,
11884 + "selectivity": 0.7708333333333334
11885 + },
11886 + {
11887 + "layer": 22,
11888 + "seed": 0,
11889 + "task_acc": 1.0,
11890 + "control_acc": 0.9375,
11891 + "selectivity": 0.0625
11892 + },
11893 + {
11894 + "layer": 22,
11895 + "seed": 1,
11896 + "task_acc": 1.0,
11897 + "control_acc": 0.0625,
11898 + "selectivity": 0.9375
11899 + },
11900 + {
11901 + "layer": 22,
11902 + "seed": 2,
11903 + "task_acc": 1.0,
11904 + "control_acc": 0.2708333333333333,
11905 + "selectivity": 0.7291666666666667
11906 + },
11907 + {
11908 + "layer": 22,
11909 + "seed": 3,
11910 + "task_acc": 1.0,
11911 + "control_acc": 0.9166666666666666,
11912 + "selectivity": 0.08333333333333337
11913 + },
11914 + {
11915 + "layer": 22,
11916 + "seed": 4,
11917 + "task_acc": 1.0,
11918 + "control_acc": 0.1875,
11919 + "selectivity": 0.8125
11920 + },
11921 + {
11922 + "layer": 23,
11923 + "seed": 0,
11924 + "task_acc": 1.0,
11925 + "control_acc": 0.875,
11926 + "selectivity": 0.125
11927 + },
11928 + {
11929 + "layer": 23,
11930 + "seed": 1,
11931 + "task_acc": 1.0,
11932 + "control_acc": 1.0,
11933 + "selectivity": 0.0
11934 + },
11935 + {
11936 + "layer": 23,
11937 + "seed": 2,
11938 + "task_acc": 1.0,
11939 + "control_acc": 0.2708333333333333,
11940 + "selectivity": 0.7291666666666667
11941 + },
11942 + {
11943 + "layer": 23,
11944 + "seed": 3,
11945 + "task_acc": 1.0,
11946 + "control_acc": 0.9583333333333334,
11947 + "selectivity": 0.04166666666666663
11948 + },
11949 + {
11950 + "layer": 23,
11951 + "seed": 4,
11952 + "task_acc": 1.0,
11953 + "control_acc": 0.20833333333333334,
11954 + "selectivity": 0.7916666666666666
11955 + },
11956 + {
11957 + "layer": 24,
11958 + "seed": 0,
11959 + "task_acc": 1.0,
11960 + "control_acc": 0.8958333333333334,
11961 + "selectivity": 0.10416666666666663
11962 + },
11963 + {
11964 + "layer": 24,
11965 + "seed": 1,
11966 + "task_acc": 1.0,
11967 + "control_acc": 0.9166666666666666,
11968 + "selectivity": 0.08333333333333337
11969 + },
11970 + {
11971 + "layer": 24,
11972 + "seed": 2,
11973 + "task_acc": 1.0,
11974 + "control_acc": 0.8333333333333334,
11975 + "selectivity": 0.16666666666666663
11976 + },
11977 + {
11978 + "layer": 24,
11979 + "seed": 3,
11980 + "task_acc": 1.0,
11981 + "control_acc": 0.9166666666666666,
11982 + "selectivity": 0.08333333333333337
11983 + },
11984 + {
11985 + "layer": 24,
11986 + "seed": 4,
11987 + "task_acc": 1.0,
11988 + "control_acc": 0.9583333333333334,
11989 + "selectivity": 0.04166666666666663
11990 + },
11991 + {
11992 + "layer": 25,
11993 + "seed": 0,
11994 + "task_acc": 1.0,
11995 + "control_acc": 0.9166666666666666,
11996 + "selectivity": 0.08333333333333337
11997 + },
11998 + {
11999 + "layer": 25,
12000 + "seed": 1,
12001 + "task_acc": 1.0,
12002 + "control_acc": 0.9166666666666666,
12003 + "selectivity": 0.08333333333333337
12004 + },
12005 + {
12006 + "layer": 25,
12007 + "seed": 2,
12008 + "task_acc": 1.0,
12009 + "control_acc": 0.75,
12010 + "selectivity": 0.25
12011 + },
12012 + {
12013 + "layer": 25,
12014 + "seed": 3,
12015 + "task_acc": 1.0,
12016 + "control_acc": 0.8333333333333334,
12017 + "selectivity": 0.16666666666666663
12018 + },
12019 + {
12020 + "layer": 25,
12021 + "seed": 4,
12022 + "task_acc": 1.0,
12023 + "control_acc": 0.3333333333333333,
12024 + "selectivity": 0.6666666666666667
12025 + },
12026 + {
12027 + "layer": 26,
12028 + "seed": 0,
12029 + "task_acc": 1.0,
12030 + "control_acc": 0.9375,
12031 + "selectivity": 0.0625
12032 + },
12033 + {
12034 + "layer": 26,
12035 + "seed": 1,
12036 + "task_acc": 1.0,
12037 + "control_acc": 1.0,
12038 + "selectivity": 0.0
12039 + },
12040 + {
12041 + "layer": 26,
12042 + "seed": 2,
12043 + "task_acc": 1.0,
12044 + "control_acc": 0.2916666666666667,
12045 + "selectivity": 0.7083333333333333
12046 + },
12047 + {
12048 + "layer": 26,
12049 + "seed": 3,
12050 + "task_acc": 1.0,
12051 + "control_acc": 0.041666666666666664,
12052 + "selectivity": 0.9583333333333334
12053 + },
12054 + {
12055 + "layer": 26,
12056 + "seed": 4,
12057 + "task_acc": 1.0,
12058 + "control_acc": 0.9791666666666666,
12059 + "selectivity": 0.02083333333333337
12060 + },
12061 + {
12062 + "layer": 27,
12063 + "seed": 0,
12064 + "task_acc": 1.0,
12065 + "control_acc": 0.9375,
12066 + "selectivity": 0.0625
12067 + },
12068 + {
12069 + "layer": 27,
12070 + "seed": 1,
12071 + "task_acc": 1.0,
12072 + "control_acc": 0.8958333333333334,
12073 + "selectivity": 0.10416666666666663
12074 + },
12075 + {
12076 + "layer": 27,
12077 + "seed": 2,
12078 + "task_acc": 1.0,
12079 + "control_acc": 0.8125,
12080 + "selectivity": 0.1875
12081 + },
12082 + {
12083 + "layer": 27,
12084 + "seed": 3,
12085 + "task_acc": 1.0,
12086 + "control_acc": 0.9583333333333334,
12087 + "selectivity": 0.04166666666666663
12088 + },
12089 + {
12090 + "layer": 27,
12091 + "seed": 4,
12092 + "task_acc": 1.0,
12093 + "control_acc": 0.10416666666666667,
12094 + "selectivity": 0.8958333333333334
12095 + }
12096 + ]
12097 + }
12098 + }
12099 + },
12100 + "wall_seconds": 48.3
12101 +}
modified site/data/atlas-index.json +13 −1
@@ -2,5 +2,17 @@
2 2 "author": "Simon-Pierre Boucher",
3 3 "contact": "contact@spboucher.ai",
4 4 "website": "https://modelmap.io",
5 − "entries": []
5 + "entries": [
6 + {
7 + "path": "atlas/qwen3-0.6b-4bit/probes/v1",
8 + "map_id": "atlas/qwen3-0.6b-4bit/probes/v1",
9 + "map_type": "probes",
10 + "model_id": "mlx-community/Qwen3-0.6B-4bit",
11 + "quantization": "q4 (mlx)",
12 + "confidence_level": 1,
13 + "replication_rate": 1.0,
14 + "negative_result": true,
15 + "created": "2026-08-12"
16 + }
17 + ]
6 18 }
added src/modelmap/capture/mlx_capture.py +93 −0
@@ -0,0 +1,93 @@
1 +# =============================================================================
2 +# Project : modelmap
3 +# File : src/modelmap/capture/mlx_capture.py
4 +# Purpose : Standard MLX capture layer — Tap wrappers over decoder layers
5 +# Author : Simon-Pierre Boucher
6 +# Contact : contact@spboucher.ai
7 +# Website : https://modelmap.io
8 +# Created : 2026-08-12
9 +# Modified : 2026-08-12
10 +# Platform : macOS / Apple Silicon (arm64) — MLX / Metal
11 +# License : All rights reserved (research code)
12 +# =============================================================================
13 +"""MLX activation capture (promoted from expH run #3, where retention
14 +measured 1.004x plain prefill on a real 4-bit checkpoint).
15 +
16 +Works on quantized mlx-lm models — the §4.9 capability gap. Layers are
17 +wrapped with Tap objects (mlx-lm decoder loops call layers positionally, so
18 +a plain callable suffices); d_model is inferred from runtime activations
19 +because quantized embeddings pack their weight shapes; bf16 activations are
20 +cast in MLX before NumPy conversion (numpy has no bfloat16).
21 +"""
22 +
23 +from __future__ import annotations
24 +
25 +import numpy as np
26 +
27 +
28 +class Tap:
29 + """Wraps a decoder layer; optionally retains its output."""
30 +
31 + def __init__(self, layer):
32 + self.layer = layer
33 + self.retained = None
34 + self.enabled = False
35 +
36 + def __call__(self, *args, **kwargs):
37 + out = self.layer(*args, **kwargs)
38 + if self.enabled:
39 + self.retained = out
40 + return out
41 +
42 + def __getattr__(self, name): # delegate (e.g. .self_attn) for introspection
43 + return getattr(self.layer, name)
44 +
45 +
46 +def install_taps(model) -> list[Tap]:
47 + """Wrap every decoder layer of an mlx-lm model in place; return the taps."""
48 + taps = [Tap(layer) for layer in model.model.layers]
49 + model.model.layers = taps
50 + return taps
51 +
52 +
53 +def capture_mean_pooled(model, taps, token_ids_list, dtype=np.float32) -> np.ndarray:
54 + """Per-prompt, per-layer mean-pooled residual representations.
55 +
56 + Returns array of shape (n_prompts, n_layers, d_model). Batch = 1 per
57 + prompt (variable lengths, no padding confound); mean over sequence
58 + positions. Each forward is a full prefill (no KV cache reuse across
59 + prompts).
60 + """
61 + import mlx.core as mx
62 +
63 + for t in taps:
64 + t.enabled = True
65 + reps = None
66 + for i, ids in enumerate(token_ids_list):
67 + x = mx.array([list(ids)])
68 + model(x)
69 + pooled = [t.retained.mean(axis=1)[0].astype(mx.float32) for t in taps]
70 + mx.eval(*pooled)
71 + if reps is None:
72 + reps = np.zeros((len(token_ids_list), len(taps), pooled[0].shape[-1]), dtype=dtype)
73 + for li, p in enumerate(pooled):
74 + reps[i, li] = np.array(p, copy=False)
75 + for t in taps:
76 + t.retained = None
77 + for t in taps:
78 + t.enabled = False
79 + return reps
80 +
81 +
82 +def random_init_twin(model_path):
83 + """Instantiate the same architecture with random weights (fp16, no
84 + quantization) — the architecture-only null mandated by charter §8.4."""
85 + import mlx.core as mx
86 + from mlx_lm.utils import _get_classes, load_config
87 +
88 + cfg = load_config(model_path)
89 + cfg_clean = {k: v for k, v in cfg.items() if k != "quantization"}
90 + model_class, args_class = _get_classes(cfg)
91 + twin = model_class(args_class.from_dict(cfg_clean))
92 + twin.set_dtype(mx.float16)
93 + return twin
94