Share your thoughts, 1 month free Claude Pro on usSee more

Counterfactual Generation on SIB200

86.7SLFR

Gemma3-12B

Updated 21d ago

Evaluation Results

Method
Gemma3-12B 2026.05	86.7	78.9	25.48	52.1	0.467	0.533
MACRO 2026.05	85	-	26.3	57	-	-
Gemma3-12B 2026.05	84.2	78.9	31.61	55.1	0.554	0.446
Gemma3-12B 2026.05	83	76.3	53.9	53	0.471	0.529
SFT 2026.05	83	-	23.6	58	-	-
Gemma3-12B 2026.05	82.8	57.6	16.96	7.3	22.704	-21.704
Gemma3-12B 2026.05	82.8	69.7	12.45	39.9	7.464	-6.464
Gemma3-12B 2026.05	81.8	55.6	25.19	6.3	8.081	-7.081
Gemma3-12B 2026.05	81.4	73.3	14.86	65.3	0.446	0.554
DG-CF 2026.05	81	-	27.6	57	-	-
MACRO 2026.05	80	-	30.1	66	-	-
Gemma3-12B 2026.05	79.4	70.9	18.54	55.7	0.508	0.492
Gemma3-12B 2026.05	78.8	66.7	13.32	23.9	7.754	-6.754
Gemma3-12B 2026.05	77.9	69.1	21.03	58.5	0.494	0.506
Qwen3-4B 2026.05	76.8	37.4	44.39	7.8	1.211	-0.211
Qwen3-4B 2026.05	76.8	37.4	40.49	8.4	0.941	0.059
Gemma3-12B 2026.05	76.8	65.7	12.51	25.5	8.144	-7.144
Qwen3-4B 2026.05	76.7	72.4	43.75	59.8	0.438	0.562
Gemma3-12B 2026.05	76.7	70.3	28.13	57.4	0.472	0.528
SFT 2026.05	76	-	28.5	65	-	-
Qwen3-8B 2026.05	75.8	40.4	34.94	8	0.951	0.049
Gemma3-12B 2026.05	75.8	62.6	15.23	24	7.769	-6.769
SFT 2026.05	74	-	24.6	60	-	-
MACRO 2026.05	74	-	17.6	67	-	-
Gemma3-4B 2026.05	73.7	41.4	11.95	26.7	7.64	-6.64
SFT 2026.05	72	-	18.2	69	-	-
SFT 2026.05	72	-	19.3	70	-	-
Qwen3-8B 2026.05	71.7	40.4	34.98	8.3	1.235	-0.235
SFT 2026.05	71	-	23.2	65	-	-
SFT 2026.05	71	-	24.4	66	-	-
Qwen3-4B 2026.05	70.9	61.2	45.15	65.8	0.306	0.694
SFT 2026.05	70	-	31.4	55	-	-
MACRO 2026.05	70	-	27.9	70	-	-
Gemma3-4B 2026.05	68.7	31.3	15.68	7.4	22.77	-21.77
Gemma3-4B 2026.05	68.7	42.4	13.59	42.7	7.62	-6.62
Qwen3-8B 2026.05	68.1	61.4	22.66	69.2	0.308	0.692
DG-CF 2026.05	68	-	18.7	68	-	-
MACRO 2026.05	68	-	24	69	-	-
Qwen3-4B 2026.05	67.8	57.8	26.15	64.9	0.32	0.68
Gemma3-4B 2026.05	67.7	43.4	10.8	31.7	7.959	-6.959
Gemma3-4B 2026.05	67.7	40.4	12.04	28.8	8.039	-7.039
Gemma3-4B 2026.05	67.6	54.9	18.71	68.1	0.446	0.554
SFT 2026.05	67	-	26.1	68	-	-
Gemma3-4B 2026.05	66.2	62.9	35.09	69.2	0.467	0.533
DG-CF 2026.05	66	-	35.1	69	-	-
MACRO 2026.05	66	-	26.3	65	-	-
Gemma3-4B 2026.05	65.7	29.3	19.29	7.3	8.045	-7.045
Qwen3-8B 2026.05	65.2	56.8	32.09	68.6	0.263	0.737
SFT 2026.05	65	-	24	71	-	-
SFT 2026.05	65	-	16.5	67	-	-
MACRO 2026.05	65	-	18.7	71	-	-
Qwen3-4B 2026.05	64.6	59.1	40.77	70.4	0.251	0.749
MACRO 2026.05	64	-	31.7	65	-	-
MACRO 2026.05	64	-	24.4	78	-	-
MACRO 2026.05	64	-	19.1	74	-	-
Qwen3-8B 2026.05	63.5	57.2	22.08	69.6	0.295	0.705
DG-CF 2026.05	63	-	48.8	61	-	-
MACRO 2026.05	63	-	27.6	75	-	-
Gemma3-4B 2026.05	62.7	55.9	48.76	61.3	0.529	0.471
DG-CF 2026.05	62	-	28.3	69	-	-
Qwen3-8B 2026.05	61.5	53.4	38.59	68.1	0.32	0.68
DG-CF 2026.05	61	-	27.7	66	-	-
DG-CF 2026.05	61	-	20.3	74	-	-
DG-CF 2026.05	61	-	38.8	72	-	-
DG-CF 2026.05	61	-	32.4	72	-	-
Gemma3-4B 2026.05	60.9	54.8	27.74	66.3	0.401	0.599
Gemma3-4B 2026.05	60.5	51.4	20.26	73.7	0.459	0.541
Qwen3-4B 2026.05	59.6	45.5	41.4	73.3	0.562	0.438
Qwen3-8B 2026.05	59.6	49.5	30.57	66.1	0.558	0.442
Qwen3-8B 2026.05	59.6	45.5	22.43	66.2	0.609	0.391
Qwen3-8B 2026.05	59.4	48.9	53.24	73.1	0.231	0.769
Gemma3-4B 2026.05	59.2	49.9	28.54	75.4	0.389	0.611
Gemma3-4B 2026.05	59	50.8	18.88	71.1	0.456	0.544
DG-CF 2026.05	59	-	28.5	75	-	-
DG-CF 2026.05	59	-	18.9	71	-	-
Qwen3-8B 2026.05	58.6	51.5	19.4	65.6	0.583	0.417
Qwen3-8B 2026.05	57.6	50.5	34.88	73.5	0.169	0.831
Qwen3-4B 2026.05	56.6	14.1	18.69	49.1	1.889	-0.889
Qwen3-8B 2026.05	55.6	24.2	22.4	52.1	1.512	-0.512
Qwen3-8B 2026.05	53.3	24.4	22.93	84.4	0.324	0.676
Qwen3-4B 2026.05	53.1	48.2	58.52	80.1	0.224	0.776
Qwen3-4B 2026.05	51.2	13.7	42.85	71.4	0.397	0.603
Qwen3-4B 2026.05	50.5	44.4	32.23	73.1	0.612	0.388
Qwen3-4B 2026.05	49.6	42.3	23.73	79.3	0.199	0.801
Qwen3-4B 2026.05	47.5	39.4	20.38	73.3	0.579	0.421