Share your thoughts, 1 month free Claude Pro on usSee more

Counterfactual Generation on TAXI1500

93.7SLFR

Gemma3-12B

Updated 21d ago

Evaluation Results

Method
Gemma3-12B 2026.05	93.7	66.7	13.01	25.7	8.387	-7.387
Gemma3-4B 2026.05	91.9	33.3	17.79	32.7	11.583	-10.583
Gemma3-12B 2026.05	91.9	76.6	13.54	47.6	7.483	-6.483
Gemma3-4B 2026.05	89.2	31.5	14.64	29.1	26.717	-25.717
Gemma3-12B 2026.05	89.2	55.9	12.08	24.7	8.459	-7.459
Gemma3-12B 2026.05	88.3	60.4	15.75	27.1	9.984	-8.984
Gemma3-4B 2026.05	87.4	32.4	11.59	29.3	8.018	-7.018
Gemma3-4B 2026.05	86.5	35.1	14.78	50.3	7.537	-6.537
Gemma3-4B 2026.05	86.5	28.8	12.73	31.7	9.932	-8.932
Gemma3-4B 2026.05	83.8	27	10.46	29.5	7.942	-6.942
Gemma3-12B 2026.05	83.8	64	17.4	25.7	26.798	-25.798
Gemma3-12B 2026.05	83.2	73.6	41.48	43.7	0.731	0.269
Gemma3-12B 2026.05	80.2	63.1	22.22	29.8	11.946	-10.946
Gemma3-12B 2026.05	79.1	67.8	26.5	53.7	0.686	0.314
MACRO 2026.05	79	-	23.8	60	-	-
Gemma3-12B 2026.05	77.7	67.2	37.25	53.7	0.699	0.301
DG-CF 2026.05	75	-	26.8	55	-	-
Gemma3-12B 2026.05	74.9	64.3	20.49	60	0.743	0.257
Gemma3-12B 2026.05	74.4	65	16.18	51.7	0.707	0.293
SFT 2026.05	71	-	21.5	58	-	-
Gemma3-12B 2026.05	69.4	57.8	17.46	68.4	0.653	0.347
Gemma3-12B 2026.05	68.2	60.5	27.9	54.4	0.85	0.15
Qwen3-4B 2026.05	68.1	34.2	86.15	57.7	0.615	0.385
Qwen3-4B 2026.05	67.6	28.8	19.18	56.5	0.814	0.186
Qwen3-8B 2026.05	66.7	33.3	41.3	60	0.808	0.192
Qwen3-8B 2026.05	65.8	38.7	50.29	63.9	0.622	0.378
Qwen3-4B 2026.05	65.1	35.6	25.67	66	0.485	0.515
Qwen3-4B 2026.05	64.9	27.9	55.98	56.3	0.839	0.161
MACRO 2026.05	64	-	24.9	70	-	-
Qwen3-4B 2026.05	63.1	32.4	25.04	62	0.848	0.152
Qwen3-8B 2026.05	63.1	36	38.92	66.2	0.689	0.311
Qwen3-4B 2026.05	62.3	48.5	32.64	61.4	0.48	0.52
Qwen3-4B 2026.05	62.2	30.6	35.72	60	0.927	0.073
MACRO 2026.05	62	-	28.9	70	-	-
SFT 2026.05	62	-	22.6	65	-	-
MACRO 2026.05	60	-	23.5	59	-	-
MACRO 2026.05	60	-	23.5	68	-	-
SFT 2026.05	60	-	28	66	-	-
Qwen3-4B 2026.05	58.7	30.9	45.84	67.4	0.589	0.411
Qwen3-8B 2026.05	57.7	32.4	33.37	63.9	0.933	0.067
Qwen3-8B 2026.05	57.7	40.5	18.83	61.3	0.621	0.379
DG-CF 2026.05	57	-	40.5	67	-	-
Gemma3-4B 2026.05	56.9	35.7	25.59	67	0.647	0.353
SFT 2026.05	56	-	23.4	59	-	-
SFT 2026.05	56	-	24	65	-	-
Qwen3-8B 2026.05	55.9	38.7	30.95	68.2	0.474	0.526
Gemma3-4B 2026.05	55.7	40.5	33.93	63.9	0.58	0.42
Gemma3-4B 2026.05	53.2	39.4	21.49	65.6	0.65	0.35
Gemma3-4B 2026.05	53.1	43.4	36.24	50.7	0.67	0.33
DG-CF 2026.05	53	-	36.2	51	-	-
DG-CF 2026.05	53	-	31	62	-	-
Qwen3-4B 2026.05	52.9	31.5	27.73	67.5	0.526	0.474
Gemma3-4B 2026.05	52.3	40.5	39.48	61.2	0.807	0.193
Gemma3-4B 2026.05	52.2	34.2	41.88	62.8	0.716	0.284
Qwen3-8B 2026.05	51.4	30.6	16.36	60.3	0.849	0.151
Qwen3-4B 2026.05	50.5	11.7	14.9	55.3	1.765	-0.765
Gemma3-4B 2026.05	50.4	37.7	18.47	61.3	0.631	0.369
DG-CF 2026.05	50	-	28.9	67	-	-
Qwen3-8B 2026.05	49.5	34.2	25.85	61.7	0.562	0.438
Qwen3-4B 2026.05	48.6	28.4	40.17	66.2	0.444	0.556
Qwen3-4B 2026.05	48.6	34.2	37.8	58.9	0.731	0.269
Qwen3-4B 2026.05	46.3	10.7	25.61	84	0.312	0.688
Qwen3-8B 2026.05	45	27	18.73	67.6	0.566	0.434
Qwen3-8B 2026.05	43.2	25.2	26.46	64.6	0.727	0.273
Qwen3-8B 2026.05	37.8	28.8	18.33	67.1	0.847	0.153
Qwen3-8B 2026.05	18	5.4	18.9	61	1.241	-0.241
Qwen3-8B 2026.05	10.8	0.9	18.96	83.3	0.693	0.307