Share your thoughts, 1 month free Claude Pro on usSee more

General language understanding and reasoning on TRACE

59C-STANCE Accuracy

MagMax

Updated 1mo ago

Evaluation Results

Method	Links
MagMax 2025.02		59	45	0.37	0.32	43	42	45	0.4	0.42
Task-Arith. 2025.02		51	2	0.39	0.56	29	40	52	0.38	0.38
Avg. (Offline) 2025.02		50	21	0.38	0.59	35	38	49	0.37	0.41
Task-Arith. 2025.02		50	47	0.38	0.6	32	42	52	0.4	0.45
Replay 2025.02		49	21	0.38	0.5	47	35	41	0.39	0.4
Replay 2025.02		49	13	0.37	0.57	21	38	56	0.37	0.38
Task-Arith. 2025.02		49	1	0.39	0.59	32	40	53	0.39	0.39
Task-Arith. 2025.02		49	28	0.39	0.57	40	44	48	0.38	0.43
MagMax 2025.02		49	44	0.25	0.26	37	31	48	0.39	0.37
Avg. (Offline) 2025.02		48	2	0.37	0.58	29	34	51	0.37	0.36
Replay 2025.02		47	13	0.38	0.55	18	40	56	0.37	0.38
MagMax 2025.02		46	41	0.39	0.26	35	43	43	0.4	0.39
Weight Space Consolidation 2025.02		46	47	0.37	0.55	46	36	53	0.4	0.45
SLERP 2025.02		45	42	0.26	0.53	33	29	48	0.43	0.4
SLERP 2025.02		45	18	0.37	0.58	39	37	53	0.38	0.41
Avg. (Online) 2025.02		45	50	0.41	0.55	35	34	52	0.39	0.44
Avg. (Online) 2025.02		45	49	0.35	0.52	16	35	47	0.36	0.39
Avg. (Offline) 2025.02		45	3	0.38	0.58	33	40	48	0.37	0.38
Avg. (Online) 2025.02		44	46	0.33	0.52	39	31	37	0.37	0.4
Avg. (Offline) 2025.02		44	56	0.38	0.59	36	38	53	0.36	0.45
Weight Space Consolidation 2025.02		44	57	0.35	0.52	47	38	55	0.39	0.46
Replay 2025.02		43	1	0.35	0.5	29	32	38	0.37	0.33
Fine-tune 2025.02		42	25	0.33	0.49	21	28	51	0.37	0.35
SLERP 2025.02		42	45	0.31	0.54	36	29	42	0.4	0.39
SLERP 2025.02		42	56	0.38	0.57	30	34	54	0.38	0.43
Avg. (Online) 2025.02		42	41	0.35	0.55	43	28	43	0.44	0.41
Weight Space Consolidation 2025.02		41	48	0.38	0.55	45	41	30	0.39	0.42
Weight Space Consolidation 2025.02		39	13	0.36	0.51	32	40	56	0.39	0.38
MagMax 2025.02		32	54	0.23	0.25	32	29	51	0.39	0.36