Faithfulness Evaluation on Halogen

73CODE

Distributional Semantics Tracing

Updated 1mo ago

Evaluation Results

Method
Distributional Semantics Tracing 2025.10	73	77	83	84	78	79
Sparse Autoencoders 2025.10	64	53	61	52	63	59
Distributional Semantics Tracing 2025.10	63	59	67	77	65	66
Patchscopes 2025.10	62	68	45	64	49	58
Subsequence Analysis Tracing 2025.10	57	45	56	55	59	54
Causal Path Tracing 2025.10	56	54	65	54	49	56
Causal Path Tracing 2025.10	56	54	60	65	56	58
Sparse Autoencoders 2025.10	54	49	51	44	55	51
Token Evolution (Logit Lens) 2025.10	54	43	45	45	54	48
Subsequence Analysis Tracing 2025.10	54	59	56	57	59	57
Patchscopes 2025.10	49	56	53	45	54	51
Token Evolution (Logit Lens) 2025.10	45	52	44	54	39	47
attention 2025.10	45	29	31	21	22	30
Reagent 2025.10	42	44	43	45	33	41
Reagent 2025.10	35	45	19	24	19	29
lime 2025.10	34	33	45	29	29	34
gradient-shap 2025.10	32	40	35	34	20	32
gradient-shap 2025.10	31	35	13	10	15	21
lime 2025.10	29	32	15	13	14	21
attention 2025.10	24	28	19	12	12	19