Share your thoughts, 1 month free Claude Pro on usSee more

Circuit Recovery Evaluation Suite Llama-3 1B to 8B

67Arithmetic (+) Accuracy

NAP-IG-Inputs (DFA (Far/Zero-Shot))

Updated 1mo ago

Evaluation Results

Method	Links
NAP-IG-Inputs (DFA (Far/Zero-Shot)) 2026.04		67	33	0.51	39	47	38	46
NAP-IG-Inputs (DFA (Near-Distribution)) 2026.04		53	-	-	28	17	37	34
NAP-IG-Inputs (Gold (Upper Bound)) 2026.04		46	63	1.02	37	97	94	73
NAP-IG-Activations (DFA (Near-Distribution)) 2026.04		41	-	-	27	25	27	30
NAP-IG-Activations (DFA (Far/Zero-Shot)) 2026.04		41	36	0.4	28	35	40	37
NAP (DFA (Far/Zero-Shot)) 2026.04		40	27	0.4	26	30	29	32
NAP-IG-Inputs (DFA (In-Distribution)) 2026.04		39	34	0.51	27	30	33	36
NAP-IG-Activations (DFA (In-Distribution)) 2026.04		38	43	0.32	27	29	27	33
NAP (DFA (In-Distribution)) 2026.04		35	33	0.46	33	49	53	41
NAP-IG-Activations (Gold (Upper Bound)) 2026.04		33	35	0.6	31	59	63	47
NAP (DFA (Near-Distribution)) 2026.04		30	-	-	26	47	46	37
Baseline (Random Alignment) 2026.04		25	25	0.25	25	25	25	25
NAP (Gold (Upper Bound)) 2026.04		25	31	0.38	29	57	59	40