Share your thoughts, 1 month free Claude Pro on usSee more

End-task performance evaluation on SRA-Bench

71.6TheoremQA Score

Oracle Skill

Updated 1mo ago

Evaluation Results

Method
Oracle Skill 2026.04	71.6	86.6	51.2	70.9	83.5	55.2	67.2
Oracle Skill 2026.04	69.7	85.1	47.1	70.9	73.5	45.5	61.6
LLM Selection 2026.04	69.3	81.1	44.1	65.9	82.5	48.1	62.4
Oracle Skill 2026.04	68.5	81.1	48.7	68.6	79.7	54.7	64.4
Progressive Disclosure 2026.04	68.1	81.1	36.9	72.2	77.1	50.3	59.9
Full-Skill Injection 2026.04	68.1	70.5	36.2	70.4	59.5	49	54.3
LLM Selection 2026.04	66.8	79.5	45.4	73.5	82.5	49.7	62.8
Oracle Skill 2026.04	66.3	78.6	48.2	66.8	78.6	54	63.2
Oracle Skill 2026.04	65.9	86.6	52.3	75.8	84.5	57	67.5
Progressive Disclosure 2026.04	64.9	74.7	35.5	58.7	71.1	44.7	55.3
Full-Skill Injection 2026.04	64.4	72.8	38	68.6	66.2	50.4	56.2
Full-Skill Injection 2026.04	64	69.1	30.2	68.6	36.1	41.5	45.5
LLM Selection 2026.04	63.1	70	39	62.3	65.7	39.7	53.3
LLM Selection 2026.04	62.7	69.7	43.3	58.7	79.4	50.4	59.2
LLM Direct 2026.04	61.6	76.1	36.4	66.4	58.2	46.7	53.3
Full-Skill Injection 2026.04	60.1	66.1	35	61.9	59.5	52.9	52.7
LLM Direct 2026.04	59.6	60.5	30.9	56.5	53.9	45	47.8
LLM Selection 2026.04	59	66.7	42.6	55.6	76.3	46.8	56.6
Full-Skill Injection 2026.04	58.8	57.2	34.4	59.6	54.6	45.6	48.5
LLM Direct 2026.04	57.4	75.9	35	65.5	53.9	43.9	50.8
Progressive Disclosure 2026.04	54.8	55.8	27.1	54.7	62.6	42.8	46.7
Progressive Disclosure 2026.04	53.3	65.3	29.9	54.3	59.5	45.7	48.5
Progressive Disclosure 2026.04	52.7	65.7	26.7	60.1	45	37.7	43.2
LLM Direct 2026.04	50.7	75	25.6	56.1	22	36.4	38.8
Oracle Skill 2026.04	49.4	69.5	23.3	40.8	62	35.2	44.5
LLM Direct 2026.04	49.3	56.1	29.2	52.5	49.6	41.1	43.4
Progressive Disclosure 2026.04	36.9	50	16.4	25.1	59.6	31.4	36.3
Full-Skill Injection 2026.04	36.5	58.3	13.6	27.8	36.7	34.2	32.7
LLM Selection 2026.04	35.1	53.8	19.4	24.7	57	32.1	37
LLM Direct 2026.04	32.4	54.6	16.7	22.4	26.9	32.3	29.8