Share your thoughts, 1 month free Claude Pro on usSee more

Conversational Agent Evaluation on tau2 Airline (Success Rate)

78Success Rate

Pass@N (oracle)

Updated 1mo ago

Evaluation Results

Method	Links
Pass@N (oracle) 2026.06		78
DeepConf B10 2026.06		72
Progress Advantage 2026.06		72
Mean-of-N 2026.06		64.8
WildReward-8B 2026.06		64
ThinkPRM-7B 2026.06		64
Self-Certainty 2026.06		64
DeepConf Tail 2026.06		62
Greedy Decoding 2026.06		60
Pass@N (oracle) 2026.06		58
ThinkPRM-14B 2026.06		58
Progress Advantage 2026.06		48
DeepConf Tail 2026.06		36
Mean-of-N 2026.06		34.5
Greedy Decoding 2026.06		34
Self-Certainty 2026.06		34
ThinkPRM-7B 2026.06		30
WildReward-8B 2026.06		28
ThinkPRM-14B 2026.06		28
DeepConf B10 2026.06		28