Share your thoughts, 1 month free Claude Pro on us
See more
Home
/
Benchmarks
Conversation Simulation on GSM8k and HumanEval 65 task intents
Loading...
86.6
Intent Coverage
4o-mini
76.304
78.977
81.65
84.323
Oct 8, 2025
Intent Coverage
Repeat Rate (%)
Non-Required Skip Rate (%)
Added Demand Rate (%)
Turn Variance
Minimum Turns
Unigram Difference
Assistant Score
Updated 4mo ago
Evaluation Results
Method
Method
Links
Intent Coverage
Repeat Rate (%)
Non-Required Skip Rate (%)
Added Demand Rate (%)
Turn Variance
Minimum Turns
Unigram Difference
Assistant Score
4o-mini
User Simulator=4o-mini
2025.10
86.6
31.8
10.9
9.5
0.9
3.7
0.43
73.2
GPT-4o
User Simulator=GPT-4o
2025.10
84.7
9.4
14.6
1.1
0.6
4
0.4
74.6
UserLM-8b
User Simulator=UserLM-8b
2025.10
76.7
54.3
37.7
43.8
2.8
2.1
0.71
57.4
Feedback
Search any
task
Search any
task
