TL;DR
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Summarization | TL;DR | Winrate91.8 | 59 | |
| Summarization | TL;DR (test) | Win Rate82.5 | 49 | |
| Preference Alignment | TL;DR (test) | Win Rate68.8 | 36 | |
| LLM Judgement Confidence Estimation | TL;DR (test) | RK0.4269 | 16 | |
| Summarization | TL;DR (distillation set) | Word Count27.24 | 16 | |
| Text Summarization | TL;DR | AlignScore94.2 | 15 | |
| Reward Modeling | TL;DR Seen (n=100) | Accuracy62.3 | 14 | |
| LLM-as-a-judge | TL;DR | Coverage82.6 | 12 | |
| Confidence Estimation | TL;DR | Rank Correlation (RK)0.421 | 11 | |
| Summarization | TL;DR | Completeness43 | 11 | |
| Reward Modeling | TL;DR Overall n=150 | Accuracy62.9 | 7 | |
| Reward Modeling | TL;DR Unseen (n=150) | Accuracy62.4 | 7 | |
| Reward Modeling | TL;DR n=150 Seen | Accuracy63.3 | 7 | |
| Reward Modeling | TL;DR n=100 Unseen | Accuracy61.5 | 7 | |
| Summarization | TL;DR | Win Rate92.8 | 6 | |
| Summarization (Groundedness) | TL;DR | Kendall's Tau0.46 | 5 | |
| Summarization (Completeness) | TL;DR | Kendall's Tau0.44 | 5 | |
| LLM Alignment | TL;DR (test) | Win Rate (GPT-4o)68.56 | 4 | |
| Preference Alignment | TL;DR | GRA (%)64.4 | 4 | |
| Summarization | TL;DR | Winrate50.5 | 4 | |
| Summarization Preference Evaluation | TL;DR (val) | Metric- | 0 | |
| Text Summarization | TL;DR (test) | Metric- | 0 |
Showing 22 of 22 rows