Natural Instructions
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Instruction Following | Natural Instructions (test) | Rouge-L97.9 | 90 | |
| Cross-Task Generalization | Super-NaturalInstructions English Track (unseen clients) | Weighted Avg Rouge-L62.2 | 27 | |
| Instruction Tuning | Natural Instructions Meta Non-IID | Rouge-L34.81 | 22 | |
| Federated Learning | Natural Instructions (NI) | Speedup48.8 | 10 | |
| Continual Pre-training | Natural Instructions (val) | Answer Verification2.391 | 7 | |
| TG task | Natural Instructions task459_matres_static_classification | Correctness69 | 3 | |
| TG task | Natural Instructions task457_matres_conditional_classification | Correctness87 | 3 | |
| TG task | Natural Instructions task108_contextualabusedetection_classification | Correctness75 | 3 | |
| TG task | Natural Instructions task022_cosmosqa_passage_inappropriate_binary | Correctness80 | 3 | |
| TG task | Natural Instructions task021_mctaco_grammatical_logical | Correctness0.5 | 3 | |
| Cross-task Generalization | Natural Instructions (test) | Answerability Classification3.076 | 3 |
Showing 11 of 11 rows