SuperNI
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Multi-Task Instruct-Tuning | SuperNI (test) | ROUGE Score57.35 | 72 | |
| Continual Learning | SuperNI (Order 2) | AP49.26 | 20 | |
| Continual Learning | SuperNI (Order 1) | AP49.48 | 20 | |
| Instruction Following | SuperNI Hold-In v1.0 (test) | ROUGE-L Score62.47 | 18 | |
| Instruction Following | SuperNI Hold-Out v1.0 (test) | ROUGE-L53.53 | 18 | |
| Continual Learning | SuperNI Benchmark | Average Score50.9 | 14 | |
| Continual Learning | SuperNI Large Number of Tasks (test) | Average Performance82.1 | 13 | |
| Continual Learning | SuperNI Standard CL Benchmark (test) | Average Performance81.9 | 13 | |
| Continual Learning | SuperNI | AP56.95 | 13 | |
| Continual Learning | SuperNI (test) | AP56.23 | 13 | |
| Instruction Following | SuperNI Unseen | ROUGE-L37.97 | 9 | |
| Instruction Following | SuperNI In-domain | ROUGE-L52.26 | 9 | |
| Continual Learning | SuperNI | FWT (O1)1.87 | 9 | |
| Unimodal Language Generation | SuperNI Order 2 | AP51.54 | 5 | |
| Unimodal Language Generation | SuperNI (Order 1) | AP50.84 | 5 | |
| Continual Learning | SuperNI (unseen tasks) | Dialog Score11.56 | 4 | |
| Continual Learning | SuperNI Benchmark | Metric- | 0 |
Showing 17 of 17 rows