SciFive: a text-to-text transformer model for biomedical literature
About
In this report, we introduce SciFive, a domain-specific T5 model that has been pre-trained on large biomedical corpora. Our model outperforms the current SOTA methods (i.e. BERT, BioBERT, Base T5) on tasks in named entity relation, relation extraction, natural language inference, and question-answering. We show that text-generation methods have significant potential in a broad array of biomedical NLP tasks, particularly those requiring longer, more complex outputs. Our results support the exploration of more difficult text generation tasks and the development of new methods in this area
Long N. Phan, James T. Anibal, Hieu Tran, Shaurya Chanana, Erol Bahadroglu, Alec Peltekian, Gr\'egoire Altan-Bonnet• 2021
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Natural Language Inference | MedNLI (test) | Accuracy86.57 | 89 | |
| Named Entity Recognition | NCBI-disease (test) | Precision88.82 | 40 | |
| Document Classification | HoC (test) | F1 (sample average)0.873 | 20 | |
| Named Entity Recognition | BC5CDR-Disease | Total F187.2 | 18 | |
| Named Entity Recognition | BC5CDR chem | Total F194.2 | 18 | |
| Question Answering | BioASQ 4b (test) | Lenient Accuracy0.8831 | 5 | |
| Question Answering | BioASQ 5b (test) | Lenient Accuracy88.28 | 5 | |
| Question Answering | BioASQ 6b (test) | Lenient Accuracy79.08 | 5 | |
| Relation Extraction | CPI (test) | Macro F188.9 | 3 |
Showing 9 of 9 rows