Multiple Choice Question Answering on MMLU 1.0 (test)
88.7Accuracy (Clinical knowledge)
Med-PaLM 2
Updated 3d ago
Evaluation Results
| Method | Links | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
2023.05 | 88.7 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 92 | 84.4 | 92.3 | 95.8 | 83.2 | |
2023.05 | 88.7 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 92 | 84.4 | 95.2 | 95.8 | 83.2 | |
2023.05 | 88.7 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 97 | 85.2 | 93.8 | 97.2 | 80.9 | |
2023.05 | 86.4 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 92 | 80 | 93.8 | 95.1 | 76.9 | |
2023.05 | 80.4 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 75 | 63.7 | 83.8 | 88.9 | 76.3 | |
2024.02 | 74.71 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 74 | 65.92 | 72.79 | 72.91 | 64.73 | |
2024.02 | 63.1 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 63.3 | 49.9 | 57.4 | 63.4 | 57.8 | |
2024.02 | 62.8 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 62.7 | 46.9 | 57 | 60.6 | 56.3 | |
2024.02 | 62.3 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 61.3 | 48.1 | 55.8 | 57.2 | 56.5 | |
2024.02 | 61.3 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 61 | 49.9 | 55.3 | 64.4 | 53.9 | |
2024.02 | 60.9 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 61.7 | 49.6 | 55.1 | 56.9 | 55.5 | |
2024.02 | 57 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 56.7 | 46.9 | 51 | 58.6 | 50.1 | |
2024.02 | 50.1 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 52 | 46.2 | 47.3 | 47.9 | 45.5 | |
2024.02 | 49.1 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 49 | 48.4 | 63.8 | 47.2 | 43.5 | |
2024.02 | 37.9 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 47 | 39.3 | 34.2 | 42.6 | 30.4 | |
2024.02 | 25.3 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 26 | 31.9 | 16.9 | 28 | 24.9 | |
2021.12 | - | - | 49.4 | 60 | 71.3 | 47.8 | 54 | 33.8 | 50 | 41.1 | 50.9 | 43 | 76.8 | 83.9 | 65.1 | 66.4 | 81.8 | 67.2 | 68.1 | 71.8 | 64.9 | 84.1 | 81 | - | - | - | - | - | - | - | - | - | |
2021.12 | - | - | 36.6 | 49.6 | 48.2 | 33.2 | 39 | 27.7 | 30.9 | 31.3 | 21.4 | 33.2 | 57.3 | 57.5 | 40.3 | 41.9 | 60.6 | 54.1 | 44.7 | 48 | 51.4 | 53.1 | 68.9 | - | - | - | - | - | - | - | - | - | |
| - | - | 44.1 | 48.2 | 52.8 | 36.1 | 49.8 | 41 | 34 | 33.3 | - | 30.2 | 65.5 | 72.3 | 47.2 | 53 | 71.7 | 49.6 | 48.3 | 58.5 | 49.3 | 64.9 | 64.4 | - | - | - | - | - | - | - | - | - | ||
2021.12 | - | - | 95 | 90 | 95 | 90 | 95 | 90 | 90 | 85 | - | 90 | 90 | 95 | 90 | 90 | 95 | 90 | 90 | 90 | 90 | 95 | 90 | - | - | - | - | - | - | - | - | - |