Kermut: Composite kernel regression for protein variant effects
About
Reliable prediction of protein variant effects is crucial for both protein optimization and for advancing biological understanding. For practical use in protein engineering, it is important that we can also provide reliable uncertainty estimates for our predictions, and while prediction accuracy has seen much progress in recent years, uncertainty metrics are rarely reported. We here provide a Gaussian process regression model, Kermut, with a novel composite kernel for modeling mutation similarity, which obtains state-of-the-art performance for supervised protein variant effect prediction while also offering estimates of uncertainty through its posterior. An analysis of the quality of the uncertainty estimates demonstrates that our model provides meaningful levels of overall calibration, but that instance-specific uncertainty calibration remains more challenging.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Protein property prediction | 21 protein property datasets 48 data points (Cross-validation) | Spearman Correlation6.65 | 40 | |
| Protein property prediction | 21 protein landscapes Unseen mutations 96 data points | Spearman Correlation6.05 | 31 | |
| Protein property prediction | 21 protein landscapes 128 training points (extrapolation) | Spearman Correlation0.639 | 22 | |
| Protein property prediction | 21 protein landscapes Extrapolation 512 training points | Spearman Correlation0.75 | 22 | |
| Protein property prediction | 21 protein landscapes 1536 train points (cross-val) | Spearman Correlation0.85 | 22 | |
| Protein Variant Effect Prediction | ProteinGym substitution benchmark | Average Spearman Correlation0.633 | 20 | |
| Protein property prediction | 21 protein property datasets 128 data points (Extrapolation) | Spearman Correlation6.81 | 18 | |
| Protein property prediction | 21 protein property datasets 512 data points (Extrapolation) | Spearman Correlation7.24 | 18 | |
| Supervised protein substitution DMS prediction | ProteinGym Contig | Spearman's rho0.657 | 12 | |
| Protein property prediction | Protein Property Datasets (Unseen mutations) | Spearman Correlation0.629 | 9 |