Adaptive and Robust Multi-Task Learning
About
We study the multi-task learning problem that aims to simultaneously analyze multiple datasets collected from different sources and learn one model for each of them. We propose a family of adaptive methods that automatically utilize possible similarities among those tasks while carefully handling their differences. We derive sharp statistical guarantees for the methods and prove their robustness against outlier tasks. Numerical experiments on synthetic and real datasets demonstrate the efficacy of our new methods.
Yaqi Duan, Kaizheng Wang• 2022
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Linear regression | Synthetic Linear Regression (n=50, d=50, K=40, ε=0.2) (train/test) | Average Local Error0.389 | 204 | |
| Linear regression | Synthetic Multi-task Linear Regression (n=d=50, K=40, epsilon=0.2) | Global Error (L2-norm)0.39 | 192 | |
| Multi-task Linear Regression (Local Parameter Estimation) | Synthetic Multi-task Linear Regression (n=d=50, ε=0.2) (test) | Local L2 Error1.173 | 102 | |
| Local prediction | HAR (20% train) | Local Prediction Error3.1 | 102 | |
| Local prediction error | HAR 50% (train) | Error Rate2 | 102 | |
| Local prediction | HAR 60% (train) | Local Prediction Error1.9 | 102 | |
| Linear regression | Synthetic Multi-task Dataset d=50, K=20, epsilon=0.2 (test) | Local Error1.091 | 102 | |
| Multi-task Logistic Regression | Synthetic Multi-task Logistic Regression (n=d=50, epsilon=0.2) | Local Error2.695 | 102 | |
| Logistic Regression | Logistic regression d=50, K=20, epsilon=0.2 varying per-task sample size n (test) | Local Error2.726 | 102 | |
| Linear regression | Linear Regression (synthetic) | Local Error0.984 | 101 |
Showing 10 of 14 rows