Synth
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Online Class-Incremental Learning | Synth-50 | Final Mean Accuracy100 | 26 | |
| Online Class-Incremental Learning | Synth-10 | Final Mean Accuracy100 | 26 | |
| Relative Depth Estimation | Synth4K-5 | Delta Threshold Accuracy (0.5)88.5 | 16 | |
| Relative Depth Estimation | Synth4K-3 | Acc (delta=0.5)82 | 16 | |
| RMFS Order Allocation and Robot Scheduling | Synth-Large synthetic (test) | Objective Value2,160.47 | 14 | |
| RMFS Order Allocation and Robot Scheduling | Synth-Medium (test) | Objective Value1,387.82 | 14 | |
| RMFS Order Allocation and Robot Scheduling | Synth Small (test) | Objective Value811.01 | 14 | |
| Graph Classification | Synth (basis+std, r = 1) OOD | Accuracy61 | 14 | |
| Graph Classification | Synth basis+std r = 1 ID | Accuracy62.5 | 14 | |
| Graph-level classification | SYNTH | MCC1 | 14 | |
| Time Series Forecasting | Synth-12 sigma=0.9 | MSE0.36 | 12 | |
| Time Series Forecasting | Synth sigma=0.7 12-variable synthetic | MSE0.379 | 12 | |
| Time Series Forecasting | Synth-12 sigma=0.5 synthetic | MSE0.391 | 12 | |
| Time Series Forecasting | Synth synthetic sigma=0.3 12-variable | MSE0.378 | 12 | |
| Time Series Forecasting | Synth 12-variable synthetic (sigma=0.1) | MSE0.386 | 12 | |
| Question Answering | SYNTH | Accuracy86.9 | 12 | |
| Online Class Incremental Learning | Synth-50 | Average Forgetting0 | 11 | |
| Time Series Forecasting | Synth factual conditions (test) | MAE0.54 | 9 | |
| Graph Classification | SYNTH | Accuracy100 | 8 | |
| Average Treatment Effect Estimation | Synth | Latency (s)0.02 | 7 | |
| ATE Estimation | Synth | Memory Consumption (MB)468.29 | 7 | |
| Linear Regression | Synth-B | PMSE2.33 | 6 | |
| Negotiation | Synth | IR100 | 5 | |
| Molecular optimization | SYNTH | Yield7,371,212 | 4 | |
| Multi-talker Automatic Speech Recognition | Synth (test) | tcpWER9.6 | 3 |
Showing 25 of 29 rows