Aster: Autonomous Scientific Discovery over 20x Faster Than Existing Methods
About
We introduce Aster, an AI agent for autonomous scientific discovery capable of operating over 20 times faster than existing frameworks. Given a task, an initial program, and a script to evaluate the performance of the program, Aster iteratively improves the program, often leading to new state-of-the-art performances. Aster's significant reduction in the number of iterations required for novel discovery expands the domain of tractable problems to include tasks with long evaluation durations, such as multi-hour machine learning training runs. We applied Aster to problems in mathematics, GPU kernel engineering, biology, neuroscience, and language model training. More specifically: the Erdos minimum overlap problem, optimizing the TriMul kernel, a single-cell analysis denoising problem, training a neural activity prediction model to perform well on ZAPBench, and the NanoGPT Speedrun Competition. Aster attains SOTA results in every task, except for ZAPBench, where it matches the performance of the best human solution with less than 1/190th of the compute. Aster is accessible via a web interface and API at asterlab.ai.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Single-cell Denoising | PBMC OpenProblems benchmark | Mean Score0.711 | 11 | |
| Mathematics | Erdős’ minimum overlap problem | Overlap Score38.0874 | 10 | |
| GPU kernel engineering | TriMul kernel | TriMul Latency (µs)1.11e+3 | 7 | |
| Biology | single-cell analysis denoising | Denoise Score71.1 | 3 | |
| Training Speed Optimization | NanoGPT Speedrun (record) | Training Speedup1.6 | 3 | |
| Language model training | FineWeb (val) | Training Time (s)95.2 | 2 | |
| Machine Learning | NanoGPT Speedrun Competition | NanoGPT Score95.2 | 2 |