A new benchmark called FrontierML examines how effectively artificial intelligence agents can turn a financial budget into machine learning performance. Researchers tested the systems on 21 complex tasks and found that stronger agents reliably improved their results when they could spend more on computing resources and data.
Tasks and budgets
The long-horizon tasks were inspired by real work at leading AI labs, newer labs and academic institutions. They cover six areas: large language model training, efficiency, vision and audio, computer use, applied natural language processing, and math and games.
Each agent started with the same setup: a CPU-based sandbox available for 12 hours, a budget of $10, $50 or $200 per task, a few validation samples and a task description. Agents could choose architectures and data sources, implement training, provision computing resources, evaluate saved model checkpoints and optimize inference.
Their solutions varied considerably. On a task involving screenshot generation, the top-scoring agent fine-tuned a diffusion model using LoRA. Other high-scoring approaches involved a large language model writing a drawing program whose output was then rendered. In 28% of the runs studied, agents added newly trained components to pretrained models or trained an entirely new model.
Results and verification
Agents submitted an inference server to serve their model on an air-gapped H100 machine. To discourage overfitting, they received only limited validation samples; evaluation took place on a large held-out test set, not visible to the agents, whose distribution differed from that of the validation data.
The researchers measured the value of additional funding through the change in mean score across the 21 tasks. A human researcher used as a baseline achieved a 56-point improvement with a $10 budget and a 66.1-point improvement with $200. The best-performing system, Opus 5.5, improved by 45.4 and 58.8 points, respectively. The highest-ranked open-weight model, Kimi K3.1 Pro, gained 25.5 points with $10 and 29.6 points with $200.
The results indicate that advanced AI agents can independently organize an increasing number of machine learning research subtasks and use available resources productively. The benchmark also points to a possible future in which AI systems subcontract parts of complex work to other systems or services.



