Researchers from Google DeepMind, Carnegie Mellon University, Columbia University and MIT have achieved a further small improvement in the known upper bound on the matrix multiplication exponent, omega. Their approach combined modern GPU-based optimization techniques with AlphaEvolve, a general-purpose LLM-based system Google uses to generate and evolve code and algorithmic ideas.
What they did
The team made the combination loss analysis—an inherently non-convex optimization problem—differentiable and applied gradient-descent-style optimization. That step alone improved the previous state-of-the-art bound by approximately 0.97 × 10^−4. They then employed AlphaEvolve to refine their optimization algorithm, which increased the total improvement over the prior SOTA to about 1.62 × 10^−4.
How AlphaEvolve was used
AlphaEvolve was allowed to modify the optimization program; each modified program was executed (taking roughly five hours on a single GPU) to produce a bound on omega. AlphaEvolve then evolved the code with the objective of minimizing omega. The researchers report better results when using AlphaEvolve’s "evolving constructions" feature, where an optimization algorithm in each generation starts from the best solution found by its parent algorithm.
Scientific significance and limitations
Although some forms of matrix multiplication appear in AI training workloads, the specific theoretical variant studied here is not the same as the practical types commonly used in model training. The work is therefore primarily a proof point that contemporary AI tools can assist researchers on theoretical frontiers: the team converted a problem into a differentiable form, ran it on GPUs, and then handed results to AlphaEvolve to further improve upon the solution.
The authors note: "While further modest improvements may be obtained in this manner, achieving larger improvements to omega likely requires new mathematical ideas and is an exciting area of research." In other words, incremental gains seem feasible with this pipeline, but substantial breakthroughs will probably need fresh mathematical insights.
Reference
The technical details are available in the arXiv paper titled "Improving the matrix multiplication exponent with modern optimization and AlphaEvolve."
