A few years ago, Caltech professor Anima Anandkumar set out to build the first open-source AI-driven weather model. She encountered skepticism from domain experts who pointed out that weather is chaotic, physics-based simulations have been refined for decades and require supercomputers, and that sufficient open data were lacking. Despite these reservations, Anandkumar and her team proceeded, and within a year they produced FourCastNet — a predictive model competitive with leading physics-based simulations.
Thanks to FourCastNet and subsequent developments, it is now possible for users to generate accurate short-term weather forecasts on consumer-grade GPUs. This demonstrates that approaches embedding physical structure can rival traditional, computationally intensive methods.
Why are large, chaotic physical systems hard to model?
Anandkumar’s work highlights that systems such as the atmosphere, fusion plasmas, and fluid or heat flows are inherently large-scale, chaotic, and multiscale. The AI community has historically underinvested in these problems, partly because open datasets in many of these domains often contain only tens or hundreds of thousands of examples — far less than the token-hungry transformers require.
A deeper challenge is the resolution-driven context length: if each spatial dimension has even a few hundred grid points — a typical industrial scale — context length quickly reaches hundreds of billions up to a trillion. Anandkumar has said this implies that "forget ever having a transformer for anything of this scale; all of the world’s compute will not be enough." Therefore, rapid scaling alone is not the solution; progress comes from incorporating structure and inductive biases into models.
The mathematical advance: Neural Operators
To address these systems, Anandkumar pioneered the use of Neural Operators, a significant theoretical development in AI. Neural Operators allow layers to represent mappings between functions rather than just finite-dimensional vectors. Instead of modeling a grid directly, these methods model a function that evolves over multiple scales.
This framework enables the incorporation of physical priors. Consider global weather forecasting: the Earth is (approximately) a sphere, so working in the right basis — the Spherical Harmonics — helps. Running a weather model on a naive grid can become unstable quickly; switching to the natural basis stabilizes the simulation, allowing forecasts to roll out months ahead rather than merely days. Anandkumar’s Fourier Neural Operator learns in the frequency domain, and its spherical variant powers FourCastNet 3, which models weather across the globe and maintains stability far into the future.
The physical world can be more forgiving than expected
Anandkumar has applied Neural Operators to other physical domains and made a striking observation: the physical world is sometimes more forgiving than anticipated. In fusion research, a few thousand samples can suffice to predict plasma disruptions, and such models can run up to a million times faster than traditional simulations. These results indicate that while scaling behaves differently than in language models, effective strategies exist through architectural and prior-driven design.
Anandkumar’s long-term aim remains to build a "foundation model for physics" — a model spanning many phenomena capable of both simulation and design. The route to that goal is through embedding the structure that the physical world already offers, not by waiting for datasets of a size that will never exist.
"All of the things that work with deep learning, let’s take them, but make them a bit more principled," she has said.
Extensions: formal verification and policy engagement
Her work extends beyond numerical simulation. Anandkumar has developed methods that integrate neural networks with automated proof techniques. TorchLean is a framework that allows PyTorch-style networks to be written inside the Lean proof assistant and formally verified. This is an important step toward proving bounds on neural networks, which is crucial when one needs guarantees — for example, if a neural controller would be part of a fusion reactor’s control loop.
Anandkumar was recently appointed to the United Nations Scientific Advisory Board. In this role she aims to bring evidence-based perspectives into policymaking and to demonstrate how AI applied in scientific domains can improve people’s lives globally.
What this means going forward
The lesson from Neural Operators and the FourCastNet family is that modeling large physical systems does not solely depend on data-scale-driven methods. Where data are limited and required resolution implies enormous context lengths, incorporating physical structure and inductive biases into model architectures enables practical progress. Weather forecasting is only the beginning; the approach has applications in fusion, materials science, and other multiscale physical problems.
Anandkumar’s work — and the Latent.Space podcast episode that explored it — ties together elegant mathematics, classical harmonic analysis, modern AI developments, and practical techniques for modeling the physical world.
