On Tuesday, Google announced that TorchTPU — its long‑anticipated PyTorch variant — is now available as an open‑source project. With this release Google joins Amazon in attempts to weaken Nvidia’s grip on the hardware commonly used to run AI applications, where many apps are by default written to target Nvidia GPUs.
Why PyTorch matters
PyTorch is a widely used open‑source toolkit for building, training and running AI models. By some estimates about 85% of AI applications use PyTorch. The framework was originally developed by Meta (then Facebook) in 2016, before large language models reshaped the AI landscape. Even then, Nvidia GPUs were the standard for AI research, so several PyTorch features were designed with Nvidia hardware in mind. In 2022 Meta transferred stewardship of PyTorch to the Linux Foundation, a nonprofit that houses many important open‑source projects.
What TorchTPU brings
TorchTPU enables PyTorch applications to run on Google’s Tensor Processing Units (TPUs). Google says existing PyTorch apps can be moved to TPUs and to Google Cloud with a simple configuration change, without rewriting the application code.
Google also reports it worked directly with several ecosystem players — including Meta, Hugging Face, Inferact (maker of vLLM) and RadixArk (maker of SGLang) — to ensure TorchTPU’s features and libraries integrate well with popular AI development tools, similar to original PyTorch.
In addition, Google launched the TPU agent suite to assist developers in migrating apps to TPUs and Google Cloud and to help diagnose and fix errors during migration. Google updated its LiteRT tool, which runs AI on phones, laptops and embedded devices, to improve compatibility with PyTorch‑based apps.
Amazon’s approach: TorchNeuron and broader changes
On Monday, Amazon released its own PyTorch variant, TorchNeuron. TorchNeuron allows applications to be ported to Amazon’s in‑house AI chip Trainium and to AWS without code rewrites.
Amazon said it is also contributing code to the main PyTorch project that decouples various features from Nvidia’s chips, making PyTorch more hardware‑agnostic by default.
What this means in practice
These developments do not imply that AI developers will immediately abandon Nvidia hardware. Choices about hardware and cloud providers depend on performance, cost, existing infrastructure and ecosystem support. Nevertheless, Google and Amazon are clearly taking concrete steps to reduce Nvidia’s advantage by making it easier to run PyTorch models on their chips and clouds.
Both TorchTPU and TorchNeuron represent significant moves toward a more heterogeneous hardware ecosystem for PyTorch, reducing vendor‑specific lock‑in and expanding options for developers.
