On Monday, Amazon Web Services (AWS) released a tool called TorchNeuron that lets PyTorch-based AI applications run on Amazon’s Trainium chip with a single change to configuration, according to the company. AWS says this requires no code rewrite.
Why this matters
PyTorch is a free, open-source toolkit developers use to build, train, and run AI models; AWS estimates roughly 85% of AI applications are written with PyTorch. Facebook originally developed PyTorch, and since 2022 it has been part of the Linux Foundation.
Nvidia has spent years making sure its CUDA software stack supports PyTorch workloads smoothly, which has entrenched Nvidia hardware in many AI toolchains. TorchNeuron aims to reduce that software advantage by making it straightforward to target Trainium instead.
How TorchNeuron works and AWS’s argument
TorchNeuron is adapted from PyTorch, so existing PyTorch users can adopt it without learning a new framework. AWS argues that moving to Trainium not only is easy but also gives startups access to the broader AWS cloud, avoiding dependence on specialized compute providers.
Colin Brace, Vice President of Technology Software at AWS Annapurna Labs (the unit that builds Amazon’s custom chips), told TechCrunch via email: “You’re building on AWS. You’re not betting on a single-purpose compute provider and scrambling to build everything else around it.” He added: “On AWS, the infrastructure to go from prototype to production is already there, allowing a founder to move fast and scale when they need to.”
AWS also notes that Trainium handles both training and inference on the same chip and is tightly integrated with the cloud services startups usually need. The company says this unified platform improves price-performance across a full workload even if AWS does not offer the lowest per-token price.
As an example, Brace pointed to Deepgram, an AI voice application provider: according to AWS, Deepgram “moved to Trainium using native PyTorch, support and they’re processing 70% more audio per dollar at production-grade accuracy.” Deepgram competes with services such as ElevenLabs.
Contributing changes upstream to PyTorch
Beyond providing an on-ramp for Trainium, AWS says it will contribute code changes upstream to the main PyTorch project to remove parts of the framework that tie it to specific chips. In its announcement, AWS wrote that the goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users.
Implications and open questions
If AWS succeeds in decoupling PyTorch from Nvidia-specific paths, developers could gain more choices and vendors other than Nvidia could compete more fairly. However, PyTorch’s purpose includes hiding hardware complexity from developers, so it remains to be seen whether this effort will reduce lock-in and lower costs as promised or introduce additional configuration burdens.
Either way, loosening Nvidia’s central position in PyTorch could have broader consequences for the GPU market beyond competition among chipmakers. As Brace put it: “The real story is removing the lock-in that kept developers from having a real choice at all.”
