Amazon Web Services released TorchNeuron on Monday. The tool allows AI startups to move applications onto Amazon’s Trainium chip with a single line of configuration change.
The software hurdle
TorchNeuron integrates with PyTorch, the open-source framework used to build and run most AI models. Amazon claims this switch requires no code rewrite. About 85% of AI apps are built with PyTorch. Facebook originally created the toolkit, though it has been maintained by the Linux Foundation since 2022.
Nvidia spent years ensuring its CUDA software supports PyTorch apps. Consequently, Nvidia hardware became the default choice for the framework. TorchNeuron aims to break that dominance by offering the same ease of use for Amazon’s silicon.
Developers familiar with PyTorch can already port apps to AMD GPUs or Apple chips using similar configuration tweaks. Amazon argues that porting to Trainium offers a distinct advantage: access to the entire AWS cloud ecosystem rather than isolated hardware providers.
Cloud lock-in versus hardware choice
Colin Brace, vice-president of technology software at AWS Annapurna Labs, told TechCrunch via email that startups are currently building on AWS while betting on single-purpose compute providers. He noted that competitors like Coreweave, Nebius, and Firmus are tied to Nvidia hardware. On AWS, the infrastructure to move from prototype to production is already in place.
Brace said the Trainium chip handles both training and inference. This tight integration with cloud infrastructure is intended to save money, even if AWS does not offer the lowest per-token price.
“Trainium’s value is price-performance plus the fact that it’s one platform for both training and inference, which improves the economics across a full workload,” Brace said.
Deepgram, an AI voice app provider competing with ElevenLabs, moved to Trainium using native PyTorch support. The company reports processing 70% more audio per dollar at production-grade accuracy.
Changing the framework
Amazon is not content with simply providing an easy migration path. The company plans to contribute code changes directly to the main PyTorch project. The goal is to remove code that ties the framework to specific chips.
“The goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users,” the company wrote in its blog post.
This approach could complicate the developer experience. PyTorch is designed to hide the difficulties of getting code to run on specific hardware. Pushing Nvidia out of the framework entirely may result in more configuration headaches rather than more choice.
Regardless of the outcome, uncoupling PyTorch from Nvidia could have significant ramifications for the GPU giant. As Brace noted, the real story is removing the lock-in that prevented developers from having a real choice.
