AWS and Google introduce new PyTorch tools

AWS and Google have each announced new tools to run PyTorch directly on their custom chips. AWS introduced TorchNeuron for its Trainium chips, while Google launched TorchTPU for its TPU chips, both aiming to reduce reliance on NVIDIA's hardware.

PyTorch and its role in AI development

PyTorch is a widely used library for building and training artificial intelligence models. It previously required an intermediate layer, such as CUDA from NVIDIA, to translate operations into instructions for hardware like GPUs.

Shift from XLA to TorchTPU

Google's new TorchTPU replaces XLA, a previous tool for running PyTorch on TPUs. XLA has been marked as deprecated and will be archived by November 15, signaling a rapid transition to the new framework.

TorchTPU features and open source

TorchTPU is available as open source under the Apache 2.0 license and offers two execution modes. One allows line-by-line debugging, while the other compiles the entire program before execution, offering flexibility for developers.

Compatibility with existing tools

Google emphasizes that TorchTPU maintains compatibility with existing AI tools and frameworks. Projects like TorchTitan, VeRL, and vLLM can run on TPUs without major changes to their code or infrastructure.

Meta's involvement and strategic goals

Meta, which originally created PyTorch, is directly involved in the TorchTPU project. This aligns with its efforts to reduce dependence on NVIDIA, as Meta is one of NVIDIA's largest customers and has tested TPU leasing with Google Cloud.

Broader implications for AI infrastructure

Both AWS and Google aim to reduce the AI ecosystem's reliance on NVIDIA by making their chips more accessible. While neither company openly challenges NVIDIA, the move reflects a growing desire for alternative AI hardware solutions.