By Hamza Faisal

For a decade, anyone who wanted to run PyTorch on Google's TPUs had to accept a compromise. The only real route was PyTorch/XLA, with its lazy tensor graphs, mystery recompiles, and debugging that never quite lined up with the code on screen. On Tuesday at PyTorch Conference 2026, Google announced the end of that era: TorchTPU, its native PyTorch stack for TPUs, is now available as open source on GitHub — and it was co-developed with the PyTorch team at Meta.

With TorchTPU, TPUs join GPUs and CPUs as a native option for PyTorch, according to Google's announcement by Mark Lohmeyer and Bill Jia. Existing workloads can move over with minimal adjustments — in most cases, swapping the device call from .to(“cuda”) to .to(“tpu”). Tensors are ordinary PyTorch tensors, with no subclasses or wrapper types, and the standard workflow survives intact: debug in eager mode, then hand the model to torch.compile for production performance.

The stack, and the numbers Google is quoting

The release spans four GitHub repositories under the Apache 2.0 license, including native backends for the two most-used serving engines: vLLM (built with Inferact) and SGLang (with RadixArk). Under the hood, TorchTPU supports roughly 450 ATen operations, stretches across thousands of TPU chips, and uses Pallas kernels plus built-in distributed primitives rather than bolted-on extras. Google reports 52% model FLOPs utilization training the Qwen 3 8B model on up to 8,192 TPU v7x (Ironwood) chips, and says some Cloud customers have already started adopting the backend in production. It is positioned as production-ready — the recommended path for all new workloads, replacing PyTorch/XLA.

Why Meta is the loudest endorser

The most interesting voice in the announcement isn't Google's. Jana van Greunen, who runs PyTorch engineering at Meta, said Meta “cannot be constrained by a single hardware architecture,” and that TorchTPU lets the company push LLM, ads, and recommendation workloads onto Google silicon — while feeding bugs back to both engineering teams. Meta already has SGLang-TorchTPU running in production, according to Google's post. That's a remarkable sentence to type: Meta, one of the largest GPU consumers on earth, is now helping shape the software layer that makes Google's chips a first-class PyTorch target.

A second front against the CUDA moat

TorchTPU lands on the same day DeepSeek open-sourced DeepInfer, its own inference engine — and just one day after Amazon took its own swing with TorchNeuron for Trainium. The pattern is hard to miss: the entire industry has decided Nvidia's real moat was never just the silicon, but the software layer that made leaving expensive. When the framework and the serving engines teams already run treat another accelerator as a native option, the cost of leaving drops. TorchTPU doesn't dismantle the CUDA moat in a day — but it is the most direct hit yet at the foundation it's built on.

By Hamza Faisal