Google Chips Finally Run Inside Meta's Servers

Google has successfully integrated its custom AI chips into Meta’s production environment, a move that represents a major step forward for the company's hardware strategy. This development was announced through a new, PyTorch-native version of the SGLang inference engine, which allows Meta to run its AI workloads directly on Google's tensor processing units, or TPUs. For the first time, these chips are being used in a live production setting by an external customer, signaling a potential shift in the competitive landscape that has long been dominated by Nvidia.

The key to this integration is a software update that removes the need for developers to rewrite their code. Previously, running on TPUs required using Google’s own JAX framework, which many major companies preferred to avoid. The new versions of SGLang and vLLM, another popular inference engine, now support PyTorch natively. This means that teams using the widely adopted PyTorch framework can deploy their models on TPUs with minimal changes to their existing codebases, making the hardware much more accessible to those who did not want to migrate their entire workflow.

Meta’s involvement in this project has been deep and collaborative. Zongwei Zhou, the PyTorch-TPU lead at Meta, noted that the SGLang-TorchTPU system is currently running in production at Meta. He described the achievement as the result of a tight three-way collaboration between Meta, Google, and the team behind SGLang. That team operates out of RadixArk, a startup founded by the creators of the SGLang engine, who worked closely with Google and Meta to build this specific stack. Zhou emphasized that the new toolset provides Meta with a mature and reliable way to serve AI models using their preferred software framework.

This shift is part of a broader strategy for Meta to maintain flexibility in its hardware choices. Jana van Greunen, who leads PyTorch engineering at Meta, explained that the company cannot afford to be locked into a single hardware architecture. By adopting TorchTPU, Meta can now deploy its PyTorch workloads onto Google’s TPUs, giving them another powerful option beyond the GPUs they have historically relied on. Google confirmed that it co-developed this specific TorchTPU integration directly with Meta’s PyTorch team to ensure it met their specific operational needs.

While this is a headline-grabbing moment, it is not the first time Meta has looked at Google’s chips. In the past, the two companies used an older bridge called PyTorch/XLA to connect their systems. Reports from last year indicated that Meta’s research teams were using TPUs through this older method from 2020 to 2023 before those contracts were eventually canceled. The current push by Google appears to be a more focused effort to win back Meta’s confidence, specifically by addressing their reluctance to switch to the JAX framework in the past.

The logistics of how Meta is using these chips remain somewhat private, but industry talks from the past year suggested a phased approach. There were reports of Google Cloud TPU rentals being discussed for 2026, with potential purchases slated for 2027. It is also reported that Meta has since signed a multi-year rental agreement, which likely forms the backbone of the current production deployment. This arrangement allows Meta to test and scale the technology without immediately committing to a massive hardware purchase.

The timing of this announcement is notable, coming just one day after Amazon revealed it was open-sourcing its own native PyTorch backend for its Trainium chips. Both Google and Amazon appear to be making concerted efforts to break Nvidia’s software lock-in, which has long allowed the company to dominate the AI hardware market by tying its GPUs to specific software ecosystems. By making their hardware more compatible with the industry-standard PyTorch framework, both companies are lowering the barrier to entry for other tech giants.

Performance benchmarks suggest these new setups are competitive with the current market leaders. A tuned version of vLLM serving a massive open-weight model reported speeds topping 700 tokens per second for a single user on Google’s seventh-generation Ironwood TPU. This test, conducted by Inferact, a startup founded by vLLM’s creators, used 16 Ironwood chips with a technique called speculative decoding. This setup outperformed a comparable configuration using 16 Nvidia GB200 chips, which reached 452 tokens per second. For smaller batch sizes, the new kernels also showed throughput roughly 1.4 to 2 times higher than the Nvidia baseline.

Looking ahead, the SGLang team plans to expand its support for various models, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk is also working toward providing day-zero support for new open models on TPUs, a goal they aim to achieve by the first quarter of 2027. This would mean that whenever a new popular AI model is released, it would be immediately compatible with Google’s hardware. Google, Meta, and RadixArk are scheduled to share more details about these efforts at the SGLang Summit, taking place in San Francisco on November 12 and 13.