Original sourceGoogle Developers Blog
Summary
Google announces native PyTorch backends for vLLM and SGLang on TPU, extending TorchTPU to run these two inference engines directly on TPU. The new vLLM-TorchTPU and SGLang-TorchTPU offer production-grade serving features and optimized Pallas kernels, supporting workloads including agentic multi-tu…
Key points
- Teams evaluating inference stacks gain another PyTorch-based TPU option with public benchmarks.
- Native support for TPU by mainstream open-source inference engines makes non-NVIDIA inference cost options more comparable.
- Teams already using vLLM or SGLang can evaluate TPU deployment without major code changes and compare per-token costs using public benchmarks.
Editorial note
This page is Code & Chain's editorial summary of public sources. It may be prepared with AI assistance and published through an automated workflow. Refer to the original sources; this content is not investment, legal, or tax advice.