Rubens de Almeida Zimbres

Measuring What the TPU Scaling Playbook Predicts: Tensor Parallelism, Collectives, and a Per Layer Type Optimization in vLLM Serving

Analytical cost models guide sharding decisions for large language model serving on TPUs: tensor parallelism adds two AllReduce collectives per transformer layer, decode steps are bound by streaming the weights from high bandwidth memory, and per chip memory is the weights divide...