Jan Franciszek Laskowski

Does Lower Precision Always Save Energy? Benchmarking Energy–Performance Trade-Offs in Weight-Quantized Large Language Model Inference on a GPU

Large language model (LLM) inference is an increasing component of AI energy demand, while the energy effects of weight quantization depend on workload phase and implementation. We benchmarked four LLM families—Gemma 4 E4B IT, Qwen3.5 4B, Nemotron 3 Nano 4B, and Ministral 3 3B In...