Greek versus English Degradation Under Consumer GGUF Quantization

Abstract

Local inference of Large Language Models (LLMs) depends on quantization to trade accuracy for efficiency on consumer hardware. This trade-off is usually measured in English, raising the question of whether speakers of other languages (like Greek) suffer more with the same file — and if so, by how much, and with what mechanism of distortion. No prior work has characterized the consumer GGUF k-quant ladder with sub-4B models and distributional metrics. I evaluate Greek and English across five k-quant levels (Q6_K to Q2_K) on base Llama 3.2 3B and 1B, measuring mean and median per-token KL divergence and top-1 token agreement from a Q8_0 reference over a parallel FLORES+ corpus. I find that the mean Greek-English KL divergence ratio never falls below 1.35 on either model; damage migrates from tail to typical tokens as precision decreases; and the practical floor of usage is Q4_K_M on 3B and Q5_K_M on 1B, while Q2_K is unusable.