Hanyu Yang

Rounding error in FlashAttention: centered, uncentered, and radial

We bound the forward rounding error of FlashAttention-2 for one query and compare it with two-pass attention. The analysis rests on one identity. An error that hits a key's weight in both the numerator and the normalizer costs only the spread of the values around the output. Most...