Files
David Heffernan d793be85b4 kernel/x86_64: make AVX-512 asum/sum kernels independent of buffer alignment
The skylakex/cooperlake d/s/c/z asum and c/z sum microkernels peel leading
elements until the input pointer reaches a 64-byte boundary (a scalar loop in
dasum/sasum, a masked header load in the complex variants) before entering an
aligned-load accumulator loop. The peel count depends on the buffer address
mod 64, so the grouping of the sum into accumulators - and therefore the
rounding of the result - depends on where the caller's buffer happens to sit
in memory. The same data at a different address can give a bitwise-different
sum.

That address dependence surfaced as non-reproducibility in OrcaFlex: LAPACK's
dstein scales each inverse-iteration eigenvector by 1/dasum(...) over a heap
array whose alignment varies with allocation history, so eigenvectors from
identical inputs differed run to run in the last bits, which zero-tolerance
regression comparison flags.

Fix by dropping the alignment peel and using unaligned loads throughout, so
the summation order is a function of the length alone. On AVX-512 hardware
unaligned load instructions on addresses that happen to be aligned cost the
same as aligned loads; only genuinely split cache lines pay a small penalty,
negligible for these level-1 reductions.
2026-08-13 13:57:49 +01:00
..
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2023-11-17 23:53:56 +00:00
2020-04-13 14:58:52 +02:00
2025-05-23 05:52:48 -07:00
2013-06-21 16:06:51 +02:00
2022-09-15 09:19:19 +02:00
2015-04-28 16:58:11 +02:00
2020-10-12 18:15:01 +03:00
2026-03-18 17:24:02 +09:00
2024-08-04 00:14:31 +02:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2016-03-07 09:39:34 +01:00
2026-04-14 21:32:36 +02:00
2022-09-15 09:19:19 +02:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2024-07-20 18:27:15 +02:00
2014-09-14 11:00:53 +02:00
2014-09-14 11:00:53 +02:00
2024-08-04 00:13:31 +02:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00
2024-07-20 18:27:15 +02:00
2024-07-20 17:42:31 +02:00
2025-05-23 05:52:48 -07:00
2020-10-12 18:15:01 +03:00
2020-10-12 18:15:01 +03:00