The skylakex/cooperlake d/s/c/z asum and c/z sum microkernels peel leading
elements until the input pointer reaches a 64-byte boundary (a scalar loop in
dasum/sasum, a masked header load in the complex variants) before entering an
aligned-load accumulator loop. The peel count depends on the buffer address
mod 64, so the grouping of the sum into accumulators - and therefore the
rounding of the result - depends on where the caller's buffer happens to sit
in memory. The same data at a different address can give a bitwise-different
sum.
That address dependence surfaced as non-reproducibility in OrcaFlex: LAPACK's
dstein scales each inverse-iteration eigenvector by 1/dasum(...) over a heap
array whose alignment varies with allocation history, so eigenvectors from
identical inputs differed run to run in the last bits, which zero-tolerance
regression comparison flags.
Fix by dropping the alignment peel and using unaligned loads throughout, so
the summation order is a function of the length alone. On AVX-512 hardware
unaligned load instructions on addresses that happen to be aligned cost the
same as aligned loads; only genuinely split cache lines pay a small penalty,
negligible for these level-1 reductions.
Per review: A72 was dropped from the default DYNAMIC_CORE list in
#4389 to limit arm64 binary size. Restore the A57 alias for default
DYNAMIC_ARCH; TARGET=CORTEXA72 and DYNAMIC_LIST=CORTEXA72 still get
the dedicated 6x8 kernels.
Co-authored-by: Cursor <cursoragent@cursor.com>
Darwin's DYNAMIC_CORE only builds ARMV8/NEOVERSEN1/ARMV9SME/VORTEXM4,
so an unconditional extern gotoblas_CORTEXA72 left Apple M builds with
an undefined symbol. Mirror the CORTEXA57 Darwin alias.
Co-authored-by: Cursor <cursoragent@cursor.com>
DYNAMIC_ARCH builds CORTEXA72 as a separate kernel and pull
dneg_tcopy from generic/neg_tcopy_$(DGEMM_UNROLL_M).c. Width 6 was
missing (only 1/2/4/8/16 existed), which broke the arm64 Graviton
Cirun and Azure DYNAMIC_ARM64 jobs.
Co-authored-by: Cursor <cursoragent@cursor.com>
Give CORTEXA72 its own param.h block (UNROLL 6x8, P=120 Q=240, R=4096
shared-L2 / R=768 single-core), add it to DYNAMIC_CORE, and stop
aliasing gotoblas_CORTEXA72 to A57 so DYNAMIC_ARCH can select the new
kernels on MIDR 0xd08.
Co-authored-by: Cursor <cursoragent@cursor.com>
TARGET=CORTEXA72 previously reused the A57 8x4 path. Add a dedicated
6x8 NEON ukernel, contiguous MR=6 panel packers (stock gemm_*copy_6 is
4+2), and UNROLL_M=6-aware TRSM kernels so HPL/dtrsm does not corrupt
the heap. DTRMM falls back to generic 2x2 until a matching kernel exists.
Co-authored-by: Cursor <cursoragent@cursor.com>
Martin Kroeker