Corrected the register constraints for the PowerPC dcbt (Data Cache Block
Touch) instruction in Power10 kernel implementations. The dcbt instruction
has special behavior where if the first operand (RA) is r0, it uses the
value 0 instead of the register contents. Therefore, RA must use the "b"
constraint (any GPR except r0), while RB can use "r" (any GPR including r0).
Changes:
- Changed first operand constraint from "r" to "b" to exclude r0
- Changed second operand constraint from "b" to "r" for flexibility
This ensures correct prefetch behavior and compliance with PowerPC ISA
specifications, preventing potential issues where r0 might be incorrectly
used as the base address register.
Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>
Replace normal vector pair pointer dereferences with the optimized
__builtin_vsx_lxvp builtin across DGEMM, ZGEMM, and DGEMV kernels.
Also done some identation corrections in dgemm_kernel_power10.c.
This is done as part of POWER code cleanup and may not have any
performance impact.
Signed-off-by: Amrita H S <amritahs@linux.vnet.ibm.com>
benchmark/sgemm.goto before:
M= 200, N= 200, K= 200 : 9262.97 MFlops 0.001727 sec
after:
M= 200, N= 200, K= 200 : 30223.64 MFlops 0.000529 sec
Conveniently the registers are already allocated suitably for vector
operation, so the conversion from vfpv3 was rather straightforward.
Prefetching was left out because it doesn't help Cortex-A76,
only hurts it slightly.
According to ARM AAPCS (Procedure Call Standard) 5.1.2.1, only registers
s16-s31 must be preserved across subroutine calls; registers s0-s15
do not need to be preserved.
continuous build / neoverse_build (push) Canceled after 0s
harmonyos / build (push) Canceled after 0s
loongarch64 qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=GENERIC, DYNAMIC_ARCH, loongarch64-linux-gnu) (push) Canceled after 0s
loongarch64 qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LA264, LA264, loongarch64-linux-gnu) (push) Canceled after 0s
loongarch64 qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LA464, LA464, loongarch64-linux-gnu) (push) Canceled after 0s
loongarch64 qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LA64_GENERIC, LA64_GENERIC, loongarch64-linux-gnu) (push) Canceled after 0s
loongarch64 qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LOONGSON2K1000, LOONGSON2K1000, loongarch64-linux-gnu) (push) Canceled after 0s
loongarch64 qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LOONGSON3R5, LOONGSON3R5, loongarch64-linux-gnu) (push) Canceled after 0s
loongarch64 qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LOONGSONGENERIC, LOONGSONGENERIC, loongarch64-linux-gnu) (push) Canceled after 0s
loongarch64 clang qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=GENERIC, DYNAMIC_ARCH) (push) Canceled after 0s
loongarch64 clang qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LA264, LA264) (push) Canceled after 0s
loongarch64 clang qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LA464, LA464) (push) Canceled after 0s
loongarch64 clang qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LA64_GENERIC, LA64_GENERIC) (push) Canceled after 0s
loongarch64 clang qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LOONGSON2K1000, LOONGSON2K1000) (push) Canceled after 0s
loongarch64 clang qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LOONGSON3R5, LOONGSON3R5) (push) Canceled after 0s
loongarch64 clang qemu test / TEST (NO_SHARED=1 DYNAMIC_ARCH=1 TARGET=LOONGSONGENERIC, LOONGSONGENERIC) (push) Canceled after 0s
mips64 qemu test / TEST (NO_SHARED=1 TARGET=I6400, I6400, mipsisa64r6el-linux-gnuabi64) (push) Canceled after 0s
mips64 qemu test / TEST (NO_SHARED=1 TARGET=I6500, I6500, mipsisa64r6el-linux-gnuabi64) (push) Canceled after 0s
mips64 qemu test / TEST (NO_SHARED=1 TARGET=MIPS64_GENERIC, MIPS64_GENERIC, mips64el-linux-gnuabi64) (push) Canceled after 0s
mips64 qemu test / TEST (NO_SHARED=1 TARGET=P6600, P6600, mipsisa64r6el-linux-gnuabi64) (push) Canceled after 0s
mips64 qemu test / TEST (NO_SHARED=1 TARGET=SICORTEX, SICORTEX, mips64el-linux-gnuabi64) (push) Canceled after 0s
riscv64 zvl256b qemu test / TEST (TARGET=RISCV64_GENERIC BINARY=64 ARCH=riscv64 DYNAMIC_ARCH=1, rv64,g=true,c=true,v=true,vext_spec=v1.0,vlen=256,elen=64, DYNAMIC_ARCH=1) (push) Canceled after 0s
riscv64 zvl256b qemu test / TEST (TARGET=RISCV64_ZVL128B BINARY=64 ARCH=riscv64, rv64,g=true,c=true,v=true,vext_spec=v1.0,vlen=128,elen=64, RISCV64_ZVL128B) (push) Canceled after 0s
riscv64 zvl256b qemu test / TEST (TARGET=RISCV64_ZVL256B BINARY=64 ARCH=riscv64 BUILD_BFLOAT16=1 BUILD_HFLOAT16=1, rv64,g=true,c=true,v=true,vext_spec=v1.0,vlen=256,elen=64,zfh=true,zvfh=true,zvfbfwma=true, RISCV64_ZVL256B) (push) Canceled after 0s
This change fixes a regression in SBGEMM where C is assumed to be BF16,
and so unconditionally casts the output to FP32 resulting in incorrect
outputs when beta=1.
zhir