Fadi Arafeh
f30202b705
Accelerate SVE128 SBGEMM/BGEMM
...
This accelerates SBGEMM/BGEMM by extending the existing 8x4 kernel to 8x8 (unrolling N by 8)
Not sure if it's a good idea to delete the previous 8x4 kernel?
Here are the speedups on single core Neoverse-V2 (SVE128) compared to prev state:
Per-shape speedup
M=N=K=64: SBGEMM 1.164x (16.42%), BGEMM 1.133x (13.30%)
M=N=K=128: SBGEMM 1.220x (22.02%), BGEMM 1.186x (18.56%)
M=N=K=256: SBGEMM 1.241x (24.08%), BGEMM 1.235x (23.54%)
M=N=K=512: SBGEMM 1.240x (23.95%), BGEMM 1.227x (22.75%)
M=N=K=1024: SBGEMM 1.251x (25.11%), BGEMM 1.232x (23.23%)
M=N=K=2048: SBGEMM 1.235x (23.47%), BGEMM 1.246x (24.64%)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-03-05 13:50:07 +00:00
Martin Kroeker
69d92490c1
move inclusion of sme_abi header into the conditional section
2026-01-29 22:24:00 +01:00
Martin Kroeker
861b3db733
Reuse ?SUM kernels from ThunderX2T99
2026-01-20 15:42:09 +01:00
Martin Kroeker
71261a7b3f
Trivially derive optimized S/DSUM for existing SASUM/DASUM kernels
2026-01-20 15:38:50 +01:00
Jameson Nash and Claude Opus 4.5
a18a4ee08a
arm64: fix clang ICE on Windows for zdot_thunderx2t99.c
...
Guard .align directive to avoid internal compiler error on
AArch64 Windows with clang.
See: https://github.com/llvm/llvm-project/issues/149547
See: #5076
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-19 15:36:14 +00:00
Martin Kroeker
6de062cfc2
Merge branch 'OpenMathLib:develop' into issue5414
2026-01-11 17:45:11 +01:00
Martin Kroeker
d1de282a4e
Improve the precision of S/CNRM2 by summing in double precision
2026-01-11 13:04:00 +01:00
Martin Kroeker
a9a6edaf17
Adapt for DYNAMIC_ARCH with multiple ...preprocess symbols
2026-01-09 15:29:36 +01:00
Martin Kroeker
2d46f1ec65
Merge branch 'develop' into issue5414
2026-01-09 15:04:06 +01:00
Martin Kroeker
c040d5ed86
Merge pull request #5591 from quic/topic/ssyr2k_direct_sme1
...
Support for SME1 based ssyr2k_direct kernel for cblas_ssyr2k level 3 API
2026-01-08 15:47:38 +01:00
Zhiqing xie
6939a43c3b
Support for SME1 based ssyr2k_direct kernel for cblas_ssyr2k level 3 API
2026-01-08 11:09:04 +08:00
Martin Kroeker
d39b77748f
Make .align conditional on not being on WoA and strip CRLF endings
2025-12-24 20:00:45 +01:00
Martin Kroeker
ac2c66321d
remove special handling of C/ZDOT for LLVM on WoA
2025-12-19 17:04:21 +01:00
Martin Kroeker
cfa28bcf71
Support compilation with LLVM for Windows on Arm
2025-12-19 17:00:47 +01:00
Martin Kroeker
e85efb8d86
remove za from clobber lists
2025-12-03 22:40:02 +01:00
Martin Kroeker
a683287006
rework for dynamic_arch
2025-11-24 22:24:06 +01:00
Martin Kroeker
b185c9a4ce
small fixes for separating sme and dummy parts
2025-11-24 22:22:14 +01:00
Martin Kroeker
8c0b13c41c
Merge branch 'OpenMathLib:develop' into issue5414
2025-11-23 23:12:49 +01:00
Martin Kroeker
ea85b6696f
Merge branch 'OpenMathLib:develop' into issue5414
2025-11-23 10:14:07 +01:00
Abhishek Kumar
a14caf464f
add tt for a64fx dot
...
Signed-off-by: Abhishek Kumar <abhishek.r.kumar@fujitsu.com >
2025-11-20 12:14:17 +05:30
Martin Kroeker
17f2e94260
Merge pull request #5539 from FRosner/arm64-dot-kernel-refactoring
...
Refactoring: ARM64 dot Kernel: don't call num_cpu_avail twice
2025-11-18 23:25:51 +01:00
Frank Rosner
762ed66c72
Refactoring: ARM64 dot Kernel: don't call num_cpu_avail twice
2025-11-18 15:34:51 +01:00
mayeut
39d5e44723
fix: dot_kernel_sve "n" usage & clobber list
2025-11-17 21:53:51 +01:00
Martin Kroeker
f2d010de12
Merge pull request #5512 from quic/topic/ssyrk_direct_sme1
...
Support for SME1 based ssyrk_direct kernel for cblas_ssyrk level 3 API
2025-11-06 14:06:43 -08:00
Martin Kroeker
585e6d0680
Merge pull request #5515 from iha-taisei/feature/ger_unroll
...
Improve single-thread performance of [SD]GER on A64FX and Neoverse V1
2025-10-24 08:17:06 -07:00
Iha, Taisei
cb66aca707
Improve single-thread performance of [SD]GER on A64FX and Neoverse V1
2025-10-22 19:56:14 +09:00
Yichao Yu
3d19d3b60a
Make dummy function have the same linkage as the real one
2025-10-20 12:42:39 -04:00
changjua
43d38d336f
Support for SME1 based ssyrk_direct kernel for cblas_ssyrk level 3 API
2025-10-20 11:35:20 +08:00
Martin Kroeker
9bfc3612f9
Merge branch 'OpenMathLib:develop' into issue5414
2025-10-12 09:18:06 -07:00
Yichao Yu
b94e9b92ad
Fix compilation on ARM
...
Define a dummy function if SME is not supported, following what sgemm does
2025-10-11 20:28:59 -04:00
Martin Kroeker
e40714cabd
Merge pull request #5450 from quic/topic/strmm_direct_sme1
...
Support for SME1 based strmm_direct kernel for cblas_strmm level 3 API
2025-10-11 15:20:19 -07:00
changjua
644ea07ef9
Support for SME1 based strmm_direct kernel for cblas_strmm level 3 API
2025-10-10 10:48:27 +08:00
Martin Kroeker
47a66aef0f
Update limits based on benchmarking the SME code on Apple M4
2025-10-08 14:36:17 +02:00
Martin Kroeker
20f5ed1a94
Merge branch 'OpenMathLib:develop' into issue5414
2025-10-08 05:27:28 -07:00
Chris Sidebottom
578e7dae85
Fix bf16->f32 conversion for NEOVERSEV1 and NEOVERSEN2 targets
...
This fixes an issue originally introduced with the BGEMM kernel.
I've updated the tests to run with `beta=1.0` so as to test loading and
updating from C.
Alongside this, the tests now return sensible return values to reduce
the risk of them being ignored.
Also fixed a bug in `generic/gemv_t.c` resulting in weird outputs for
`bgemv`.
2025-10-06 18:05:58 +00:00
Martin Kroeker
c889558317
Rework for DYNAMIC_ARCH use and use of SGEMM functions by SSYMM
2025-10-02 07:39:24 -07:00
Martin Kroeker
fc516af155
Merge branch 'develop' into issue5414
2025-10-01 14:12:59 -07:00
Rajendra Prasad Matcha
19268471cc
Support for SME1 based ssymm_direct kernel for cblas_ssymm level 3 API
2025-09-30 15:05:33 +05:30
Martin Kroeker
2b5d8c789d
remove debugging printout
2025-08-24 13:50:08 -07:00
Martin Kroeker
b4fc09e9e1
Add registers d8 to d15 to clobber lists as the code does not expressly save them
2025-08-23 14:39:27 -07:00
Martin Kroeker
8e50b8d525
Add d8 to d15 to clobber lists as the code does not expressly save them
2025-08-23 14:36:49 -07:00
h-motoki
855945befb
Implementing SVE in [SD]AXPY Kernels for A64FX and Graviton3E
2025-08-21 20:56:58 +09:00
Martin Kroeker
edaa73fd24
Hide the local 2VLx2VL symbol as static is insufficient for this with gcc
2025-08-20 06:33:28 -07:00
Martin Kroeker
501728a354
adjust register 20 accesses to 21 after moving x18
2025-08-20 06:24:38 -07:00
Martin Kroeker
05dbb54362
Delete misplaced file
2025-08-19 05:12:09 -07:00
Martin Kroeker
0bc19a1335
Update SME kernel details
2025-08-18 14:38:16 -07:00
Martin Kroeker
ca542f319f
Add VORTEXM4
2025-08-18 08:41:38 -07:00
Martin Kroeker
53d3bb50cc
Get symbol name from build system; change b.first to b.mi for AppleClang compatibility
2025-08-18 01:37:50 -07:00
Martin Kroeker
08a00326a4
Build symbol name from build system variables
2025-08-18 01:35:41 -07:00
Martin Kroeker
89898fc499
Add sgemm_direct_performant for switching between direct and regular kernels
2025-08-18 01:31:40 -07:00