Commit Graph
369 Commits
Author SHA1 Message Date
Abhishek Kumar a14caf464f add tt for a64fx dot
Signed-off-by: Abhishek Kumar <abhishek.r.kumar@fujitsu.com>
2025-11-20 12:14:17 +05:30
Martin Kroeker 17f2e94260 Merge pull request #5539 from FRosner/arm64-dot-kernel-refactoring
Refactoring: ARM64 dot Kernel: don't call num_cpu_avail twice
2025-11-18 23:25:51 +01:00
Frank Rosner 762ed66c72 Refactoring: ARM64 dot Kernel: don't call num_cpu_avail twice 2025-11-18 15:34:51 +01:00
mayeut 39d5e44723 fix: dot_kernel_sve "n" usage & clobber list 2025-11-17 21:53:51 +01:00
Martin Kroeker f2d010de12 Merge pull request #5512 from quic/topic/ssyrk_direct_sme1
Support for SME1 based ssyrk_direct kernel for cblas_ssyrk level 3 API
2025-11-06 14:06:43 -08:00
Martin Kroeker 585e6d0680 Merge pull request #5515 from iha-taisei/feature/ger_unroll
Improve single-thread performance of [SD]GER on A64FX and Neoverse V1
2025-10-24 08:17:06 -07:00
Iha, Taisei cb66aca707 Improve single-thread performance of [SD]GER on A64FX and Neoverse V1 2025-10-22 19:56:14 +09:00
Yichao Yu 3d19d3b60a Make dummy function have the same linkage as the real one 2025-10-20 12:42:39 -04:00
changjua 43d38d336f Support for SME1 based ssyrk_direct kernel for cblas_ssyrk level 3 API 2025-10-20 11:35:20 +08:00
Yichao Yu b94e9b92ad Fix compilation on ARM
Define a dummy function if SME is not supported, following what sgemm does
2025-10-11 20:28:59 -04:00
Martin Kroeker e40714cabd Merge pull request #5450 from quic/topic/strmm_direct_sme1
Support for SME1 based strmm_direct kernel for cblas_strmm level 3 API
2025-10-11 15:20:19 -07:00
changjua 644ea07ef9 Support for SME1 based strmm_direct kernel for cblas_strmm level 3 API 2025-10-10 10:48:27 +08:00
Chris Sidebottom 578e7dae85 Fix bf16->f32 conversion for NEOVERSEV1 and NEOVERSEN2 targets
This fixes an issue originally introduced with the BGEMM kernel.

I've updated the tests to run with `beta=1.0` so as to test loading and
updating from C.

Alongside this, the tests now return sensible return values to reduce
the risk of them being ignored.

Also fixed a bug in `generic/gemv_t.c` resulting in weird outputs for
`bgemv`.
2025-10-06 18:05:58 +00:00
Rajendra Prasad Matcha 19268471cc Support for SME1 based ssymm_direct kernel for cblas_ssymm level 3 API 2025-09-30 15:05:33 +05:30
h-motoki 855945befb Implementing SVE in [SD]AXPY Kernels for A64FX and Graviton3E 2025-08-21 20:56:58 +09:00
Martin Kroeker f3b2a15fad Merge pull request #5420 from yuanjia111/develop
Move the value assignment of vector x in gemv_n_sve.c to the outermos…
2025-08-16 12:06:53 -07:00
yuanjia 803e8d4838 Move the value assignment of vector x in gemv_n_sve.c to the outermost loop to reduce the repeated data retrieval.
1.Verify correctness using BLAS-Tester
    2.Using the built-in benchmark to verify performance, the performance of float and doule type improved by about 60% and about 40% respectively.The test command is:
     export OMP_NUM_THREADS=1;numactl -C 10 -l ./sgemv.goto 3000 4000 100
     export OMP_NUM_THREADS=1;numactl -C 10 -l ./dgemv.goto 3000 4000 100
2025-08-12 18:03:16 +08:00
Chris Sidebottom 5f47b872f1 Remove older kernels for BGEMM on NEOVERSEV1 2025-08-11 09:25:19 +00:00
Chris Sidebottom 114316f361 Optimize SBGEMM / BGEMM for NEOVERSEV1 further
This changes the kernels to pack full SVE vectors and reduces the
overall complexity of the inner GEMM loop.
2025-08-11 09:25:13 +00:00
Martin Kroeker f1ee61ea30 Include NEON header for the bfloat conversion functions 2025-08-04 00:21:39 -07:00
Martin Kroeker b3ffd5524a Include NEON header for the bfloat conversion functions 2025-08-04 00:20:28 -07:00
Martin Kroeker a5e7c0e3e0 Merge pull request #5396 from abhishek-iitmadras/abhishekk_bfloat16
ARM64: Enable bfloat16 kernels by default
2025-07-28 13:39:08 -07:00
abhishek-fujitsu 0bc79da587 add neon header 2025-07-25 11:10:20 +05:30
Chris Sidebottom ea2faf0c9a Add optimized BGEMM for NEOVERSEN2 target
This re-uses the existing NEOVERSEN2 8x4 `sbgemm` kernel to implement `bgemm`.
2025-07-24 10:59:28 +00:00
Chris Sidebottom 2c3cdaf74e Optimized BGEMV for NEOVERSEV1 target
- Adds bgemv T based off of sbgemv T kernel
- Adds bgemv N which is slightly alterated to not use Y as an
accumulator due to the output being bf16 which results in loss of
precision
- Enables BGEMM_GEMV_FORWARD to proxy BGEMM to BGEMV with new kernels
2025-07-23 10:51:41 +01:00
Martin Kroeker 39c90f9859 Merge pull request #5380 from quic/topic/sgemm_direct_sme1_alpha_beta
SME1 based direct kernel (with alpha and beta) for cblas_sgemm level 3
2025-07-18 23:23:39 +02:00
Rajendra Prasad Matcha eae0abfdb6 SME1 based direct kernel with alpha and beta for cblas_sgemm level 3 API. 2025-07-17 16:14:31 +05:30
Chris SidebottomandYe Tao 740efd71c4 Add optimized BGEMM kernel for NEOVERSEV1 target
This also improves the testing and generic kernel by re-using the BF16
conversion functions.

Built on top of https://github.com/OpenMathLib/OpenBLAS/pull/5357 and derived from https://github.com/OpenMathLib/OpenBLAS/pull/5287

Co-authored-by: Ye Tao <ye.tao@arm.com>
2025-07-10 23:23:27 +00:00
Martin Kroeker fd37406817 Merge branch 'develop' into optimized_gemv_n_1x3 2025-07-08 21:05:30 +02:00
Iha, Taisei f7ad906b49 Performance improvements of [SD]DOT with loop-unrolling on A64FX 2025-07-04 22:57:44 +09:00
Martin Kroeker ee26caffb3 Merge pull request #5309 from davidz-ampere/dev-ampereone
Add support for Ampere AmpereOne processors
2025-06-24 12:27:08 +02:00
davidz-ampere aa90ab4142 Add support for Ampere AmpereOne processors 2025-06-24 00:12:34 -04:00
Ian McInerney badef1d32e Update sbgemm_tcopy_4_neoversev1 kernel to use standard C types 2025-06-19 14:26:16 +01:00
davidz-ampere 84730068af reduce duplicate kernel code 2025-06-17 03:05:34 -04:00
davidz-ampere be68ef03b4 Add support for Ampere processors 2025-06-15 22:00:40 -04:00
Martin Kroeker 58eeb9041c fix handling of dummy2 2025-06-12 03:03:01 -07:00
Martin Kroeker 1589d0b21e Merge pull request #5281 from martin-frbg/zscal_arm64
kernel/arm64: fixed cscal and zscal
2025-06-12 01:04:18 -07:00
Sharif Inamdar 8279e68805 Optimize gemv_n_sve_v1x3 kernel
- Calculate predicate outside the loop
- Divide matrix in blocks of 3
2025-06-11 10:16:56 +00:00
Arne Juul 5442aff218 Accumulate results in output register explicitly 2025-06-09 19:03:22 +00:00
Martin Kroeker 28f8fdaf0f support flag for NaN/Inf handling and fix scaling of NaN/Inf values 2025-05-23 14:59:59 +02:00
Martin Kroeker 5141a90993 Fix ARMV9SME target in DYNAMIC_ARCH and add SME query code for MacOS (#5222)
* Fix ARMV9SME target and add support_sme1 code for MacOS
* make sgemm_direct unconditionally available on all arm64
* build a (dummy) sgemm_direct kernel on all arm64





* Update dynamic_arm64.c
2025-05-10 22:39:32 +02:00
Martin Kroeker 151b74284e Merge pull request #5203 from quic/fix-sgemmdirect-sme1
Add vector registers to clobber list to prevent compiler optimization.
2025-05-09 05:39:47 -07:00
abhishek-fujitsu 9c02cdb073 optimise dot using thread throttling for NEOVERSE V1 2025-04-23 22:35:05 +05:30
Martin Kroeker d0e8fd6d40 Merge pull request #5239 from annop-w/gemv_n_sve
Use SVE kernel for S/DGEMVN for SVE machines
2025-04-22 10:19:49 -07:00
Iha, Taisei 08b5c18d70 fixed a potential out-of-bounds on gemv. 2025-04-22 19:56:44 +09:00
Annop Wongwathanarat e11744a411 Use SVE kernel for S/DGEMVN for SVE machines 2025-04-22 09:40:13 +00:00
Martin Kroeker dd38b4e811 Merge pull request #5225 from annop-w/gemv_n
Improve performance for SGEMVN on NEONVERSEN1
2025-04-17 01:54:10 -07:00
Martin Kroeker 0241d516f6 Merge pull request #5220 from iha-taisei/sdgemv_n_unroll
Further performance improvements to non-transposed [SD]GEMV kernels for A64FX and Neoverse V1.
2025-04-16 12:55:55 -07:00
Annop Wongwathanarat d535728803 Improve performance for SGEMVN on NEONVERSEN1 2025-04-16 09:54:30 +00:00
Usui, Tetsuzo d711906e3e Add symv kernels for arm64 2025-04-11 20:39:52 +09:00