Chip Kerchner
daa3215fb4
Remove shadow variable.
2026-03-31 14:41:38 +00:00
Chip Kerchner
d69be17b6f
Convert 2X LMUL1 instructions to 1X LMUL2. Improved FP64 GEMM edges - up to more than 3X faster.
2026-03-30 18:50:27 +00:00
Chip Kerchner
8fc0004024
Fix another typo.
2026-03-24 13:04:54 +00:00
Chip Kerchner
ebf4cd1c6e
Fix typo.
2026-03-22 18:28:21 +00:00
Chip Kerchner
1bb72b223d
Only initialize unused variables to prevent GCC warnings.
2026-03-20 16:00:33 +00:00
Chip Kerchner
a8a00bbf4f
Avoid greater than 4 segment load and store penalties by using 2. Fix mf2 length.
2026-03-19 21:17:20 +00:00
Chip Kerchner
1e48686df8
Merge remote-tracking branch 'origin/develop' into fasterRVVEdges
2026-03-19 12:15:19 +00:00
Martin Kroeker
adba2c3c02
Merge pull request #5685 from teddygood/wasm-intrin-backend-exp
...
Add a WebAssembly SIMD backend for reusable intrinsics kernels
2026-03-18 21:49:53 +01:00
Chip Kerchner
d832ee5086
More global optimzation and clean up.
2026-03-18 13:43:03 +00:00
teddygood
99d05575d0
Enable SAXPY for WebAssembly SIMD backend
2026-03-18 21:27:45 +09:00
teddygood
7ff3588833
Refine WebAssembly SIMD backend scope
2026-03-18 17:24:02 +09:00
Martin Kroeker
79a50d80d3
Fix potential over-optimization by gcc15
2026-03-17 23:13:58 +01:00
Chip Kerchner
477dd40f07
Simplier loops.
2026-03-17 21:18:30 +00:00
teddygood
53d0be88f8
Add WebAssembly SIMD backend for universal intrinsics
2026-03-18 03:23:31 +09:00
Chip Kerchner
79d9fe39c2
Use mf2 instead of m1.
2026-03-17 16:06:54 +00:00
Martin Kroeker
7a95460bb1
Merge pull request #5680 from teddygood/wasm128-generic-target-exp
...
Add WebAssembly SIMD SGEMM and DGEMM kernels
2026-03-17 14:25:39 +01:00
Martin Kroeker
a1fd7a4658
Merge pull request #5677 from CheryDan/riscv/zdrot
...
Optimize ZROT_RVV for the non-unit-stride case
2026-03-17 11:10:32 +01:00
Chip Kerchner
f927b9448a
Global optimizations.
2026-03-16 21:32:17 +00:00
teddygood
86d1451cbe
Add WebAssembly SIMD GEMM kernels
2026-03-17 05:51:54 +09:00
Chip Kerchner
010f24f961
Better K.
2026-03-16 13:22:31 +00:00
daichengrong
aa967ef6ba
Optimize ZROT_RVV for the non-unit-stride case
...
Optimize the RVV implementation of ZROT when inc_x and inc_y are
non-unit strides (inc_x != 1, inc_y != 1).
Reorder several operations to reduce vector register pressure and
avoid unnecessary vector register spill to the stack. This helps GCC
keep vector values in registers and reduces redundant spill/reload
instructions, improving runtime performance.
No functional change.
Signed-off-by: daichengrong <daichengrong@iscas.ac.cn >
2026-03-16 14:22:54 +08:00
Martin Kroeker
4a888bcb73
set USE_TRMM for WASM
2026-03-15 23:07:16 +01:00
Martin Kroeker
ef3315527f
Don't include the CPUID code in WebAssembly builds
2026-03-15 19:30:13 +01:00
Martin Kroeker
48f0a0f0ec
Generate WASM kernel including existing intrinsics-based kernels
2026-03-15 19:28:08 +01:00
Martin Kroeker
cc64ce68c3
Create generic C KERNEL as baseline for WASM
2026-03-15 19:26:42 +01:00
Chip Kerchner
b0ee407943
Preserve K.
2026-03-14 17:46:42 +00:00
Chip Kerchner
eb9bbcc299
Common unroll code.
2026-03-14 16:38:12 +00:00
Chip Kerchner
fda433f25a
Unroll K for N less than or equal to 4.
2026-03-13 22:17:47 +00:00
Chip Kerchner
9c16449dfd
Add K-unrolling to M = 8. Other small changes.
2026-03-13 15:59:29 +00:00
Chip Kerchner
6d6af1dee5
Add bool types for C.
2026-03-12 13:13:34 +00:00
Chip Kerchner
376d3a138f
Fast performing edges for FP32 GEMM of RVV.
2026-03-12 12:33:25 +00:00
Martin Kroeker
37262654d9
Merge pull request #5667 from fadara01/accelerate_sve128_sbgemm
...
Accelerate SVE128 SBGEMM/BGEMM
2026-03-06 09:14:44 +01:00
Fadi Arafeh
f30202b705
Accelerate SVE128 SBGEMM/BGEMM
...
This accelerates SBGEMM/BGEMM by extending the existing 8x4 kernel to 8x8 (unrolling N by 8)
Not sure if it's a good idea to delete the previous 8x4 kernel?
Here are the speedups on single core Neoverse-V2 (SVE128) compared to prev state:
Per-shape speedup
M=N=K=64: SBGEMM 1.164x (16.42%), BGEMM 1.133x (13.30%)
M=N=K=128: SBGEMM 1.220x (22.02%), BGEMM 1.186x (18.56%)
M=N=K=256: SBGEMM 1.241x (24.08%), BGEMM 1.235x (23.54%)
M=N=K=512: SBGEMM 1.240x (23.95%), BGEMM 1.227x (22.75%)
M=N=K=1024: SBGEMM 1.251x (25.11%), BGEMM 1.232x (23.23%)
M=N=K=2048: SBGEMM 1.235x (23.47%), BGEMM 1.246x (24.64%)
Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com >
2026-03-05 13:50:07 +00:00
Martin Kroeker
df29cc0205
Use AVX2 in the tail loop too for consistent FMA rounding
2026-03-03 15:51:51 +01:00
Martin Kroeker
ef27ec6bed
Add pragma to limit optimization level
2026-02-22 13:42:41 +01:00
Martin Kroeker
30cf14c548
Merge pull request #5640 from ChipKerchner/RVV_Narrow_Accumulate_FP16_GEMM
...
Added ability to accumulate in FP16. Convert BF16 to FP32. For FP16 and BF16 GEMM in RISC-V (BF16 now works for pre-RVA23)
2026-02-20 14:22:27 +01:00
Martin Kroeker
46b963b9a0
Use generic C kernels for SCAL on FreeBSD
2026-02-19 22:46:03 +01:00
Chip Kerchner
efe63e7970
Add pre-RVA23 to BF16 GEMM.
2026-02-15 15:49:59 +00:00
Chip Kerchner
1d6aa0dc31
Add dummy memsets - just in case.
2026-02-13 20:03:35 +00:00
Chip Kerchner
7a1d23400f
Add flag for not converting A & B - will be used in future to do conversion during packing.
2026-02-13 19:00:41 +00:00
Chip Kerchner
1cc377ef61
Only convert B if M is greater or equal to 4.
2026-02-13 18:14:11 +00:00
Chip Kerchner
0acb60aab3
Conversion from BF16 to FP32 only once.
2026-02-13 17:55:15 +00:00
Chip Kerchner
9701a80a9f
One small change.
2026-02-12 20:35:41 +00:00
Chip Kerchner
4121a22c02
Convert BF16 values once (and vectorized).
2026-02-12 18:45:39 +00:00
Chip Kerchner
33560437f5
Convert inputs from BF16 to FP32 and use FP32 vector madds. 18% faster.
2026-02-11 19:50:48 +00:00
Chip Kerchner
e3cb067bf4
Fixed MADD to use float16 values. Use LMUL = 2 in main loop. Now 1.85X faster on BananaPi.
2026-02-11 00:27:27 +00:00
Chip Kerchner
74d9fe2832
Forget to add defintion.
2026-02-10 19:00:26 +00:00
Chip Kerchner
aa1cebd45b
128-bit versions.
2026-02-10 18:30:02 +00:00
Chip Kerchner
b5f2a50fe9
Added ability to accumulate in FP16 for GEMM. Widens once at the end of loops.
2026-02-10 17:30:05 +00:00
Martin Kroeker
69d92490c1
move inclusion of sme_abi header into the conditional section
2026-01-29 22:24:00 +01:00
Chip Kerchner