Per review, the U74-specific DGEMM micro-kernel does not belong under
kernel/generic (reserved for portable, architecture-neutral C kernels).
Move both the hybrid dispatcher and its readable assembly source into
kernel/riscv64, next to KERNEL.U74:
- git mv kernel/generic/gemmkernel_4x4_u74.c -> kernel/riscv64/
- git mv kernel/generic/kern_u74.S -> kernel/riscv64/
- KERNEL.U74: DGEMMKERNEL ../generic/gemmkernel_4x4_u74.c
-> ../riscv64/gemmkernel_4x4_u74.c
- gemmkernel_4x4_u74.c: #include "conversion_macros.h"
-> #include "../generic/conversion_macros.h"
(that header stays under kernel/generic; same form kernel/wasm
already uses for it)
No functional change. The portable gemmkernel_4x4.c (SGEMMKERNEL) is
untouched and stays in kernel/generic. DGEMMKERNEL still resolves through
$(KERNELDIR), so both the Makefile ($(KERNELDIR)/$(DGEMMKERNEL)) and CMake
(GenerateNamedObjects "${KERNELDIR}/${DGEMMKERNEL}") builds pick the kernel
up from its new location.
RISC-V was the only major architecture without a get_L2_size() /
blas_set_parameter() implementation, so the GEMM cache-blocking parameters
(P/Q/R) were fixed at compile time regardless of the actual L2 cache size.
Because the blocking is now derived from the L2 cache detected at runtime
rather than a fixed compile-time constant, future RISC-V cores - which are
arriving with progressively larger and more varied L2 caches - get more
optimal blocking automatically, and the port gains the same runtime-tuning
hook x86 and LoongArch already use.
This adds, under ARCH_RISCV64:
- get_L2_size(): reads the level-2 (unified) cache size from Linux sysfs
(/sys/devices/system/cpu/cpu0/cache/index*/{level,size}); RISC-V has no
architectural cache-size query like x86 CPUID or LoongArch CPUCFG. Falls
back to 512 KB when sysfs is unavailable.
- blas_set_parameter(): scales each precision's packed-A dimension P from the
detected L2. The base blocking and the reference cache size come from the
active core's own param.h block (*_DEFAULT_P_BASE, RISCV_L2_REFERENCE_KB),
so the function carries no core-specific constants and is a no-op for cores
that do not opt in. Q and R keep their param.h defaults.
- driver/others/memory.c and common_macro.h: add ARCH_RISCV64 to the existing
architecture lists that call blas_set_parameter() and declare the runtime
parameter variables (sgemm_p, dgemm_p, ...).
- param.h RISCV64_ZVL256B: declares the per-core base blocking + reference and
maps SGEMM/DGEMM/CGEMM/ZGEMM DEFAULT_P to the runtime variables for static
builds; DYNAMIC_ARCH keeps the literals, since kernel/setparam-ref.c
init_parameter() initialises the gotoblas table from these macros and
blas_set_parameter() is not called on the dynamic path.
Only RISCV64_ZVL256B opts in so far; its base + reference are tuned on the
SpaceMiT X60, where a 512 KB L2 reproduces the stock blocking, so this is
performance-neutral on current hardware. Verified: a static RISCV64_ZVL256B
build reproduces the stock 128/128/16384 (SGEMM) and 64/128/8192 (DGEMM)
blocking; a DYNAMIC_ARCH build compiles cleanly (per-core setparam-ref objects
build without error); and get_L2_size() reads the correct size on both a
SpaceMiT X60 (512 KB L2) and a SiFive U74 / VisionFive 2 (2 MB L2).
These stale duplicate sources are not referenced by the build. The canonical implementations remain under lapack-netlib/LAPACKE/src.
Signed-off-by: moluopro <moluopro@qq.com>
Until now, the code in `num_cpu_avail()`,
if (blas_cpu_number != openmp_nthreads) {
goto_set_num_threads(openmp_nthreads);
}
would just always set the threads back to OpenMP's thread count.
Mirror the RISCV64_GENERIC target blocks for U74 in cmake/prebuild.cmake
(cache defines: 32 KiB L1D / 2 MiB L2, 64 B lines, matching getarch),
cmake/system.cmake and cmake/cc.cmake
(-march=rv64imafdc_zba_zbb -mabi=lp64d -mtune=sifive-u74), so
'cmake -DTARGET=U74' builds alongside the Makefile path.
Verified on a VisionFive 2: configure recognizes the target and a full
'cmake --build' produces libopenblas.a with the U74 march/mtune flags
applied to the kernel objects (0 errors).
Full-memory HPL N=27456 (asm hybrid + Q=256) measured at 5.99 GFLOPS
(residual PASSED, ~50% of the 12 GF peak, +10.7% over the tuned C kernel,
1.81x the stock 2x2) - the best clean figure. Doc-only header update.
Adds a hand-written RV64GC scalar 4x4 DGEMM micro-kernel (kern_u74.S) and
wires it as the U74 DGEMMKERNEL via a hybrid dispatcher
(gemmkernel_4x4_u74.c): the fast path (bm,bn multiples of 4, even bk,
non-TRMM) runs the asm; all other shapes, odd bk, and the TRMM builds fall
back to the portable C 4x4 kernel.
The asm uses a 4x4 register tile (16 accumulators), full operand double-
buffering (P/Q ping-pong) with one-iteration lookahead, and load-before-FMA
issue ordering matched to the U74's dual-issue in-order front end. Probes
show this reaches the FP-pipe peak (~16.5 cycles / 16 fmadd.d = 2.9 GF on
L1-resident data); the streaming plateau is memory-latency-bound, not the
schedule.
Measured on a VisionFive 2 (single-core, KC=256): micro-kernel 1.88 vs 1.54
GF (+22%), full blocked DGEMM 1.77 vs 1.48 GF (+20%); the advantage holds
under 4-core contention (+17%). End-to-end HPL N=10000 (4 cores, Q=256):
5.17 vs 4.97 GF (+4.0%), residual PASSED. Correctness validated against the
full BLAS Level-3 test suite (DGEMM 17,496 computational calls, 0 failures).
kern_u74.S is the readable source; the .c embeds it via top-level __asm__ so
it builds as a single OpenBLAS kernel object with no build-system changes.
A same-board controlled HPL A/B (VisionFive 2, TARGET=U74, N=10000,
NB=192, 2x2 grid, both libraries clean-built, residual PASSED) measures
4.97 vs 4.86 GFLOPS (+2.25%) for DGEMM_Q=256 over the inherited 128; at
a memory-filling N=27456 the tuned build sustains 5.41 vs 5.24 GFLOPS
(+3.24%). With HPL's NB=192, KC=256>=192 lets each trailing-update K
panel run in a single KC block, halving the C read-modify-write traffic
versus KC=128 (which splits K=192 into 128+64). A single-core P/Q sweep
over the real packing + micro-kernel objects confirms the GEMM is
compute-bound (whole grid within 3%), so P and R are left unchanged;
SGEMM_Q was already 240.
The StarFive JH7110's U74 cores implement rv64imafdc_..._zba_zbb, so the U74
target now compiles with -march=rv64imafdc_zba_zbb. The generic RISCV64_GENERIC
target stays at bare rv64imafdc for portability across unknown RV64GC cores.
Measured on the VisionFive 2 (GCC 13.3, -mtune=sifive-u74 held constant): GCC
emits Zba shift-add instructions in the packing routines, but DGEMM is unchanged
-- the 4x4 kernel holds 1.533 GF either way and packing 1.74 vs 1.75 GB/s. The
4x4 micro-kernel is FMA-bound (fused fmadd.d with immediate-offset loads) and
packing is LPDDR4-bandwidth-bound, so integer address generation is not on the
critical path. The flag is nonetheless the correct -march for the silicon, is
free, and can only help address-gen-bound code elsewhere in the library. The
remaining GEMM headroom on the U74 is microarchitectural scheduling (a
hand-written assembly micro-kernel), not the ISA.
Martin Kroeker