Long-running BLAS calls (a large gemm can run for minutes) cannot
currently be interrupted: callers embedding OpenBLAS (e.g. the Julia
runtime responding to a user's ^C) can only wait for completion or kill
the process. Add a minimal cooperative cancellation protocol:
Every thread owns a pointer-sized generation slot in thread-local
storage, whose stable address is returned by openblas_cancel_token().
Instrumented compute drivers advance the slot to a fresh even
generation at operation entry on the issuing thread (forwarding the
slot and generation to worker threads through blas_arg_t) and poll it
at block granularity. openblas_cancel(token, loaded_token) - callable
from any thread - sets the cancel bit (bit 0) iff the slot still holds
loaded_token, so a canceller that loaded the value while an operation
was in flight stops exactly that operation, while stale or racing
requests either miss or dirty an already-dead generation, both
harmless. There is no object lifecycle: nothing to allocate, bind,
reset, or free.
A cancelled operation returns quickly, leaving its output buffer in an
unspecified partially-updated state that the caller must discard; every
synchronization point in the threaded driver is still executed, so
sibling threads never stall and the library remains consistent for
subsequent calls. Coverage: the level-3 gemm/symm/hemm drivers
(level3.c and level3_thread.c).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012nCkyKUguncLLJrH9K5o7m
RISC-V was the only major architecture without a get_L2_size() /
blas_set_parameter() implementation, so the GEMM cache-blocking parameters
(P/Q/R) were fixed at compile time regardless of the actual L2 cache size.
Because the blocking is now derived from the L2 cache detected at runtime
rather than a fixed compile-time constant, future RISC-V cores - which are
arriving with progressively larger and more varied L2 caches - get more
optimal blocking automatically, and the port gains the same runtime-tuning
hook x86 and LoongArch already use.
This adds, under ARCH_RISCV64:
- get_L2_size(): reads the level-2 (unified) cache size from Linux sysfs
(/sys/devices/system/cpu/cpu0/cache/index*/{level,size}); RISC-V has no
architectural cache-size query like x86 CPUID or LoongArch CPUCFG. Falls
back to 512 KB when sysfs is unavailable.
- blas_set_parameter(): scales each precision's packed-A dimension P from the
detected L2. The base blocking and the reference cache size come from the
active core's own param.h block (*_DEFAULT_P_BASE, RISCV_L2_REFERENCE_KB),
so the function carries no core-specific constants and is a no-op for cores
that do not opt in. Q and R keep their param.h defaults.
- driver/others/memory.c and common_macro.h: add ARCH_RISCV64 to the existing
architecture lists that call blas_set_parameter() and declare the runtime
parameter variables (sgemm_p, dgemm_p, ...).
- param.h RISCV64_ZVL256B: declares the per-core base blocking + reference and
maps SGEMM/DGEMM/CGEMM/ZGEMM DEFAULT_P to the runtime variables for static
builds; DYNAMIC_ARCH keeps the literals, since kernel/setparam-ref.c
init_parameter() initialises the gotoblas table from these macros and
blas_set_parameter() is not called on the dynamic path.
Only RISCV64_ZVL256B opts in so far; its base + reference are tuned on the
SpaceMiT X60, where a 512 KB L2 reproduces the stock blocking, so this is
performance-neutral on current hardware. Verified: a static RISCV64_ZVL256B
build reproduces the stock 128/128/16384 (SGEMM) and 64/128/8192 (DGEMM)
blocking; a DYNAMIC_ARCH build compiles cleanly (per-core setparam-ref objects
build without error); and get_L2_size() reads the correct size on both a
SpaceMiT X60 (512 KB L2) and a SiFive U74 / VisionFive 2 (2 MB L2).
This adds all the relevant bits and pieces to add a `shgemv` path as
well as a future `hgemm`/`hgemv` path in a similar model to `sb` and `b`
interfaces.
I've also fixed a few bits and pieces around `shgemm` which didn't build
in a few situations.
- Sets up all the various entrypoints for `bgemv`
- Adds `bscal` for use in the `bgemv` interface
- Adds test cases for comparing `sgemv` and `bgemv`
- Adds generic kernels for `bgemv_n` and `bgemv_t` which are accurate
enough to pass above tests
driver/others/parameter.c does not get build during DYNAMIC_ARCH, thus,
do not declare its symbols. This will make the build fail early and in
an obvious way if functions are trying to use these symbols.
Signed-off-by: Egbert Eich <eich@suse.com>
1. Add a new API -- sbgemv to support bfloat16 based gemv
2. Implement a generic kernel for sbgemv
3. Implement an avx512-bf16 based kernel for sbgemv
Signed-off-by: Chen, Guobing <guobing.chen@intel.com>
1. Added bfloat16 based dot as new API: shdot
2. Implemented generic kernel and cooperlake-specific (AVX512-BF16) kernel for shdot
3. Added 4 conversion APIs for bfloat16 data type <=> single/double: shstobf16 shdtobf16 sbf16tos dbf16tod
shstobf16 -- convert single float array to bfloat16 array
shdtobf16 -- convert double float array to bfloat16 array
sbf16tos -- convert bfloat16 array to single float array
dbf16tod -- convert bfloat16 array to double float array
4. Implemented generic kernels for all 4 conversion APIs, and cooperlake-specific kernel for shstobf16 and shdtobf16
5. Update level1 thread facilitate functions and macros to support multi-threading for these new APIs
6. Fix Cooperlake platform detection/specify issue when under dynamic-arch building
7. Change the typedef of bfloat16 from unsigned short to more strict uint16_t
Signed-off-by: Chen, Guobing <guobing.chen@intel.com>
This patch adds support for bfloat16 data type matrix multiplication kernel.
For architectures that don't support bfloat16, it is defined as unsigned short
(2 bytes). Default unroll sizes can be changed as per architecture as done for
SGEMM and for now 8 and 4 are used for M and N. Size of ncopy/tcopy can be
changed as per architecture requirement and for now, size 2 is used.
Added shgemm in kernel/power/KERNEL.POWER9 and tested in powerpc64le and
powerpc64. For reference, added a small test compare_sgemm_shgemm.c to compare
sgemm and shgemm output.
This patch does not cover OpenBLAS test, benchmark and lapack tests for shgemm.
Complex type implementation can be discussed and added once this is approved.
The Ximatcopy functions create a copy of the input matrix
although they seem to work inplace. The new routines
XIMATCOPY_K_YY perform the operations inplace if the leading
dimension does not change.
