The optional NaN checks performed by LAPACKE front-ends computed array
offsets and lengths in 32-bit lapack_int arithmetic, which overflows for
large matrices (e.g. LAPACKE_dpotrf with n = lda >= 46341 overflows
j*lda in LAPACKE_dtr_nancheck, leading to out-of-bounds reads and
crashes). Several nancheck helpers (ge, gb, tp, tf) already cast to
size_t; this applies the same treatment to the ones that were missed:
- ?tr_nancheck: cast a[i+j*lda] index to size_t (also covers po/sy/he
checks, i.e. the POTRF/Cholesky path)
- ?_nancheck (vector): iterate by element count instead of n*inc, with
size_t indexing
- ?sp/?pp/?pf/?hp/?tp/?tf_nancheck: compute n*(n+1)/2 length in size_t
- ?tz_nancheck: widen the part offsets to int64_t (keeping the -1
sentinel) and compute them with a widening cast
Verified with UBSan: LAPACKE_str_nancheck at n = lda = 47000 previously
reported signed integer overflow and crashed with SEGV at -O0; it now
passes and still detects the NaN. Old and new code produce identical
results across 1907 enumerated small-size cases (all layouts, uplo,
diag, shapes and NaN positions for tr/tz/vector/packed variants).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Update make.inc entries for LAPACK 3.9.0
Reference-LAPACK PR 347 changed some variable names and relative paths
* Update LAPACK to 3.9.0
* Add new functions from LAPACK 3.9.0
* Add new functions from LAPACK 3.9.0
* Restore LOADER command
as it makes it easier to specify pthread as needed
* Restore LOADER
* Restore EIG/LIN prefixes in cmdbase
* add binary path to lapack_testing.py call
* Restore OpenMP version check
* Restore OpenMP version check
* Restore fix for out-of-bounds array accesses
from #2096
Martin Kroeker