Shares error_percent with the Markdown report, including the 0.01%
floor for nonzero counts; the error columns are rebalanced by one
character so (100.00%) still fits under the rule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A nonzero error count whose percentage would display as 0.00% is
shown as 0.01% instead, so it cannot read as a zero error rate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Summary tables first, then one entry per failing output file with
its notable lines collapsed; sized in UTF-8 bytes to stay under
GitHub's 1 MiB step-summary limit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collapses a library's two API sections into one table when they report
the same errors. Compared per output file, so a missing file or two
different failures with equal totals keep the sections split.
Add the CBLAS suite as its own library, alongside LAPACK and BLAS:
- A test case per precision and BLAS level. The CBLAS inputs carry no
output file name, so the same input serves both APIs and the harness
redirects the output for every level, unlike the BLAS Level 2 and 3
drivers.
- --cblas-dir, defaulting to CBLAS/testing, which is the correct
relative path from both a CMake build root and the source tree. A
missing directory is skipped silently, since CBLAS is off by
default.
- The verdict and count patterns accept the COLUMN-MAJOR and ROW-MAJOR
tags the CBLAS drivers add, which the BLAS drivers do not emit.
The parser itself is shared with BLAS: both suites report their counts
in the same form, so only the layout tags and the test tables differ.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The BLAS tests run under both build systems but nothing reported on
them: the script knew only the LAPACK drivers and scanned only the
LAPACK testing directory.
Add the BLAS suite as its own library:
- A test case per precision and BLAS level, with the Level 1 drivers
marked as reading no input, and the Level 2 and 3 drivers marked as
opening their own output file.
- parse_blas, which takes run and failure counts from the driver's
report lines. Computational failures are numerical errors and
error-exit failures are other errors.
- --blas-dir, defaulting to BLAS/TESTING. That relative path is
correct from a CMake build root and from the source tree alike, so
no existing invocation needs changing. A missing directory is
skipped silently, since builds against an optimized BLAS have none.
- --run support, which needs the input-less Level 1 drivers handled
and the output redirection suppressed for Level 2 and 3. A _64 run
of Level 2 or 3 requires the generated _64 input; without it the
driver would write over the default-API output, so it is skipped
with a message instead.
The drivers exit 0 even when they abandon a run, and print END OF TESTS
even when routines failed, so breakage is read from the text: an
abandoned or misconfigured run, and a Level 2 or 3 file that never
reached its footer, each count as one other error. Detail lines behind
an already counted verdict are shown but not counted, which otherwise
inflates a failing file several fold.
Output from a build whose drivers do not report counts is still
summarized, by counting one test per verdict. That is coarse, but never
silently zero.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The script is built around a single testing directory and a single set
of test families. Adding another test suite means teaching it that a
test case belongs to a library, that each library has its own directory,
and that each gets its own section of the summary table.
Introduce that structure without changing what is analyzed:
- TestCase gains a library field, defaulting to LAPACK.
- The testing directories are held in a per-library mapping, which
read_output_file, run_test_case and the unrecognized-output scan now
resolve through instead of assuming one directory.
- find_unrecognized_outputs takes that mapping and matches each
directory against only the cases of the library that owns it, so
--fail-on-unrecognized stays accurate once more directories exist.
- discover_suffixes takes the cases and directory of one library, so
the extended API is detected per library rather than globally.
- The summary loop iterates over (library, API) sections, and
section_title names both.
- -t gains a lapack value selecting all LAPACK families, so the
previous default remains expressible once other libraries are added.
The only visible change is that section headings now read
"LAPACK: Default API" rather than "Default API". Analyzed output is
otherwise byte for byte identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
When building with cmake, the test executables and data files may not
be in the location where lapack_testing.py. By adding the -b option,
we can specify this manually. For example, if we do
mkdir build && cd build && cmake -DBUILD_TESTING=ON .. && make
we can then call
./lapack_testing.py -r -b bin -d ../TESTING
which now runs the tests.
The purpose of the patches is to print lapack results in a single line
test-case-name sub-tests-running pass/fail status and count
Modified sub test case name to single word
Nonsymmetric Eigenvalue Problem to Nonsymmetric-Eigenvalue-Problem
Signed-off-by: Naresh Kamboju <naresh.kamboju@linaro.org>
Update Makefile to handle .F
Update Cmake build
Note: We have duplicate loop numbers due to Define statement that stops
the compilation with Cmake default option.
Fix coming very shortly
The following variables will control the precision to be built:
BUILD_SINGLE
BUILD_DOUBLE
BUILD_COMPLEX
BUILD_COMPLEX16
For mixed precision SINGLE/DOUBLE routines, both BUILD_SINGLE and BUILD_DOUBLE needs to be on.
(same for COMPLEX/COMPLEX16)
lapack_testing.py has been moved to the top directory
One argument was added :
- d [dir] is to indicate where is the LAPACK testing directory (.out files). By default, the script will use .-d or --dir to
The -f argument was replaced by -r argument. Now by default the script will use the LAPACK output.
- r is to use to run the LAPACK tests then analyse the output (.out files). By default, the script will not run all the LAPACK tests
INTEGRATION:
Integration of the LAPACK Testing parsing in the Makefile and CMake build process.
In the Makefile build: now the lapack_testing target will call the script after running the tests
In the CMAKE build: after the tests are run, the script will be executed with the option -s (short)
the end of the output will look like this:
100% tests passed, 0 tests failed out of 98
Total Test time (real) = 211.83 sec
SUMMARY numerical error other error
================ ================= ================
REAL 40 (0.004%) 0 (0.000%)
DOUBLE PRECISION 202 (0.020%) 0 (0.000%)
COMPLEX 2 (0.000%) 0 (0.000%)
COMPLEX16 28 (0.005%) 0 (0.000%)
--> ALL PRECISIONS 272 (0.009%) 0 (0.000%)
