phynexis-deps/mfem
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
69a4a38053d0323cd656facfdbc8d4e724148220
mfem/INSTALL
T
Tzanio Kolev 9be0bfe7cb Merge branch 'master' into mfem-4.7-dev 
Conflicts:
	CHANGELOG
2024-05-04 19:34:35 -07:00

1133 lines
46 KiB
Plaintext
Raw Blame History

Finite Element Discretization Library
__
_ __ ___ / _| ___ _ __ ___
| '_ ` _ \ | |_ / _ \| '_ ` _ \
| | | | | || _|| __/| | | | | |
|_| |_| |_||_| \___||_| |_| |_|
https://mfem.org
This file provides a detailed description of how to build and install the MFEM
library. For a simple build, see the step-by-step instructions on the website
at https://mfem.org/building.
The MFEM library has a serial and an MPI-based parallel version, which largely
share the same code base. The only prerequisite for building the serial version
of MFEM is a (modern) C++ compiler, such as g++. The parallel version of MFEM
requires an MPI C++ compiler, as well as the following external libraries:
- hypre (a library of high-performance preconditioners)
https://github.com/hypre-space/hypre
- METIS (a family of multilevel partitioning algorithms)
https://github.com/mfem/tpls
Note: We recommend our mirror of metis-4.0.3/5.1.0 above because the METIS
webpage, http://glaros.dtc.umn.edu/gkhome/metis/metis/overview, is often down
and we don't support yet the new repo https://github.com/KarypisLab/METIS.
The hypre dependency can be downloaded as a tarball from GitHub or from the
project webpage https://www.llnl.gov/casc/hypre. For example, the 2.24.0 release
of hypre is available at
https://github.com/hypre-space/hypre/archive/v2.24.0.tar.gz
The METIS dependency can be disabled but that is not generally recommended, see
the option MFEM_USE_METIS.
MFEM also includes support for devices such as GPUs, and programming models such
as CUDA, HIP, OCCA, OpenMP and RAJA.
- Starting with version 4.0, MFEM requires a C++11 compiler. We recommend using
a newer compiler, e.g. GCC version 4.9 or higher.
- CUDA support requires an NVIDIA GPU and an installation of the CUDA Toolkit
https://developer.nvidia.com/cuda-toolkit
- HIP support requires an AMD GPU and an installation of the ROCm software stack
https://rocmdocs.amd.com
- OCCA support requires the OCCA library
https://libocca.org
- OpenMP support requires a compiler implementing the OpenMP API
https://www.openmp.org
- RAJA support requires installation of the RAJA performance portability layer
with (optionally) support for CUDA and OpenMP
https://github.com/LLNL/RAJA
The library supports two build systems: one based on GNU make, and a second one
based on CMake. Both build systems are described below. Some hints for building
without GNU make or CMake can be found at the end of this file.
In addition to the native build systems, MFEM packages are also available in the
following package managers:
- Spack, https://github.com/spack/spack
- OpenHPC, http://openhpc.community
- Conda-forge, https://conda-forge.org (pre-built binaries linked with
OpenMPI/MPICH, hypre, and METIS)
We also recommend downloading and building the MFEM-based GLVis visualization
tool which can be used to visualize the meshes and solution in MFEM's examples
and miniapps. See https://glvis.org and https://mfem.org/building.
Quick start with GNU make
=========================
See also: https://mfem.org/building
Serial build:
make serial -j 4
Parallel build:
(download hypre and METIS 4 from above URLs)
(build METIS 4 in ../metis-4.0 relative to mfem/)
(build hypre in ../hypre relative to mfem/)
make parallel -j 4
(For METIS 5, see https://mfem.org/building/#parallel-build-using-metis-5)
CUDA build:
make cuda -j 4
(build for a specific compute capability: 'make cuda -j 4 CUDA_ARCH=sm_70')
HIP build:
make hip -j 4
(build for a specific AMD GPU chip: 'make hip -j 4 HIP_ARCH=gfx900')
Example codes (serial/parallel, depending on the build):
cd examples
make -j 4
Build everything (library, examples and miniapps) with current configuration:
make all -j 4
Quick-check the build by running Example 1/1p (optional):
make check
Quick start with CMake
======================
Serial build:
mkdir <mfem-build-dir> ; cd <mfem-build-dir>
cmake <mfem-source-dir>
make -j 4 (assuming "UNIX Makefiles" generator)
Parallel build:
(download hypre and METIS 4 from above URLs)
(build METIS 4 in ../metis-4.0 relative to mfem/)
(build hypre in ../hypre relative to mfem/)
mkdir <mfem-build-dir> ; cd <mfem-build-dir>
cmake <mfem-source-dir> -DMFEM_USE_MPI=YES
make -j 4
(For METIS 5, see https://mfem.org/building/#parallel-build-using-metis-5)
CUDA build:
(this build requires CMake 3.8 or newer)
mkdir <mfem-build-dir> ; cd <mfem-build-dir>
cmake <mfem-source-dir> -DMFEM_USE_CUDA=YES
make -j 4
Example codes (serial/parallel, depending on the build):
make examples -j 4
Build everything (library, examples and miniapps) with current configuration:
make exec -j 4
Quick-check the build by running Example 1/1p (optional):
make check
Building with GNU make
======================
The MFEM build system consists of two steps: configuration and compilation.
The configuration step can be used to adjust paths to external libraries,
compilers, flags, etc, similar to "./configure". It is performed by running
make config [OPTIONS] ...
The OPTIONS are of the form VARIABLE=VALUE. Detailed description of the
configuration options is given below. Alternatively, the options can be
specified with an input file:
cp config/defaults.mk config/user.mk
(edit config/user.mk)
make config
Note that config/user.mk, if present, is loaded after config/defaults.mk and
its path/name can be changed with
make config USER_CONFIG=<user_config_file>
The build system can be configured to use a separate build directory, for an
out-of-source build. There are two ways to do that: the first one is
mkdir <mfem-build-dir> ; cd <mfem-build-dir>
make -f <mfem-source-dir>/makefile config [OPTIONS] ...
The second one is
cd <mfem-source-dir>
make BUILD_DIR=<mfem-build-dir> config [OPTIONS] ...
Note that in both cases the default location for the (optional) user
configuration file is <mfem-build-dir>/config/user.mk.
Once configured, the library can be built simply with
cd <mfem-build-dir> (if building out-of-source)
make
Note that re-configuration is only needed to change the currently configured
options. Several shortcut targets combining (re-)configuration and compilation
are also defined:
make serial -> Builds serial optimized version of the library
make parallel -> Builds parallel optimized version of the library
make debug -> Builds serial debug version of the library
make pdebug -> Builds parallel debug version of the library
make cuda -> Builds serial cuda optimized version of the library
make pcuda -> Builds parallel cuda optimized version of the library
make cudebug -> Builds serial cuda debug version of the library
make pcudebug -> Builds parallel cuda debug version of the library
make hip -> Builds serial hip optimized version of the library
make phip -> Builds parallel hip optimized version of the library
make hipdebug -> Builds serial hip debug version of the library
make phipdebug -> Builds parallel hip debug version of the library
Note that any of the above shortcuts accept configuration options, either at the
command line or through a user configuration file.
The build can be quick-tested by running
make check
which will simply compile and run Example 1/1p. For more extensive tests that
check the results from all the serial/parallel MFEM examples and miniapps use:
make test
Note that by default MFEM uses "mpirun -np" in its test runs (this is also what
is used in the sample runs of its examples and miniapps). The MPI launcher can
be changed by the user as described in the "Specifying an MPI job launcher"
section at the end of this file.
Running all the tests may take a while. Implementation details about the check
and test targets can be found in the top-level makefile and the config/test.mk
file.
An optional installation of the library and the headers can be performed with
make install [PREFIX=<dir>]
The library will be installed in $(PREFIX)/lib, the headers in
$(PREFIX)/include, and the configuration makefile (config.mk) in
$(PREFIX)/share/mfem. The PREFIX option can also be set during configuration.
Information about the current build configuration can be viewed using
make status
make info
To clean the library and object files, but keep the current configuration, use
make clean
To clean everything, including the current configuration, use
make distclean
For a short help message, use
make help
The build process creates the MFEM library (libmfem.a) and the include file
(mfem.hpp) needed in MFEM-based applications, see e.g. the example codes in the
examples/ directory or the miniapps in the miniapps/ directory. A selected
subset of configuration options and derived makefile variables are also written
to the file config/config.mk. This file can be included by other makefiles to
obtain information about the MFEM configuration, see e.g. the makefile in the
examples/ directory.
Configuration options (GNU make)
================================
See the configuration file config/defaults.mk for the default settings.
Compilers:
CXX - C++ compiler, serial build
MPICXX - MPI C++ compiler, parallel build
CUDA_CXX - The CUDA compiler, 'nvcc'
Compiler options:
OPTIM_FLAGS - Options for optimized build
DEBUG_FLAGS - Options for debug build
CXXFLAGS - If not set, defined based on the above optimized/debug flags
CPPFLAGS - Additional compiler options
Build options:
STATIC - Build a static version of the library (YES/NO), default = YES
SHARED - Build a shared version of the library (YES/NO), default = NO
Installation options:
PREFIX - Specify the installation directory. The library (libmfem.a) will be
installed in $(PREFIX)/lib, the headers in $(PREFIX)/include, and
the configuration makefile (config.mk) in $(PREFIX)/share/mfem.
INSTALL - Specify the install program, e.g /usr/bin/install
MFEM library features/options (GNU make)
----------------------------------------
MFEM_USE_MPI = YES/NO
Choose parallel/serial build. The parallel build requires proper setup of the
HYPRE_* and METIS_* library options, see below.
MFEM_USE_METIS = YES/NO
Enable/disable the use of the METIS library. By default, this option is set
to the value of MFEM_USE_MPI. If this option is explicitly disabled in a
parallel build, then the only parallel partitioning (domain decomposition)
option in the library will be Cartesian partitioning with box meshes, and
thus most of the parallel examples and miniapps will fail.
MFEM_PRECISION = double/Double/DOUBLE/single/Single/SINGLE
Use single (float type) or double floating-point precision. In the
configuration header 'config/_config.hpp' this option is represented by
defining exactly one of the macros: MFEM_USE_DOUBLE, or MFEM_USE_SINGLE.
In the exported config files 'config.mk' and 'MFEMConfig.cmake', the option
is represented by the variables MFEM_USE_DOUBLE and MFEM_USE_SINGLE defined
as YES/NO (make) or ON/OFF (cmake). For more details see
https://github.com/orgs/mfem/discussions/4207
MFEM_DEBUG = YES/NO
Choose debug/optimized build. The debug build enables a number of messages
and consistency checks that may simplify bug-hunting.
MFEM_USE_EXCEPTIONS = YES/NO
Enable the use of exceptions. In particular, modifies the default behavior
when errors are encountered: throw an exception, instead of aborting.
MFEM_USE_LIBUNWIND = YES/NO
Use libunwind to print a stacktrace whenever mfem_error is raised. The
information printed is enough to determine the line numbers where the
error originated, provided MFEM_DEBUG=YES or build flags include `-g'.
MFEM_USE_METIS_5 = YES/NO
Specify the version of the METIS library - 5 (YES) or 4 (NO).
MFEM_USE_LAPACK = YES/NO
Use LAPACK routines for various dense linear algebra operations. When
enabled, this option uses the LAPACK_* library options, see below. (When not
enabled MFEM provides simple internal implementations where appropriate.)
MFEM_THREAD_SAFE = YES/NO
Use thread-safe implementation for some classes/methods. This comes at the
cost of extra memory allocation and de-allocation.
MFEM_USE_LEGACY_OPENMP = YES/NO
Enable (basic) experimental OpenMP support. Requires MFEM_THREAD_SAFE.
This option is deprecated.
MFEM_USE_OPENMP = YES/NO
Enable the OpenMP backend.
MFEM_USE_MEMALLOC = YES/NO
Internal MFEM option: enable batch allocation for some small objects.
Recommended value is YES.
MFEM_TIMER_TYPE = 0/1/2/3/4/5/6/NO
Specify which library functions to use in the class StopWatch used for
measuring time. The available options are:
0 - use std::clock from <ctime>, standard C++
1 - use times from <sys/times.h>
2 - use high-resolution POSIX clocks (see option POSIX_CLOCKS_LIB)
3 - use QueryPerformanceCounter from <windows.h>
4 - use mach_absolute_time from <mach/mach_time.h> + std::clock (Mac)
5 - use gettimeofday from <sys/time.h>
6 - use MPI_Wtime from <mpi.h>
NO - use option 3 if the compiler macro _WIN32 is defined, 0 otherwise
MFEM_USE_SUNDIALS = YES/NO
Enable MFEM time integrators and non-linear solvers based on the SUNDIALS
library. When enabled, this option uses the SUNDIALS_* library options,
see below.
MFEM_USE_SUITESPARSE = YES/NO
Enable MFEM functionality based on the SuiteSparse library. Currently, this
option adds the classes UMFPackSolver and KLUSolver (both sparse serial
direct solvers). When enabled, this option uses the SUITESPARSE_* library
options, see below.
MFEM_USE_SUPERLU = YES/NO
Enable MFEM functionality based on the SuperLU_DIST library. Currently, this
option adds the classes SuperLUSolver (a parallel sparse direct solver) and
SuperLURowLocMatrix a distributed CSR matrix class needed by SuperLU. When
enabled, this option uses the SUPERLU_* library options, see below.
MFEM_USE_SUPERLU5 = YES/NO
If SuperLU functionality is enabled, use the older 5.1.0 version rather than
the more recent 6+ versions.
MFEM_USE_MUMPS = YES/NO
Enable MFEM functionality based on the MUMPS library. Currently, this
option adds the class MUMPSSolver (a parallel sparse direct solver).
When enabled, this option uses the MUMPS_* library options, see below.
MFEM_USE_STRUMPACK = YES/NO
Enable MFEM functionality based on the STRUMPACK sparse direct solver and
preconditioner through the STRUMPACKSolver and STRUMPACKRowLocMatrix
classes. When enabled, this option uses the STRUMPACK_* library options, see
below.
MFEM_USE_GINKGO = YES/NO
Enable MFEM functionality based on the Ginkgo library, which provides
iterative linear solvers and preconditioners with OpenMP, CUDA backends, see
https://github.com/ginkgo-project/ginkgo. When enabled, the user can use
Ginkgo's solvers and preconditioners as shown in examples/ginkgo/.
MFEM_USE_AMGX = YES/NO
Enable MFEM functionality based on the AmgX multigrid library from NVIDIA.
Allows the user to use SparseMatrices and HypreParMatrices to solve linear
systems with the routines from the AmgX library.
MFEM_USE_GNUTLS = YES/NO
Enable secure socket support in class socketstream, using the auxiliary
GnuTLS_* classes, based on the GnuTLS library. This option may be useful in
multi-user environment to prevent users from sending/receiving visualization
data to/from other users. When this option is enabled, the default behavior
in class socketstream is to use secure sockets, e.g. when connecting to a
GLVis visualization server. In order for this to work, one needs to generate
GLVis server/client key pairs (in ~/.config/glvis), similar to ssh keys --
the script 'glvis-keygen.sh' in the main GLVis directory can be used to do
that:
bash glvis-keygen.sh ["Your Name"] ["Your Email"]
In MFEM v3.3.2 and earlier, the secure authentication is based on OpenPGP
keys, while later versions use X.509 certificates. The latest version of the
script 'glvis-keygen.sh' can be used to generate both types of keys.
When MFEM_USE_GNUTLS is enabled, the additional build options, GNUTLS_*, are
also used, see below.
MFEM_USE_NETCDF = YES/NO
NetCDF is the library that is used by the SNL Cubit mesh generator to create
Genesis mesh files. This option enables a reader for these files, which
requires that NetCDF be installed, see the NETCDF_* build options below.
MFEM_USE_PETSC = YES/NO
Enable MFEM linear and non-linear solvers, preconditioners, time integrators
and other features based on the PETSc package. When enabled, this option uses
the PETSC_* library options, see below.
MFEM_USE_SLEPC = YES/NO
Enable MFEM eigensolvers based on the SLEPc package. When enabled, this
option uses the SLEPC_* library options, see below.
MFEM_USE_MPFR = YES/NO
MPFR is a library for multiple-precision floating-point computations. This
option enables the use of MPFR in MFEM, e.g. for precise computation of 1D
quadrature rules. When enabled, this option uses the MPFR_* library options,
see below.
MFEM_USE_SIDRE = YES/NO
Sidre is a component of LLNL's axom project, https://github.com/LLNL/axom,
that provides an HDF5-based file format for visualization or restart
capability following the Conduit (https://github.com/LLNL/conduit) mesh
blueprint specification. When enabled, this option requires installation of
HDF5 (see also MFEM_USE_NETCDF), Conduit and LLNL's axom project.
MFEM_USE_SIMD = YES/NO
Enables the high performance templated classes to use architecture dependent
SIMD intrinsics instead of the generic implementation of class AutoSIMD in
linalg/simd/auto.hpp. This option should be combined with suitable
compiler options, such as -march=native, to enable optimal vectorization.
MFEM_USE_CONDUIT = YES/NO
Enables support for converting MFEM Mesh and Grid Function objects to and
from Conduit Mesh Blueprint Descriptions (https://github.com/LLNL/conduit/)
and support for JSON and Binary I/O via Conduit Relay. This option requires
an installation of Conduit. If Conduit was built with HDF5 support, it also
requires an installation of HDF5 (see also MFEM_USE_NETCDF).
MFEM_USE_ADIOS2 = YES/NO
Enables support for ADIOS2, version 2 of the adaptable input output system
for scientific data management. In MFEM, ADIOS2 provides parallel I/O with
ParaView visualization.
MFEM_USE_ZLIB = YES/NO
Enables use of on-the-fly gzip compressed streams. With this feature enabled
(YES), MFEM can compress its output files on-the-fly. In addition, it can
read back files compressed with zlib (or any compression utility capable
of creating a gzip-compatible output such as gzip).
MFEM will write compressed files if the mode argument in the constructor
includes a 'z' character. With this feature disabled (NO), MFEM will not be
able to properly read an input file if it is gzip compressed. In that case,
the solution is to uncompress the file with an external tool (such as gunzip)
before attempting to use it with MFEM.
When enabled, this option uses the ZLIB_* library options, see below.
MFEM_USE_PUMI = YES/NO
Enable the usage of PUMI (https://scorec.rpi.edu/pumi/) in MFEM. The Parallel
Unstructured Mesh Infrastructure (PUMI) is an unstructured, distributed mesh
data management system that is capable of handling general non-manifold
models and effectively supports automated adaptive analysis. PUMI enables
support for parallel unstructured mesh modifications in MFEM.
The develop branch of PUMI repository (https://github.com/SCOREC/core)
should be used for most updated features.
MFEM_USE_UMPIRE = YES/NO
Enables support for Umpire, a resource management library that allows the
discovery, provision, and management of memory on machines with multiple
memory devices like NUMA and GPUs.
MFEM_USE_BENCHMARK = YES/NO
Enables support for Google Benchmark, a library to support the benchmarking
of functions, in the tests/benchmarks directory.
MFEM_USE_HIOP = YES/NO
Enable the usage of HiOp (https://github.com/LLNL/hiop) in MFEM. HiOp is an
HPC solver for nonlinear optimization problems.
MFEM_USE_CODIPACK = YES/NO
Enable automatic differentiation using the CoDiPack library.
www.scicomp.uni-kl.de/codi/
MFEM_USE_ALGOIM = YES/NO
Enable the usage of Algoim - a collection of high-order accurate numerical
methods and C++ algorithms for working with implicitly-defined geometry and
level set methods. The Algoim library requires the Blitz++ library. The MFEM
provides interface to Algoim v1. Thus, to check out the specific state use:
git checkout 9c9ca0ef094d8ab0390ed36367a1151b459bbe0a
https://algoim.github.io
MFEM_USE_ADFORWARD = YES/NO
Enable forward mode for AD packages. This option is valid
only if the AD package supports two modes (backward/forward).
MFEM_USE_CUDA = YES/NO
Enables support for CUDA devices in MFEM. CUDA is a parallel computing
platform and programming model for general computing on graphical processing
units (GPUs). The variable CUDA_ARCH is used to specify the CUDA compute
capability used during compilation (by default, CUDA_ARCH=sm_60). When
enabled, this option uses the CUDA_* build options, see below.
MFEM_USE_HIP = YES/NO
Enables support for AMD devices in MFEM. HIP is a heterogeneous-compute
interface for portability developed by AMD that can target both AMD and
NVIDIA GPUs. The variable HIP_ARCH is used to specify the AMD GPU processor
used during compilation (by default, HIP_ARCH=gfx900). When enabled, this
option uses the HIP_* build options, see below.
MFEM_USE_RAJA = YES/NO
Enable support for the RAJA performance portability layer in MFEM. RAJA
provides a portable abstraction for loops, supporting different programming
model backends. When using RAJA built with CUDA support, CUDA support must be
also enabled in MFEM, i.e. MFEM_USE_CUDA=YES must be set.
MFEM_USE_OCCA = YES/NO
Enables support for the OCCA library in MFEM. OCCA is an open-source library
which aims to make it easy to program different types of devices (e.g. CPU,
GPU, FPGA) by providing an unified API for interacting with JIT-compiled
backends. In order to use the OCCA CUDA backend, CUDA support must be enabled
in MFEM as well, i.e. MFEM_USE_CUDA=YES must be set.
MFEM_USE_GSLIB = YES/NO
Enables MFEM functionality based on the GSLIB library, and specifically its
FindPoints component, which provides a robust algorithms to evaluate finite
element functions in a collection of points in physical space. When enabled,
the user can use the GSLIB-FindPoints methods as shown in miniapps/gslib.
MFEM_USE_CEED = YES/NO
Enables support for the libCEED library in MFEM. libCEED is a portable
library for performant high-order operator evaluation developed by the Center
for Efficient Exascale Discretizations in the Exascale Computing Project.
MFEM_USE_MKL_CPARDISO = YES/NO
Enables the interface to MKL CPardiso: the Intel MKL Parallel Direct Sparse
Solver for Clusters. Make sure to set the correct values for MKL_MPI_WRAPPER
and MKL_LIBRARY_SUBDIR as shown in defaults.mk. If you configure MFEM with
MFEM_USE_LAPACK=YES, verify that the MKL LAPACK libraries are used. The
OpenMP capabilities are disabled at link time.
MFEM_USE_MOONOLITH = YES/NO
Enables the ParMoonolith interface for parallel non-conforming, non-matching,
variational, volumetric mesh information transfer. It requires the variable
MOONOLITH_DIR=<path to installation> to be defined in the environment in
order to be used with the Makefile. Makefile users are also required to
install moonolith using the command `make install_all`, see
https://bitbucket.org/zulianp/par_moonolith for details.
Although Moonolith is an MPI-based library, both serial (MFEM_USE_MPI=NO) and
parallel (MFEM_USE_MPI=YES) versions of MFEM are supported.
MFEM_USE_CALIPER = YES/NO
Enables the interface to Caliper. Caliper is a library to integrate
performance profiling capabilities into applications. To use Caliper,
developers mark code regions of interest using either Caliper's annotation
API or their equivalent in MFEM. Applications can then enable performance
profiling at runtime with Caliper's configuration API. Alternatively, one
can configure Caliper through environment variables or config files.
MFEM_USE_FMS = YES/NO
Enables support for the FMS library which consists of the DataCollection
sub-class mfem::FMSDataCollection for I/O in FMS formats, see the header file
fem/fmsdatacollection.hpp. In addition, this option enables in-memory
conversion routines between FMS's FmsDataCollection structure and MFEM's
DataCollection class, see the header file fem/fmsconvert.hpp.
MFEM_USE_PARELAG = YES/NO
Enables the miniapps that use the ParELAG library. MFEM does not currently
use ParELAG. In fact, ParELAG is dependent on MFEM. Therefore, this option
currently only concerns the miniapps.
MFEM_USE_TRIBOL = YES/NO
Enables the miniapps that use the Tribol library. MFEM does not currently
use Tribol. In fact, Tribol is dependent on MFEM. Therefore, this option
currently only concerns the miniapps.
MFEM_USE_ENZYME = YES/NO
Enables automatic differentiation support through the LLVM plugin Enzyme.
This requires the compiler to be set to clang (>=14.0.0). We also advise to
use the link time optimization (LTO) plugin, to enable functions that you
define over multiple files (compilation units) and want to be differentiated
automatically, to work. This requires to also use LLVM/LLD for linking.
Recommended options are in config/defaults.mk.
MFEM_BUILD_TAG = (any value)
An optional tag to characterize the build. Exported to config/config.mk.
Can be used to identify the MFEM build from other makefiles.
VERBOSE = YES/NO
Print some informational messages when building.
External libraries (GNU make):
------------------------------
Two types of library configuration options are used:
<LIBNAME>_OPT - for compiler options which usually specify an include path,
e.g.: -I/home/user/hypre/include
<LIBNAME>_LIB - for link options which usually specify link path and library
name, e.g.: -L/home/user/hypre/lib -lHYPRE
If specifying relative paths, they should be relative to the top-level MFEM
directory and use the string @MFEM_DIR@, e.g. HYPRE_OPT = -I@MFEM_DIR@/../hypre.
The specific libraries and their options are:
- HYPRE, required for the parallel build, i.e. when MFEM_USE_MPI = YES.
See also the "Specific options for hypre" section at the end of this file.
URL: https://github.com/hypre-space/hypre and https://www.llnl.gov/casc/hypre
Options: HYPRE_OPT, HYPRE_LIB.
Versions: HYPRE >= 2.10.0b (HYPRE built without CUDA)
HYPRE >= 2.20.0 (HYPRE built with '--enable-mixedint')
HYPRE >= 2.22.1 (HYPRE built with CUDA)
HYPRE >= 2.23.0 (HYPRE built with HIP)
HYPRE >= 2.31.0 (runtime selectable HYPRE execution on CPU/GPU)
- METIS, used when MFEM_USE_METIS = YES. If using METIS 5, set
MFEM_USE_METIS_5 = YES (default is to use METIS 4). For building instructions,
see the following:
- METIS 4.0.3: https://mfem.org/building/#parallel-mpi-version-of-mfem
- METIS 5.1.0: https://mfem.org/building/#parallel-build-using-metis-5
URL: https://github.com/mfem/tpls (MFEM mirror, see above)
Options: METIS_OPT, METIS_LIB.
Versions: METIS 4.0.3 or 5.1.0.
- LAPACK (optional), used when MFEM_USE_LAPACK = YES. Alternative, optimized
implementations can also be used, e.g. the ATLAS project.
URL: http://www.netlib.org/lapack (LAPACK)
http://math-atlas.sourceforge.net (ATLAS)
Options: LAPACK_OPT (currently not used/needed), LAPACK_LIB.
- OpenMP (optional), usually part of compiler, used when either MFEM_USE_OPENMP
or MFEM_USE_LEGACY_OPENMP is set to YES.
Options: OPENMP_OPT, OPENMP_LIB.
- High-resolution POSIX clocks: when using MFEM_TIMER_TYPE = 2, it may be
necessary to link with a system library (e.g. librt.so).
Option: POSIX_CLOCKS_LIB (default = -lrt).
- SUNDIALS (optional), used when MFEM_USE_SUNDIALS = YES.
Beginning with MFEM v3.3, SUNDIALS v2.7.0 is supported.
Beginning with MFEM v3.3.2, SUNDIALS v3.0.0 is also supported.
Beginning with MFEM v4.1, only SUNDIALS v5.0.0+ is supported.
When MFEM_USE_CUDA is enabled, only SUNDIALS v5.4.0+ is supported.
If MFEM_USE_MPI is enabled, we expect that SUNDIALS is built with support for
both MPI and hypre.
If MFEM_USE_CUDA is enabled, we expect that SUNDIALS is built with support
for CUDA.
If MFEM_USE_HIP is enabled, we expect that SUNDIALS is built with support
for HIP.
URL: http://computing.llnl.gov/projects/sundials/sundials-software
Options: SUNDIALS_OPT, SUNDIALS_LIB.
Versions: SUNDIALS >= 5.0.0,
SUNDIALS >= 5.4.0 for CUDA support, and
SUNDIALS >= 5.7.0 for HIP support.
- SuiteSparse (optional), used when MFEM_USE_SUITESPARSE = YES.
URL: http://faculty.cse.tamu.edu/davis/suitesparse.html
Options: SUITESPARSE_OPT, SUITESPARSE_LIB.
Versions: SuiteSparse >= 4.5.4, older versions may work too.
- SuperLU_DIST (optional), used when MFEM_USE_SUPERLU = YES. Note that
SuperLU_DIST requires ParMETIS, which includes METIS 5 in its distribution.
Both ParMETIS and the included METIS 5 should be built and installed in the
same location. If using SuperLU_Dist v5, set MFEM_USE_SUPERLU5=YES.
URL: http://crd-legacy.lbl.gov/~xiaoye/SuperLU
Options: SUPERLU_OPT, SUPERLU_LIB.
Versions: SuperLU_DIST >= 5.1.0.
- MUMPS (optional), used when MFEM_USE_MUMPS = YES. Note that MUMPS
requires LAPACK, SCALAPACK and a reordering package such as PORD or METIS.
URL: http://mumps.enseeiht.fr
Options: MUMPS_OPT, MUMPS_LIB.
Versions: MUMPS >= 5.1.1
- STRUMPACK (optional), used when MFEM_USE_STRUMPACK = YES. Note that STRUMPACK
requires the PT-Scotch and Scalapack libraries as well as ParMETIS, which
includes METIS 5 in its distribution. Starting with STRUMPACK v2.2.0, ParMETIS
and PT-Scotch are optional dependencies.
The support for STRUMPACK was added in MFEM v3.3.2.
URL: http://portal.nersc.gov/project/sparse/strumpack
Options: STRUMPACK_OPT, STRUMPACK_LIB.
Versions: STRUMPACK >= 3.0.0.
- Ginkgo (optional), used when MFEM_USE_GINKGO = YES. Note that Ginkgo needs a
C++ compiler that supports the C++-14 standard. For additional requirements
and dependencies of specific modules, see the Ginkgo webpage below.
URL: https://ginkgo-project.github.io
Options: GINKGO_OPT, GINKGO_LIB, GINKGO_DIR, GINKGO_BUILD_TYPE (Release or
Debug).
Versions: Ginkgo >= 1.4.0.
- AmgX (optional), used when MFEM_USE_AMGX = YES.
URL: https://github.com/NVIDIA/AMGX
Options: AMGX_OPT, AMGX_LIB.
Versions: AmgX >= 2.1, older versions may work too.
- GnuTLS (optional), used when MFEM_USE_GNUTLS = YES. On most Linux systems,
GnuTLS is available as a development package, e.g. gnutls-devel. On Mac OS X,
one can get the library through the Homebrew package manager (http://brew.sh).
URL: http://gnutls.org
Options: GNUTLS_OPT, GNUTLS_LIB.
Versions: GnuTLS >= 2.12.0, older versions may work too.
- NetCDF (optional), used when MFEM_USE_NETCDF = YES, required for reading Cubit
mesh files. Also requires installation of HDF5 and ZLIB, as explained at the
NetCDF web site. Note that we use the plain vanilla "C" version of NetCDF, you
don't need the C++ or parallel versions.
URL: www.unidata.ucar.edu/software/netcdf
Options: NETCDF_OPT, NETCDF_LIB.
Versions: NetCDF >= 4.4.0.
- PETSc (optional), used when MFEM_USE_PETSC = YES. Version 3.21 or higher of
the PETSC dev branch is required, though depending on the functionality older
versions may work too. The MFEM and PETSc builds can share common libraries,
e.g., hypre and SUNDIALS. Here's an example configuration, assuming
PETSc has been cloned on the same level as mfem and hypre:
./configure --download-fblaslapack=yes --download-scalapack=yes \
--download-mumps=yes --download-suitesparse=yes \
--with-hypre-dir=../hypre/src/hypre \
--with-shared-libraries=0
When building PETSc with HIP, one may need to add a flag like -std=c2x to
CFLAGS to allow proper parsing of the hipsparse header under C.
URL: https://www.mcs.anl.gov/petsc
Options: PETSC_OPT, PETSC_LIB.
Versions: PETSc >= 3.21.0, older versions may work too.
- SLEPc (optional), used when MFEM_USE_SLEPC = YES. SLEPc depends on PETSc and
uses some of the PETSc options when compiled.
URL: https://slepc.upv.es/
Options: SLEPC_OPT, SLEPC_LIB.
Versions: SLEPc >= 3.8.0.
- Sidre (optional), part of LLNL's axom project, used when MFEM_USE_SIDRE = YES.
Starting with MFEM v4.1, Axom version 0.3.1 or later is required.
URL: https://github.com/LLNL/axom
https://github.com/LLNL/conduit (Conduit)
https://support.hdfgroup.org/HDF5 (HDF5)
Options: SIDRE_OPT, SIDRE_LIB.
Versions: Axom >= 0.3.1.
- Conduit (optional), used when MFEM_USE_CONDUIT = YES. Conduit Mesh Blueprint
support requires Conduit >= v0.3.1 and VisIt >= v2.13.1 to read the output.
URL: https://github.com/LLNL/conduit (Conduit)
https://support.hdfgroup.org/HDF5 (HDF5)
Options: CONDUIT_OPT, CONDUIT_LIB.
Versions: Conduit >= 0.3.1.
- ADIOS2 (optional) used when MFEM_USE_ADIOS2 = YES.
URL: https://adios2.readthedocs.io/
Versions: ADIOS >= 2.5.0.
- PUMI (optional), used when MFEM_USE_PUMI = YES.
URL: https://scorec.rpi.edu/pumi
https://github.com/SCOREC/core
Options: PUMI_OPT, PUMI_LIB.
Versions: PUMI >= 2.2.6.
- HiOp (optional), used when MFEM_USE_HIOP = YES.
URL: https://github.com/LLNL/hiop
Options: HIOP_OPT, HIOP_LIB.
Versions: HIOP >= 0.4.6.
- CoDiPack (optional), used with MFEM_USE_CODIPACK = YES
URL: https://www.scicomp.uni-kl.de/codi/
Options: CODIPACK_OPT
Versions: 1.9.3
- GSLIB (optional), used when MFEM_USE_GSLIB = YES. The gslib library must be
built prior to the MFEM build, as follows: download gslib-1.0.7, untar it at
the same level as MFEM and create a symbolic link: "ln -s gslib-1.0.7 gslib".
Build gslib in parallel or in serial based on the desired MFEM build: "make
clean; make CC=mpicc" or "make clean; make CC=gcc MPI=0". Build MFEM with
MFEM_USE_GSLIB=YES.
URL: https://github.com/gslib/gslib/archive/v1.0.7.tar.gz
Options: GSLIB_OPT, GSLIB_LIB.
Versions: GSLIB >= 1.0.7.
- ALGOIM (optional), used when MFEM_USE_ALGOIM=YES. The library provides only
headers so it just needs to be downloaded at the same level as MFEM. Download
the specific version we use as:
"git clone https://github.com/algoim/algoim.git;
git checkout 9c9ca0ef094d8ab0390ed36367a1151b459bbe0a"
ALGOIM depends on BLITZ and the library must be built prior to the MFEM build.
Download v1.0.2, untar it at the same level as MFEM and create a symbolic link:
"ln -s blitz-1.0.2 blitz".
Build Blitz using CMake as:
"cmake . -DCMAKE_INSTALL_PREFIX=.; make lib; make install"
URL: https://github.com/blitzpp/blitz/archive/refs/tags/1.0.2.tar.gz
Options: BLITZ_OPT, BLITZ_LIB
Versions: BLITZ = 1.0.2
- MKL CPardiso (optional), used when MFEM_USE_MKL_CPARDISO = YES.
URL: https://software.intel.com/content/www/us/en/develop/tools/math-kernel-library.html
Options: MKL_CPARDISO_OPT, MKL_CPARDISO_LIB.
Versions: Intel MKL >= 2020.
- CUDA (optional), used when MFEM_USE_CUDA = YES.
URL: https://developer.nvidia.com/cuda-toolkit
Options: CUDA_CXX, CUDA_ARCH, CUDA_OPT, CUDA_LIB.
Versions: CUDA >= 10.1.168.
- HIP (optional), used when MFEM_USE_HIP = YES.
URL: https://rocmdocs.amd.com
Options: HIP_CXX, HIP_ARCH, HIP_OPT, HIP_LIB.
- OCCA (optional), used when MFEM_USE_OCCA = YES.
URL: https://libocca.org
Options: OCCA_DIR, OCCA_OPT, OCCA_LIB.
Versions: OCCA >= 1.1.0.
- libCEED (optional), used when MFEM_USE_CEED = YES.
URL: https://github.com/CEED/libCEED
https://ceed.exascaleproject.org/libceed
Options: CEED_DIR, CEED_OPT, CEED_LIB.
Versions: libCEED >= 0.12.
- RAJA (optional), used when MFEM_USE_RAJA = YES.
Beginning with MFEM v4.5.1, only RAJA v2022.10.3+ is supported.
URL: https://github.com/LLNL/RAJA
Options: RAJA_DIR, RAJA_OPT, RAJA_LIB.
Versions: RAJA >= 2022.10.3.
- Moonolith (optional), use when MFEM_USE_MOONOLITH = YES.
URL: https://bitbucket.org/zulianp/par_moonolith
Options: MOONOLITH_DIR
Versions: MOONOLITH >= 1.1.0.
- Caliper (optional), used when MFEM_USE_CALIPER = YES.
URL: https://github.com/LLNL/Caliper
Options: CALIPER_DIR
Versions: CALIPER >= 2.5.0, older versions may work too.
- Umpire, used when MFEM_USE_UMPIRE = YES.
Umpire requires camp when the Umpire version is >= 3.0.0.
URL: https://github.com/LLNL/Umpire
Options: UMPIRE_DIR, UMPIRE_OPT, UMPIRE_LIB.
Versions: Umpire >= 3.0.0.
- Benchmark, used when MFEM_USE_BENCHMARK = YES.
URL: https://github.com/google/benchmark
Options: BENCHMARK_DIR, BENCHMARK_LIB.
Versions: Benchmark >= 1.5.6.
- MPFR (optional), used when MFEM_USE_MPFR = YES.
URL: http://mpfr.org, it depends on the GMP library: https://gmplib.org
Options: MPFR_OPT, MPFR_LIB.
- Libunwind (optional), used when MFEM_USE_LIBUNWIND = YES. The library is
included with OS X (as of version 10.11). On Linux it could be installed with
the libunwind-devel package.
URL: http://www.nongnu.org/libunwind
Options: LIBUNWIND_OPT, LIBUNWIND_LIB.
- ZLIB (optional), used when MFEM_USE_ZLIB = YES, or when MFEM_USE_NETCDF =
YES (in the default settings for NETCDF_OPT and NETCDF_LIB).
URL: https://zlib.net
Options: ZLIB_OPT, ZLIB_LIB.
- FMS (optional), used when MFEM_USE_FMS = YES.
URL: https://github.com/CEED/FMS
Options: FMS_OPT, FMS_LIB.
Versions: FMS >= 0.2.
- ParELAG, used when MFEM_USE_PARELAG = YES.
URL: https://github.com/LLNL/parelag
Options: PARELAG_DIR, PARELAG_OPT, PARELAG_LIB.
- Tribol, used when MFEM_USE_TRIBOL = YES.
URL: https://github.com/LLNL/Tribol
Options: TRIBOL_DIR, TRIBOL_OPT, TRIBOL_LIB.
- Enzyme, used when MFEM_USE_ENZYME = YES. Requires LLVM/Clang >= 14.0.0.
URL: https://github.com/EnzymeAD/Enzyme
Options: ENZYME_DIR, ENZYME_OPT, ENZYME_LIB.
Versions: Enzyme >= v0.0.33.
Building with CMake
===================
The MFEM build system consists of two steps: configuration and compilation.
The configuration step can be used to adjust paths to external libraries,
compilers, flags, etc, similar to any CMake build system. It is performed by
running
mkdir <mfem-build-dir> ; cd <mfem-build-dir>
cmake <mfem-source-dir> [OPTIONS] ...
The OPTIONS are of the form -D<VARIABLE>=<VALUE>, e.g. -DMFEM_USE_MPI=YES.
Detailed description of the configuration options is given below. Alternatively,
the options can be specified with an input file:
cd <mfem-source-dir>/config
cp defaults.cmake user.cmake
(edit user.cmake)
cd <mfem-build-dir>
cmake <mfem-source-dir>
Note that user.cmake, if present, is loaded before defaults.cmake (and thus the
former takes precedence over the latter) and its path/name can be changed with
cmake <mfem-source-dir> -DUSER_CONFIG=<user_config_file>
Debug and optimization options are controlled through the CMake variable
CMAKE_BUILD_TYPE which can be set to standard values like "Debug", and "Release"
(default).
To use a specific generator use the "-G <generator>" option of cmake:
cmake <mfem-source-dir> -G "Xcode"
cmake <mfem-source-dir> -G "Visual Studio 12 2013"
cmake <mfem-source-dir> -G "MinGW Makefiles"
With CMake it is possible to build MFEM as a shared library using the standard
CMake option -DBUILD_SHARED_LIBS=1.
Once configured, the library can be built simply with (assuming a UNIX type
system, where the default is to generate "UNIX Makefiles")
make -j 4
or
cmake --build .
or
cmake --build . --config Release [Visual Studio, Xcode]
The build can be quick-tested by running
make check
or
cmake --build . --target check
or
cmake --build . --config Release --target check [Visual Studio, Xcode]
which will simply compile and run Example 1/1p. For more extensive tests that
check the results from all the serial/parallel MFEM examples and miniapps use:
make exec -j 4
make test
or
cmake --build . --target exec
cmake --build . --target test
or
cmake --build . --config Release --target exec [Visual Studio, Xcode]
cmake --build . --config Release --target RUN_TESTS [Visual Studio, Xcode]
Note that running all the tests may take a while.
Installation prefix can be configured by setting the standard CMake variable
CMAKE_INSTALL_PREFIX. To install the library, use
make install
or
cmake --build . --target install
or
cmake --build . --config Release --target install [Xcode]
cmake --build . --config Release --target INSTALL [Visual Studio]
The library will be installed in <PREFIX>/lib, the headers in <PREFIX>/include,
and the configuration CMake files in <PREFIX>/lib/cmake/mfem.
Configuration variables (CMake)
===============================
See the configuration file config/defaults.cmake for the default settings.
Note: the option MFEM_USE_CUDA requires CMake version 3.8 or newer!
Non-standard CMake variables for compilers:
CXX - If set, overwrite the auto-detected C++ compiler, serial build
MPICXX - If set, overwrite the auto-detected MPI C++ compiler, parallel build
The compiler options for the various build types can be controlled using
standard CMake variables like CMAKE_CXX_FLAGS_RELEASE and CMAKE_CXX_FLAGS_DEBUG.
MFEM library features/options (CMake)
-------------------------------------
The following options are equivalent to the GNU make options with the same name:
[see "MFEM library features/options (GNU make)" above]
MFEM_USE_MPI
MFEM_USE_METIS - Set to ${MFEM_USE_MPI}, can be overwritten.
MFEM_PRECISION
MFEM_USE_LIBUNWIND
MFEM_USE_LAPACK
MFEM_THREAD_SAFE
MFEM_USE_LEGACY_OPENMP
MFEM_USE_OPENMP
MFEM_USE_MEMALLOC
MFEM_TIMER_TYPE - Set automatically, can be overwritten.
MFEM_USE_SUITESPARSE
MFEM_USE_SUPERLU
MFEM_USE_MUMPS
MFEM_USE_STRUMPACK
MFEM_USE_GINKGO
MFEM_USE_AMGX
MFEM_USE_GNUTLS
MFEM_USE_NETCDF
MFEM_USE_MPFR
MFEM_USE_ZLIB
MFEM_USE_PUMI
MFEM_USE_HIOP
MFEM_USE_CODIPACK
MFEM_USE_ADFORWARD
MFEM_USE_CUDA
MFEM_USE_HIP
MFEM_USE_OCCA
MFEM_USE_CEED
MFEM_USE_RAJA
MFEM_USE_UMPIRE
MFEM_USE_SIDRE
MFEM_USE_MOONOLITH
MFEM_USE_CALIPER
MFEM_USE_FMS
MFEM_USE_BENCHMARK
MFEM_USE_PARELAG
MFEM_USE_TRIBOL
MFEM_USE_ENZYME
The following options are CMake specific:
MFEM_ENABLE_TESTING - Enable the ctest framework for testing.
MFEM_ENABLE_EXAMPLES - Build all of the examples by default.
MFEM_ENABLE_MINIAPPS - Build all of the miniapps by default.
External libraries (CMake):
---------------------------
For details about the external libraries, see the "External libraries (GNU
make)" section above.
The MFEM CMake build system provides auto-detection for some packages/libraries,
as listed below. The following configuration options are used/defined:
<LIBNAME>_DIR
Directory to search for <LIBNAME> first. The exact subdirectories
searched, for headers and libraries, are chosen based on <LIBNAME>. If the
library is not found in this location, then standard locations are
searched.
<LIBNAME>_REQUIRED_PACKAGES
Specifies a list of package names that have to be explicitly added when
linking with <LIBNAME> in addition to its main library, e.g. ParMETIS
requires METIS, so we set ParMETIS_REQUIRED_PACKAGES to "METIS", see
defaults.cmake.
<LIBNAME>_INCLUDE_DIRS
Location of the <LIBNAME> headers. Set by auto-detection, if successful.
Can be set explicitly, e.g. if auto-detection fails.
<LIBNAME>_LIBRARIES
List of the <LIBNAME> library files/names/link-options. Set by
auto-detection, if successful. Can be set explicitly, e.g. if
auto-detection fails.
The CMake build system adds auto-detection for the following packages/libraries:
- HYPRE
- METIS - The option MFEM_USE_METIS_5 is auto-detected.
- ParMETIS
- SuiteSparse
- SuperLUDist, STRUMPACK
- Ginkgo
- AMGX
- GNUTLS - Extends the built-in CMake support, to search GNUTLS_DIR as well.
- NETCDF
- MPFR
- LIBUNWIND
- POSIXCLOCKS
- PUMI
- HIOP
- CoDiPack
- OCCA
- RAJA
- UMPIRE
- AXOM - Used when MFEM_USE_SIDRE is enabled
- MOONOLITH
- CALIPER
- FMS
- BENCHMARK
- ParELAG
- Enzyme
The following built-in CMake packages are also used:
- MPI, OpenMP, ZLIB
- LAPACK, BLAS - Both are enabled via MFEM_USE_LAPACK. If auto-detection fails,
set the <LIBNAME>_LIBRARIES option directly; the configuration option
<LIBNAME>_DIR is not supported.
Building without GNU make or CMake
==================================
Before using another build system (e.g. Visual Studio) it is necessary to create
a proper configuration header file, config/config.hpp, using the template from
config/config.hpp.in:
cp config/config.hpp.in config/_config.hpp
The file config/_config.hpp can then be edited to enable desired options. The
MFEM library is simply a combination of all object files obtained by compiling
the .cpp source files in the source directories: general, linalg, mesh, and fem.
Specifying an MPI job launcher
==============================
By default, MFEM will use 'mpirun -np #' to launch any of its parallel tests or
miniapps, where # is the number of MPI tasks. An alternate MPI launcher can be
provided by setting the MFEM_MPIEXEC and MFEM_MPIEXEC_NP config variables.
MFEM will expect the launcher command, plus the command line option to allow it
to specify a number of MPI tasks.
MFEM_MPIEXEC = mpirun # default
MFEM_MPIEXEC_NP = -np # default
MFEM_MPIEXEC = srun # example for platforms using SLURM
MFEM_MPIEXEC_NP = -n # example for platforms using SLURM
Specific options for hypre
==========================
The hypre library has multiple options to define local and global index storage
sizes. By default, all indices are stored as an architecture aware integer. For
most platforms, this will be 32-bit. This limits the maximum number of global
degrees of freedom in a vector or matrix to about 2 billion. In order to solve
larger problems, there are two options:
1. Building hypre with '--enable-bigint' defines the local and global indices to
be 64-bit. This is convenient, but requires more memory than necessary.
2. Building hypre with '--enable-mixedint' defines the local indiced to be
32-bit, while using a 64-bit storage for global indices. This option is
currently tested only in ex1p, and may not work in more general settings.
Reference in New Issue View Git Blame Copy Permalink