Fixing copyright year

Removing partial assembly code from example
Updating other DiffusionIntegrator methods
2021-04-09 15:49:41 -07:00 · 2021-04-09 15:49:26 -07:00 · 2021-04-09 15:48:59 -07:00 · 2021-03-28 12:32:35 -04:00 · 2021-03-28 12:31:36 -04:00 · 2021-03-26 17:03:46 -07:00
203 changed files with 1614 additions and 5242 deletions
@@ -36,7 +36,7 @@ staleLabel: stale
 # Comment to post when marking an issue as stale. Set to `false` to disable
 markComment: >
  :warning: This issue or PR has been automatically marked as stale because it has not
-  had any activity in the last month. *If no activity occurs in the next week, it will 
+  had any activity in the last month. *If no activity occurs in the next week, it will
  be automatically closed.* Thank you for your contributions.

 # Comment to post when closing a stale issue. Set to `false` to disable
@@ -76,7 +76,7 @@ stages:
  - baseline_check
  - baseline_publish

-# The setup job in setup stage don’t rely on MFEM git repo. It prepares a
+# The setup job in setup stage don't rely on MFEM git repo. It prepares a
 # pipeline-wide working directory downloading/updating external repos.
 # TODO: updating tests and tpls is not necessary anymore since pipelines are
 # now using unique directories so repo are never shared with another pipeline.
@@ -226,7 +226,7 @@ setup:
    echo "${BASELINE_TEST}: Differences found, replacement file generated"
    cp ${_base_out} ${CI_PROJECT_DIR}/${ARTIFACTS_DIR}/${_base_out}.txt
  fi
-  # _base_diff won’t even exist if there is no difference.
+  # _base_diff won't even exist if there is no difference.
  if [[ -f ${_base_diff} ]]
  then
    echo "${BASELINE_TEST}: Relevant differences (filtered diff) ..."
@@ -5,21 +5,123 @@
                      | | | | | ||  _||  __/| | | | | |
                      |_| |_| |_||_|   \___||_| |_| |_|

-                               http://mfem.org
+                               https://mfem.org


-Version 4.1.1 (development)
-===========================
+Version 4.2, released on October 30, 2020
+=========================================

-libCEED integration improvements
--------------------------------
- Add support for all types of (scalar) Coefficient.
+High-Performance Computing
+--------------------------
+- Added support for explicit vectorization in the high-performance templated
+  code, which can now take advantage of specific classes on the following
+  architectures:
+  * x86 (SSE/AVX/AVX2/AVX512),
+  * Power8 & Power9 (VSX),
+  * BG/Q (QPX).
+  These are disabled by default, but can be enabled with MFEM_USE_SIMD=YES.
+  See the new file linalg/simd.hpp and the new directory linalg/simd.

- Add support for VectorMassIntegrator and VectorDiffusionIntegrator.
+- Added an Element Assembly mode compatible with GPU device execution for H1 and
+  L2 spaces in the mass, convection, diffusion, transpose, and the face DG trace
+  integrators. See option '-ea' in Example 9. When enabled, this assembly level
+  stores independent dense matrices for the elements, and independent dense
+  matrices for the faces in the DG case.

- Add support for AssemblyLevel::NONE for MassIntegrator, DiffusionIntegrator,
-  VectorMassIntegrator, and VectorDiffusionIntegrator. This level of assembly
-  fully applies on the fly the finite element operator.
+- Added a Full Assembly mode compatible with GPU device execution. This assembly
+  level builds on top of the Element Assembly kernels to compute a global sparse
+  matrix. All integrators supported by element assembly are also supported by
+  full assembly. See the '-fa' option in Example 9.
+
+- Optimized the AMD/HIP kernel support and enabled HIP support in the libCEED
+  integration. This is now available via the "ceed-hip" device backend.
+
+- Improved the libCEED integration to support:
+  * AssemblyLevel::NONE for Mass, Diffusion, VectorMass, and VectorDiffusion
+    Integrators. This level computes the full operator evaluation "on the fly".
+  * VectorMassIntegrator and VectorDiffusionIntegrator.
+  * All types of (scalar) Coefficients.
+
+- Added partial assembly / device support for:
+  * H(div) bilinear forms and VectorFEDivergenceIntegrator.
+  * BlockOperator, see the updated Example 5.
+  * Complex operators, see the updated Example 22.
+  * Chebyshev accelerated polynomial smoother.
+  * Convergent diagonal preconditioning on non-conforming adaptively refined
+    meshes, see Example 6/6p.
+
+- Added CUDA support for:
+  * Sparse matrix-vector multiplication with cuSPARSE,
+  * SUNDIALS ODE integrators, see updated SUNDIALS modification of Example 9/9p.
+
+Linear and nonlinear solvers
+----------------------------
+- Added a new solver class for simple integration with NVIDIA's multigrid
+  library, AmgX. The AmgX class is designed to work as a standalone solver or
+  preconditioner for existing MFEM solvers. It uses MFEM's sparse matrix format
+  for serial runs and the HypreParMatrix format for parallel runs. The new
+  solver may be configured to run with one GPU per MPI rank or with more MPI
+  ranks than GPUs. In the latter case, matrices and vectors are consolidated to
+  ranks communicating with the GPUs and the solution is then broadcasted.
+
+  Although CUDA is required to build, the AmgX support is compatible with the
+  MFEM CPU device configuration. The examples/amgx folder illustrates how to
+  integrate AmgX in existing MFEM applications. The AmgX solver class is
+  partially based on: "AmgXWrapper: An interface between PETSc and the NVIDIA
+  AmgX library", by Pi-Yueh Chuang and Lorena A. Barba, doi:10.21105/joss.00280.
+
+- Added initial support for geometric h- and p-multigrid preconditioners for
+  matrix-based and matrix-free discretizations with basic GPU capability, see
+  Example 26/26p.
+
+- Added support for the CVODES package in SUNDIALS which provides ODE solvers
+  with sensitivity analysis capabilities. See the CVODESSolver class and the new
+  adjoint miniapps in the miniapps/adjoint directory.
+
+- Added an interface to the MKL CPardiso solver, an MPI-parallel sparse direct
+  solver developed by Intel. See Example 11p for an illustration of its usage.
+
+- Added support for the SLEPc eigensolver package, https://slepc.upv.es.
+
+- Upgraded SuperLU interface to use SuperLU_DIST 6.3.1. Added a simple SuperLU
+  example in the new directory examples/superlu.
+
+- Extended the KINSOL (SUNDIALS) nonlinear solver interface to support the
+  Jacobian-free Newton-Krylov method. A usage example is shown in Example 10p.
+
+- Block arrays of parallel matrices can now be merged into a single parallel
+  matrix with the function HypreParMatrixFromBlocks. This could be useful for
+  solving block systems with parallel direct solvers such as STRUMPACK.
+
+- Added wrappers for hypre's flexible GMRES solver and the new parallel ILU
+  preconditioner. The latter requires hypre version 2.19.0 or later.
+
+Discretization improvements
+---------------------------
+- Extended GSLIB-FindPoints integration to support simplices and interpolation
+  of functions from L2, H(div) and H(curl) spaces.
+
+- Added support for computing asymptotic error estimates and convergence rates
+  for the whole de Rham sequence based on the new class ConvergenceStudy and new
+  member methods in GridFunction and ParGridFunction. See the rates.cpp file in
+  the tests/convergence directory for sample usage.
+
+- Extended the GetValue and GetVectorValue methods of GridFunction to support
+  evaluation on boundary elements and, in the continuous field case, arbitrary
+  mesh edges and faces. This requires passing an ElementTransformation argument.
+
+- Added support for matrix-free interpolation and restriction operators between
+  continuous H1 finite element spaces of different order on the same mesh or
+  with the same order on uniformly refined meshes.
+
+- The Coefficient classes based on a C-function pointer (FunctionCoefficient,
+  VectorFunctionCoefficient and MatrixFunctionCoefficient) now use the more
+  general std::function class template. This allows the classes to be backward
+  compatible (i.e. they can still work with C-functions) and, in addition,
+  support any "callable", e.g. lambda functions.
+
+- Non-conforming meshes are now supported with block nonlinear forms. See the
+  updated Example 19/19p.

 Meshing improvements
 --------------------
@@ -28,188 +130,55 @@ Meshing improvements
  always available. The interface has also been improved, see for example the
  Mesh Explorer miniapp.

- Improved Gmsh reader (version 2.2), which now supports both high-order and
-  periodic meshes. Segments, triangles, quadrilaterals, and tetrahedra are
-  supported up to order 10. Wedges and hexahedra are supported up to order 9.
-  For sample periodic meshes, see the periodic*.msh files in the data directory.
-
 - Added support for finite difference-based gradient and Hessian approximation
  in the TMOP mesh optimization algorithms. This improves the accuracy of the
  Hessian for r-adaptivity using discrete fields, and allows use of skewness
  and orientation based metrics.

- Added support for r-adaptivity with more than one discrete field. This allows
-  the user to specify different discrete functions for controlling the
-  size, aspect-ratio, orientation, and skew of elements in the mesh.
+- Improved Gmsh reader (version 2.2), which now supports both high-order and
+  periodic meshes. Segments, triangles, quadrilaterals, and tetrahedra are
+  supported up to order 10. Wedges and hexahedra are supported up to order 9.
+  For sample periodic meshes, see the periodic*.msh files in the data directory.

- Added TMOP capability for approximate tangential mesh relaxation. Added
-  support and examples for using TMOP on mixed meshes.
-
- Added complete action of the TMOP Integrator to account for the spatial
-  derivatives of discrete and analytic targets.
-
- Added support for initialization of (serial) non-conforming meshes. Hanging
+- Added support for construction of (serial) non-conforming meshes. Hanging
  nodes can be marked with Mesh::AddVertexParents when building the mesh with
-  the "init" constructor. The usage is demonstrated in a new meshing miniapp
-  (polar-nc) which generates meshes that are non-conforming from the start.
+  the "init" constructor. The usage is demonstrated in a new meshing miniapp,
+  Polar NC, which generates meshes that are non-conforming from the start.

-Performance improvements
------------------------
- Added support for explicit vectorization in the high-performance templated
-  code, which can now take advantage of specific intrinsics classes on the
-  following architectures:
-    - x86 (SSE/AVX/AVX2/AVX512),
-    - Power8 & Power9 (VSX),
-    - BG/Q (QPX).
-  These are disabled by default, and can be enabled with MFEM_USE_SIMD=YES.
-  See the new file linalg/simd.hpp and the new directory linalg/simd.
+- Added support for r-adaptivity with more than one discrete field. This allows
+  the user to specify different discrete functions for controlling the size,
+  aspect-ratio, orientation, and skew of elements in the mesh.

-Improved GPU capabilities
-------------------------
- Added a new solver class for simple integration with NVIDIA's multigrid
-  library, AmgX. The AmgX class is designed to work as a standalone solver or
-  preconditioner for existing MFEM solvers. It uses MFEM's sparse matrix format
-  for serial runs and the HypreParMatrix format for parallel runs.
-
-  The new solver may be configured to run with one GPU per MPI rank or with more
-  MPI ranks than GPUs. In the latter case, matrices and vectors are consolidated
-  to ranks communicating with the GPUs and the solution is then broadcasted.
-
-  Although CUDA is required to build, the AmgX support is compatible with the
-  MFEM CPU device configuration. The examples/amgx folder illustrates how to
-  integrate AmgX in existing MFEM applications.
-
-  The AmgX solver class is partially based on: "AmgXWrapper: An interface
-  between PETSc and the NVIDIA AmgX library", by Pi-Yueh Chuang and Lorena
-  A. Barba, doi:10.21105/joss.00280.
-
- Added support for Chebyshev accelerated polynomial smoother on GPU.
-
- Optimized AMD/HIP kernel support.
-
- Enabled HIP support in the libCEED integration, which is now available via the
-  "ceed-hip" device backend.
-
- Added a Full Assembly mode compatible with Device kernel execution. This
-  assembly level builds on top of the current Element Assembly kernels to
-  compute a global sparse matrix. All integrators supported by element assembly
-  are also supported by full assembly. See the '-fa' option in Example 9.
-
- Added CUDA support for sparse matrix-vector multiplication with cuSPARSE.
-
- Added support for BlockOperator on GPU. See the updated Example 5.
-
- Added partial assembly and GPU support for complex operators, including the
-  classes ComplexOperator, [Par]ComplexGridFunction, [Par]ComplexLinearForm, and
-  [Par]SesquilinearForm. See the updated Example 22.
-
- Added CUDA support for SUNDIALS ODE integrators. See the updated SUNDIALS
-  modification of Example 9/9p.
-
-Discretization improvements
---------------------------
- Added support for matrix-free interpolation and restriction operators between
-  continuous H1 finite element spaces of different order on the same mesh or
-  with the same order on uniformly refined meshes.
-
- Added support for simplices in GSLIB-FindPoints.
-
- Added support for H1 and L2 element matrix assembly in the mass, convection,
-  diffusion, transpose, and the face DG trace integrators. This is compatible
-  with GPU device execution and is illustrated in Example 9/9p, see the option
-  '-ea'. When enabled, this level of assembly stores independent dense matrices
-  for the elements, and independent dense matrices for the faces in the DG case.
-
- Added new partial assembly kernels for H(div) bilinear forms, as well as
-  VectorFEDivergenceIntegrator.
-
- Improved the documentation of the GridFunction GetValue and GetVectorValue
-  methods. Expanded the GetValue and GetVectorValue methods which accept an
-  ElementTransformation argument to support evaluation on boundary elements
-  and, in the continuous field case, arbitrary mesh edges and faces.
-
- Added new coefficient and vector coefficient classes for QuadratureFunctions.
-  Additionally, new LinearForm integrators were also added which make use of
-  these new QuadratureFunction coefficient classes.
-
- Non-conforming meshes are now supported with block nonlinear forms. See the
-  updated Example 19/19p.
-
- Added support face integrals on the boundaries of NURBS meshes.
-
- Added support for interpolation of functions in L2, H(div) and H(curl)
-  spaces using GSLIB-FindPoints.
-
- Added support for computing asymptotic error estimates and convergence rates
-  for the whole de Rham sequence based on the new class ConvergenceStudy and new
-  member methods in GridFunction and ParGridFunction. See the rates.cpp file in
-  the tests/convergence directory for sample usage.
-
- The C-function based coefficient classes (FunctionCoefficient,
-  VectorFunctionCoefficient, and MatrixFunctionCoefficient) now use the more
-  general std::function class template. This allows the classes to be backward
-  compatible (i.e. they can still work with C-functions) and, in addition,
-  support any "callable", e.g. lambda functions.
-
-Linear and nonlinear solvers
----------------------------
- Added power method to iteratively estimate the largest eigenvalue and the
-  corresponding eigenvector of an operator.
-
- Added initial support for h- and p-multigrid solvers and preconditioners for
-  matrix-based and matrix-free discretizations with basic GPU capability.
-
- Added wrappers for Hypre's flexible GMRES solver and the new parallel ILU
-  preconditioner. The latter requires hypre version 2.19.0 or later.
-
- Added a new IterativeSolverMonitor class that allows to monitor the residual
-  and solution during the solving process of an IterativeSolver after every
-  iteration.
-
- Added support for the CVODES package in SUNDIALS which provides ODE
-  solvers with sensitivity analysis capabilities. See the CVODESSolver
-  class and the new adjoint miniapps below.
-
- Block arrays of parallel matrices can now be merged into a single parallel
-  matrix with the function HypreParMatrixFromBlocks. This could be useful for
-  solving block systems with parallel direct solvers such as STRUMPACK.
-
- In SLISolver, changed the residual inner product from (Br,r) to (Br,Br) so the
-  solver can work with non-SPD preconditioner B.
-
- Added support for the SLEPc eigensolver package.
-
- Added partially assembled convergent diagonal preconditioner for adaptively
-  refined meshes (i.e. non-conforming finite element spaces), see Example 6/6p.
-
- Upgraded SuperLU interface to use SuperLU_DIST 6.3.1. Added a simple SuperLU
-  example in the new directory examples/superlu.
-
- Extended the KINSOL (SUNDIALS) nonlinear solver interface to support the
-  Jacobian-free Newton-Krylov method. A usage example is shown in Example 10p.
-
- Added an interface to the MKL CPardiso solver -- an MPI-parallel sparse direct
-  solver developed by Intel. See Example 11p for an illustration of its usage.
+- Additional TMOP improvements:
+  * Capability for approximate tangential mesh relaxation.
+  * Support and examples for using TMOP on mixed meshes.
+  * Complete integrator action accounting for spatial derivatives of discrete
+    and analytic targets.

 New and updated examples and miniapps
 -------------------------------------
- Added a new example, Example 25/25p, to demonstrate the use of a Perfectly
-  Matched Layer (PML) for the simulation of electromagnetic wave propagation.
-  The example defines and solves several indefinite Maxwell problems.
-
- Added a new Example 26/26p to demonstrate the construction of a matrix-free
-  geometric and p-multigrid preconditioner for the Laplace problem.
-
- Added a new example, Example 27/27p, to demonstrate the enforcement of various
-  boundary conditions with the Laplace operator. The example shows the procedure
-  for applying Dirichlet, Neumann (both homogeneous and inhomogeneous), Robin,
-  and periodic boundary conditions with either H1 or DG discretizations.
-
 - Added a new miniapp, Navier, that solves the time-dependent Navier-Stokes
  equations of incompressible fluid dynamics. See the miniapps/navier directory
  for more details.

- Added a new miniapps/adjoint directory with two miniapps demonstrating how to
+- Added 10 new example codes:
+  * Example 25/25p demonstrates the use of a Perfectly Matched Layer (PML) for
+    electromagnetic wave propagation (indefinite Maxwell).
+  * Example 26/26p shows how to construct matrix-free geometric and p-multigrid
+    preconditioner for the Laplace problem.
+  * Example 27/27p demonstrates the enforcement of Dirichlet, Neumann, Robin,
+    and periodic boundary conditions with either H1 or DG Laplace problems.
+  * Versions of Example 1/1p in examples/amgx demonstrating the use of AmgX,
+    to solve the Laplace problem with AMG preconditioning on GPUs.
+  * A version of Example 11p in examples/petsc demonstrating the use of SLEPc,
+    to solve the Laplace eigenproblem with shift-and-invert transformation.
+  * A version of Example 1 in examples/superlu demonstrating the use of SuperLU
+    to solve the Laplace problem.
+
+- Added a new Field Interpolation miniapp in miniapps/gslib that demonstrates
+  transfer of grid functions between different meshes using GSLIB-FindPoints.
+
+- Added 2 miniapps in the new miniapps/adjoint directory demonstrating how to
  solve adjoint problems in MFEM using the CVODES package in SUNDIALS. Both of
  these miniapps require the MFEM_USE_SUNDIALS configuration option.
  * The cvsRoberts_ASAi_dns miniapp solves a backward adjoint problem for a
@@ -217,82 +186,74 @@ New and updated examples and miniapps
  * The adjoint_advection_diffusion miniapp solves a backward adjoint problem
    for an advection diffusion PDE, evaluating adjoint quadratures in parallel.

- Ported Example 11p to SLEPc, to demonstrate solving the Laplace eigenvalue
-  equation with the shift-and-invert spectral transformation method.
+- Added 4 additional meshing miniapps:
+  * The Minimal Surface miniapp solves Plateau's problem: the Dirichlet problem
+    for the minimal surface equation.
+  * The Twist miniapp demonstrates how to stitch together opposite surfaces of a
+    mesh to create a topologically periodic mesh.
+  * The Trimmer miniapp trims away parts of a mesh based on element attributes.
+  * Polar NC shows the construction of polar non-conforming meshes.

- Added a simple meshing miniapp, Twist, which demonstrates MFEM's strategy of
-  stitching together opposite surfaces of a mesh to create a topologically
-  periodic mesh.
+- Several examples and miniapps were updated to include:
+  * Full and element assembly support in Example 9/9p.
+  * Partial assembly with diagonal preconditioning in Examples 4/4p/5/5p/22/22p.
+  * Diagonal preconditioner in Example 6/6p for partial assembly with AMR.
+  * The option to plot a function in Mesh Explorer.
+  * A new test problem showing a mixed bilinear form for H1, H(curl), H(div) and
+    L2, with partial assembly support in Example 24/24p.
+  * Weak Dirichlet boundary conditions (Nitsche) to the NURBS miniapp.

- Added a new meshing miniapp, Minimal Surface, which solves Plateau's problem:
-  the Dirichlet problem for the minimal surface equation.
+Data management and Visualization
+---------------------------------
+- Added support for ADIOS2 for parallel I/O with ParaView visualization. See
+  Examples 5, 9, 12, 16. The classes adios2stream and ADIOS2DataCollection
+  provide the interface to generate ADIOS2 Binary Pack (BP4) directory datasets.

- Added a new meshing miniapp, Polar NC, which demonstrates the construction of
-  polar non-conforming meshes.
-
- Added partial assembly support to Example 4/4p and Example 5/5p, with diagonal
-  preconditioning.
-
- Added full assembly support in Example 9/9p.
-
- Added a new test problem in Example 24/24p, demonstrating a mixed bilinear
-  form for H1, H(curl), H(div) and L_2, with partial assembly support.
-
- Added weak Dirichlet boundary conditions (Nitsche) to the NURBS miniapp.
-
- Added a simple mesh editing miniapp, Trimmer, which trims away portions of a
-  mesh based on element attributes. Any newly exposed boundary elements are
-  assigned attribute numbers related to the trimmed element attributes.
-
- Added a new miniapp (field-interp) that demonstrates transfer of grid function
-  between different meshes using GSLIB-FindPoints.
-
- Added diagonal preconditioner in Example 6/6p for partial assembly with AMR.
-
- Added device support in Example 5/5p.
-
- Added partial assembly and device support to Example 22/22p, with diagonal
-  preconditioning.
-
- Added the option to plot a function in Mesh Explorer.
+- Added VTU output of boundary elements and attributes and parallel VTU (PVTU)
+  output of parallel meshes for visualization using ParaView.

 Improved testing
 ----------------
- Upgraded the Catch unit test framework from version 1.6.1 to version 2.13.0.
-
 - Added a GitLab pipeline that automates PR testing on supercomputing systems
  and Linux clusters at Lawrence Livermore National Lab (LLNL). This can be
  triggered only by LLNL developers, see .gitlab-ci.yml, the .gitlab directory
  and the updated CONTRIBUTING.md file.

- Add tests for the libCEED integration in MFEM.
+- Added additional testing for convergence, the parallel mesh I/O, and for the
+  libCEED integration in MFEM in the tests/ directory.

- Added testing of the parallel mesh format in tests/par-mesh-format.
+- Upgraded the Catch unit test framework from version 1.6.1 to version 2.13.0.

 Miscellaneous
 -------------
- Added support for ADIOS2 for parallel I/O with ParaView visualization. The
-  classes adios2stream and ADIOS2DataCollection are introduced in mfem as the
-  interfaces to generate ADIOS2 Binary Pack (BP4) directory datasets for the
-  entire spatial and temporal node data. Cell centered data is accessible by
-  ADIOS2 data readers (e.g. Python), but currently not yet implement as of
-  ParaView v5.8.1. In addition, ADIOS2 allows for setting a user-defined number
-  of data substreams/subfiles at scale. See examples 5, 9, 12, 16.
+- Renamed "Backend::DEBUG" to "Backend::DEBUG_DEVICE" to avoid conflicts,
+  as DEBUG is sometimes used as a macro.

- Added VTU output of boundary elements and attributes and parallel VTU (PVTU)
-  output of parallel meshes for visualization using ParaView.
+- Added a new IterativeSolverMonitor class that allows to monitor the residual
+  and solution with an IterativeSolver after every iteration.
+
+- Added power method to iteratively estimate the largest eigenvalue and the
+  corresponding eigenvector of an operator.
+
+- Added support for face integrals on the boundaries of NURBS meshes.
+
+- In SLISolver, changed the residual inner product from (Br,r) to (Br,Br) so the
+  solver can work with non-SPD preconditioner B.
+
+- Added new coefficient and vector coefficient classes for QuadratureFunctions,
+  with new LinearForm integrators which use them.

 - The integration order used in the ComputeLpError and ComputeElementLpError
  methods of class GridFunction has been increased.

- Various other simplifications, extensions, and bugfixes in the code.
-
- Renamed "Backend::DEBUG" to "Backend::DEBUG_DEVICE" to avoid conflicts,
-  as DEBUG is sometimes used as a macro.
-
 - Change the IntegrationRule inside VectorDiffusionIntegrator to use the same
  quadrature as DiffusionIntegrator.

+- The README.html files previously included in several source directories have
+  been removed. Use the corresponding pages at mfem.org instead.
+
+- Various other simplifications, extensions, and bugfixes in the code.
+

 Version 4.1, released on March 10, 2020
 =======================================
@@ -623,7 +584,7 @@ Other meshing improvements
  follows precisely the paper:

     D. Arnold, A. Mukherjee, and L. Pouly, "Locally Adapted Tetrahedral Meshes
-     Using Bisection", SIAM J. Sci. Comput. 22 (2000), 431–448.
+     Using Bisection", SIAM J. Sci. Comput. 22 (2000), 431-448.

  This guarantees that the shape regularity of the elements will be preserved
  under refinement.
@@ -768,7 +729,7 @@ Discretization improvements

 - New specialized time integrators: symplectic integrators of orders 1-4 for
  systems of first order ODEs derived from a Hamiltonian and generalized-alpha
-  ODE solver for the filtered Navier–Stokes equations with stabilization. See
+  ODE solver for the filtered Navier-Stokes equations with stabilization. See
  classes SIASolver and GeneralizedAlphaSolver in linalg/ode.hpp.

 - Inherit finite element classes from the new base class TensorBasisElement,
@@ -1258,7 +1219,7 @@ New and improved linear solvers
  it can only be enabled along with MFEM_USE_MPI. When MFEM is configured with
  MFEM_USE_SUPERLU, one also needs to alter the version of METIS, since SuperLU
  requires ParMETIS (which comes packaged with a serial version of METIS).  See
-  http://http://crd-legacy.lbl.gov/~xiaoye/SuperLU for SuperLU_DIST details.
+  http://crd-legacy.lbl.gov/~xiaoye/SuperLU for SuperLU_DIST details.

 - Added a wrapper for the KLU solver in SuiteSparse see
  http://faculty.cse.tamu.edu/davis/suitesparse.html for details of KLU.
@@ -50,7 +50,7 @@ project(mfem NONE)
 # Current version of MFEM, see also `makefile`.
 #   mfem_VERSION = (string)
 #   MFEM_VERSION = (int)   [automatically derived from mfem_VERSION]
-set(${PROJECT_NAME}_VERSION 4.1.1)
+set(${PROJECT_NAME}_VERSION 4.2.0)

 # Prohibit in-source build
 if (${PROJECT_SOURCE_DIR} STREQUAL ${PROJECT_BINARY_DIR})
@@ -1,12 +1,12 @@
 <p align="center">
-<a href="http://mfem.org/"><img alt="mfem" src="http://mfem.org/img/logo-300.png"></a>
+<a href="https://mfem.org/"><img alt="mfem" src="https://mfem.org/img/logo-300.png"></a>
 </p>

 <p align="center">
 <a href="https://github.com/mfem/mfem/blob/master/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-BSD-brightgreen.svg"></a>
 <a href="https://travis-ci.org/mfem/mfem"><img alt="Build Status" src="https://travis-ci.org/mfem/mfem.svg?branch=master"></a>
 <a href="https://ci.appveyor.com/project/mfem/mfem"><img alt="Build Status" src="https://ci.appveyor.com/api/projects/status/19non9sqm6msi2wy?svg=true"></a>
-<a href="http://mfem.github.io/doxygen/html/index.html"><img alt="Doxygen" src="https://img.shields.io/badge/code-documented-brightgreen.svg"></a>
+<a href="https://mfem.github.io/doxygen/html/index.html"><img alt="Doxygen" src="https://img.shields.io/badge/code-documented-brightgreen.svg"></a>
 </p>


@@ -26,7 +26,7 @@ request (PR) toward the `mfem:master` branch to propose your contribution. If
 you are planning significant code changes or have questions, you may want to
 open an [issue](https://github.com/mfem/mfem/issues) before issuing a PR. In
 addition to technical contributions, we are also interested in your results and
-[simulation images](http://mfem.org/gallery/), which you can share via a pull
+[simulation images](https://mfem.org/gallery/), which you can share via a pull
 request in the [mfem/web](https://github.com/mfem/web) repo.

 See the [Quick Summary](#quick-summary) section for the main highlights of our
@@ -51,7 +51,7 @@ back to them before issuing pull requests:
 Contributing to MFEM requires knowledge of Git and, likely, finite elements. If
 you are new to Git, see the [GitHub learning
 resources](https://help.github.com/articles/git-and-github-learning-resources/).
-To learn more about the finite element method, see our [FEM page](http://mfem.org/fem).
+To learn more about the finite element method, see our [FEM page](https://mfem.org/fem).

 *By submitting a pull request, you are affirming the [Developer's Certificate of
 Origin](#developers-certificate-of-origin-11) at the end of this file.*
@@ -92,18 +92,16 @@ The MFEM source code has the following structure:
  .
  ├── config
  │   └── cmake
-  │       └── modules
+  │       └── ...
  ├── data
  ├── doc
-  │   └── web
-  │       └── examples
  ├── examples
  │   ├── amgx
  │   ├── ginkgo
  │   ├── hiop
  │   ├── petsc
  │   ├── pumi
-  │   └── sundials
+  │   ├── sundials
  |   └── superlu
  ├── fem
  │   └── libceed
@@ -126,9 +124,8 @@ The MFEM source code has the following structure:
      ├── convergence
      ├── par-mesh-format
      ├── scripts
-      ├── unit
-      │   ├── ...
-      └── ...
+      └── unit
+          └── ...
 ```

 #### Main directories and classes
@@ -138,24 +135,24 @@ classes implementing the finite element, mesh and linear algebra concepts
 respectively.

 - The main mesh classes are:
-  + [`Mesh`](http://mfem.github.io/doxygen/html/classmfem_1_1Mesh.html)
-  + [`NCMesh`](http://mfem.github.io/doxygen/html/classmfem_1_1NCMesh.html)
-  + [`Element`](http://mfem.github.io/doxygen/html/classmfem_1_1Element.html)
-  + [`ElementTransformation`](http://mfem.github.io/doxygen/html/classmfem_1_1ElementTransformation.html)
+  + [`Mesh`](https://mfem.github.io/doxygen/html/classmfem_1_1Mesh.html)
+  + [`NCMesh`](https://mfem.github.io/doxygen/html/classmfem_1_1NCMesh.html)
+  + [`Element`](https://mfem.github.io/doxygen/html/classmfem_1_1Element.html)
+  + [`ElementTransformation`](https://mfem.github.io/doxygen/html/classmfem_1_1ElementTransformation.html)

 - The main finite element classes are:
-  + [`FiniteElement`](http://mfem.github.io/doxygen/html/classmfem_1_1FiniteElement.html)
-  + [`FiniteElementCollection`](http://mfem.github.io/doxygen/html/classmfem_1_1FiniteElement.html)
-  + [`FiniteElementSpace`](http://mfem.github.io/doxygen/html/classmfem_1_1FiniteElementSpace.html)
-  + [`GridFunction`](http://mfem.github.io/doxygen/html/classmfem_1_1GridFunction.html)
-  + [`BilinearFormIntegrator`](http://mfem.github.io/doxygen/html/classmfem_1_1BilinearFormIntegrator.html) and [`LinearFormIntegrator`](http://mfem.github.io/doxygen/html/classmfem_1_1LinearFormIntegrator.html)
-  + [`LinearForm`](http://mfem.github.io/doxygen/html/classmfem_1_1LinearFormIntegrator.html), [`BilinearForm`](http://mfem.github.io/doxygen/html/classmfem_1_1BilinearForm.html) and [`MixedBilinearForm`](http://mfem.github.io/doxygen/html/classmfem_1_1MixedBilinearForm.html)
+  + [`FiniteElement`](https://mfem.github.io/doxygen/html/classmfem_1_1FiniteElement.html)
+  + [`FiniteElementCollection`](https://mfem.github.io/doxygen/html/classmfem_1_1FiniteElement.html)
+  + [`FiniteElementSpace`](https://mfem.github.io/doxygen/html/classmfem_1_1FiniteElementSpace.html)
+  + [`GridFunction`](https://mfem.github.io/doxygen/html/classmfem_1_1GridFunction.html)
+  + [`BilinearFormIntegrator`](https://mfem.github.io/doxygen/html/classmfem_1_1BilinearFormIntegrator.html) and [`LinearFormIntegrator`](https://mfem.github.io/doxygen/html/classmfem_1_1LinearFormIntegrator.html)
+  + [`LinearForm`](https://mfem.github.io/doxygen/html/classmfem_1_1LinearFormIntegrator.html), [`BilinearForm`](https://mfem.github.io/doxygen/html/classmfem_1_1BilinearForm.html) and [`MixedBilinearForm`](https://mfem.github.io/doxygen/html/classmfem_1_1MixedBilinearForm.html)

 - The main linear algebra classes and sources are
-  + [`Operator`](http://mfem.github.io/doxygen/html/classmfem_1_1Operator.html) and [`BilinearForm`](http://mfem.github.io/doxygen/html/classmfem_1_1BilinearForm.html)
-  + [`Vector`](http://mfem.github.io/doxygen/html/classmfem_1_1BilinearForm.html) and [`LinearForm`](http://mfem.github.io/doxygen/html/classmfem_1_1LinearForm.html)
-  + [`DenseMatrix`](http://mfem.github.io/doxygen/html/classmfem_1_1DenseMatrix.html) and [`SparseMatrix`](http://mfem.github.io/doxygen/html/classmfem_1_1SparseMatrix.html)
-  + Sparse [smoothers](http://mfem.github.io/doxygen/html/sparsesmoothers_8hpp.html) and linear [solvers](http://mfem.github.io/doxygen/html/solvers_8hpp.html)
+  + [`Operator`](https://mfem.github.io/doxygen/html/classmfem_1_1Operator.html) and [`BilinearForm`](https://mfem.github.io/doxygen/html/classmfem_1_1BilinearForm.html)
+  + [`Vector`](https://mfem.github.io/doxygen/html/classmfem_1_1BilinearForm.html) and [`LinearForm`](https://mfem.github.io/doxygen/html/classmfem_1_1LinearForm.html)
+  + [`DenseMatrix`](https://mfem.github.io/doxygen/html/classmfem_1_1DenseMatrix.html) and [`SparseMatrix`](https://mfem.github.io/doxygen/html/classmfem_1_1SparseMatrix.html)
+  + Sparse [smoothers](https://mfem.github.io/doxygen/html/sparsesmoothers_8hpp.html) and linear [solvers](https://mfem.github.io/doxygen/html/solvers_8hpp.html)

 #### Parallel implementation

@@ -165,13 +162,13 @@ shared geometric entities between different tasks. The parallel source files
 have a `p` prefix, e.g. `pmesh.cpp` vs. the serial `mesh.cpp`.

 - The main parallel classes are
-  + [`ParMesh`](http://mfem.github.io/doxygen/html/solvers_8hpp.html)
-  + [`ParNCMesh`](http://mfem.github.io/doxygen/html/classmfem_1_1ParMesh.html)
-  + [`ParFiniteElementSpace`](http://mfem.github.io/doxygen/html/classmfem_1_1ParFiniteElementSpace.html)
-  + [`ParGridFunction`](http://mfem.github.io/doxygen/html/classmfem_1_1ParGridFunction.html)
-  + [`ParBilinearForm`](http://mfem.github.io/doxygen/html/classmfem_1_1ParBilinearForm.html) and [`ParLinearForm`](http://mfem.github.io/doxygen/html/classmfem_1_1ParLinearForm.html)
-  + [`HypreParMatrix`](http://mfem.github.io/doxygen/html/classmfem_1_1HypreParMatrix.html) and [`HypreParVector`](http://mfem.github.io/doxygen/html/classmfem_1_1HypreParVector.html)
-  + [`HypreSolver`](http://mfem.github.io/doxygen/html/classmfem_1_1HypreSolver.html) and other [hypre classes](http://mfem.github.io/doxygen/html/hypre_8hpp.html)
+  + [`ParMesh`](https://mfem.github.io/doxygen/html/solvers_8hpp.html)
+  + [`ParNCMesh`](https://mfem.github.io/doxygen/html/classmfem_1_1ParMesh.html)
+  + [`ParFiniteElementSpace`](https://mfem.github.io/doxygen/html/classmfem_1_1ParFiniteElementSpace.html)
+  + [`ParGridFunction`](https://mfem.github.io/doxygen/html/classmfem_1_1ParGridFunction.html)
+  + [`ParBilinearForm`](https://mfem.github.io/doxygen/html/classmfem_1_1ParBilinearForm.html) and [`ParLinearForm`](https://mfem.github.io/doxygen/html/classmfem_1_1ParLinearForm.html)
+  + [`HypreParMatrix`](https://mfem.github.io/doxygen/html/classmfem_1_1HypreParMatrix.html) and [`HypreParVector`](https://mfem.github.io/doxygen/html/classmfem_1_1HypreParVector.html)
+  + [`HypreSolver`](https://mfem.github.io/doxygen/html/classmfem_1_1HypreSolver.html) and other [hypre classes](https://mfem.github.io/doxygen/html/hypre_8hpp.html)

 #### GPU and general device support

@@ -180,10 +177,10 @@ backends (CUDA, OCCA, RAJA, OpenMP, etc.) and an internal lightweight
 device/host memory manager.

 - The main device-relevant classes and sources are:
-  + [`Device`](http://mfem.github.io/doxygen/html/device_8hpp.html)
-  + [`MemoryManager`](http://mfem.github.io/doxygen/html/mem_manager_8hpp.html)
-  + the [`MFEM_FORALL`](http://mfem.github.io/doxygen/html/forall_8hpp.html) macro
-  + the [`cuda.hpp`](http://mfem.github.io/doxygen/html/cuda_8hpp.html) and [`occa.hpp`](http://mfem.github.io/doxygen/html/occa_8hpp.html) files
+  + [`Device`](https://mfem.github.io/doxygen/html/device_8hpp.html)
+  + [`MemoryManager`](https://mfem.github.io/doxygen/html/mem_manager_8hpp.html)
+  + the [`MFEM_FORALL`](https://mfem.github.io/doxygen/html/forall_8hpp.html) macro
+  + the [`cuda.hpp`](https://mfem.github.io/doxygen/html/cuda_8hpp.html) and [`occa.hpp`](https://mfem.github.io/doxygen/html/occa_8hpp.html) files

 #### Utilities, building and documentation
 - The `general/` directory contains C++ classes that serve as utilities for
@@ -192,7 +189,7 @@ device/host memory manager.
  Makefile and the CMake build options.
 - The `doc/` directory contains configuration for the Doxygen code documentation
  that can either be built locally or browsed online at
-  http://mfem.github.io/doxygen/html/index.html.
+  https://mfem.github.io/doxygen/html/index.html.

 #### Examples and tests
 - `examples` and `miniapps` respectively gather simple and more fully-featured
@@ -201,7 +198,7 @@ device/host memory manager.
 - The `tests/` directory contains a unit test suite and will later contain more
  tests that run example codes.

-See also the [code overview](http://mfem.org/code-overview/) section on the MFEM
+See also the [code overview](https://mfem.org/code-overview/) section on the MFEM
 website.

 ## GitHub Workflow
@@ -441,7 +438,6 @@ Before a PR can be merged, it should satisfy the following:
 - [ ] Run `make unittest` to make sure all unit tests pass.
 - [ ] Run the tests in `tests/scripts`.
 - [ ] (LLNL only) After merging:
-   - [ ] Regenerate `README.html` files from companion documentation pull requests.
   - [ ] Update internal tests to include the new features.

 ### Master/Next Workflow
@@ -514,7 +510,10 @@ MFEM uses a `master`/`next`-branch workflow as described below:
 - [ ] Check that continuous integration server configurations reflect the dependency version requirements of the new release
    - [ ] `.travis.yml`
    - [ ] `.appveyor.yml`
- [ ] (LLNL only) Make sure all `README.html` files in the source repo are up to date.
+- [ ] Update the `CHANGELOG` to organize all release contributions
+- [ ] Review the whole source code once over
+- [ ] Ask MFEM-based applications to test the pre-release branch
+- [ ] Test on additional platforms and compilers
 - [ ] Tag the repository:

  ```
@@ -525,14 +524,13 @@ MFEM uses a `master`/`next`-branch workflow as described below:
 - [ ] Recreate the `next` branch as described in previous section.
 - [ ] Update and push documentation  to `mfem/doxygen`.
 - [ ] Update URL shortlinks:
-    - [ ] Create a shortlink at [http://bit.ly/](http://bit.ly/) for the release tarball, e.g. http://mfem.github.io/releases/mfem-3.1.tgz.
+    - [ ] Create a shortlink at [http://bit.ly/](http://bit.ly/) for the release tarball, e.g. https://mfem.github.io/releases/mfem-3.1.tgz.
    - [ ] (LLNL only) Add and commit the new shortlink in the `links` and `links-mfem` files of the internal `mfem/downloads` repo.
    - [ ] Add the new shortlinks to the MFEM packages in `spack`, `homebrew/science`, `VisIt`, etc.
 - [ ] Update website in `mfem/web` repo:
    - Update version and shortlinks in `src/index.md` and `src/download.md`.
    -  Use [cloc-1.62.pl](http://cloc.sourceforge.net/) and `ls -lh` to estimate the SLOC and the tarball size in `src/download.md`.

-
 ## LLNL Workflow

 ### Mirroring on Bitbucket
@@ -5,7 +5,7 @@
                      | | | | | ||  _||  __/| | | | | |
                      |_| |_| |_||_|   \___||_| |_| |_|

-                               http://mfem.org
+                               https://mfem.org

 The MFEM library has a serial and an MPI-based parallel version, which largely
 share the same code base. The only prerequisite for building the serial version
@@ -19,10 +19,10 @@ requires an MPI C++ compiler, as well as the following external libraries:
  http://glaros.dtc.umn.edu/gkhome/metis/metis/overview

 The hypre dependency can be downloaded as a tarball from GitHub or from the
-project webpage https://www.llnl.gov/casc/hypre. For example, the 2.16.0 release
+project webpage https://www.llnl.gov/casc/hypre. For example, the 2.20.0 release
 of hypre is available at

-   https://github.com/hypre-space/hypre/archive/v2.16.0.tar.gz
+   https://github.com/hypre-space/hypre/archive/v2.20.0.tar.gz

 The METIS dependency can be disabled but that is not generally recommended, see
 the option MFEM_USE_METIS.
@@ -62,7 +62,7 @@ following package managers:

 We also recommend downloading and building the MFEM-based GLVis visualization
 tool which can be used to visualize the meshes and solution in MFEM's examples
-and miniapps. See http://glvis.org and http://mfem.org/building.
+and miniapps. See https://glvis.org and https://mfem.org/building.

 Quick start with GNU make
 =========================
@@ -558,11 +558,12 @@ The specific libraries and their options are:
  for CUDA.
  URL: http://computation.llnl.gov/projects/sundials/sundials-software
  Options: SUNDIALS_OPT, SUNDIALS_LIB.
-  Versions: SUNDIALS >= 5.0.0, SUNDIALS >= 5.4.0 for CUDA suppport.
+  Versions: SUNDIALS >= 5.0.0, SUNDIALS >= 5.4.0 for CUDA support.

 - Mesquite (optional), used when MFEM_USE_MESQUITE = YES.
  URL: http://trilinos.org/oldsite/packages/mesquite
  Options: MESQUITE_OPT, MESQUITE_LIB.
+  The Mesquite support is deprecated and will be removed in the future.

 - SuiteSparse (optional), used when MFEM_USE_SUITESPARSE = YES.
  URL: http://faculty.cse.tamu.edu/davis/suitesparse.html
@@ -572,7 +573,7 @@ The specific libraries and their options are:
 - SuperLU_DIST (optional), used when MFEM_USE_SUPERLU = YES. Note that
  SuperLU_DIST requires ParMETIS, which includes METIS 5 in its distribution.
  Both ParMETIS and the included METIS 5 should be built and installed in the
-  same location.
+  same location. If using SuperLU_Dist v5, set MFEM_USE_SUPERLU5=YES.
  URL: http://crd-legacy.lbl.gov/~xiaoye/SuperLU
  Options: SUPERLU_OPT, SUPERLU_LIB.
  Versions: SuperLU_DIST >= 5.1.0.
@@ -585,7 +586,7 @@ The specific libraries and their options are:
  2.0.0 or later.
  URL: http://portal.nersc.gov/project/sparse/strumpack
  Options: STRUMPACK_OPT, STRUMPACK_LIB.
-  Versions: STRUMPACK >= 3.0.0, requires HYPRE < 2.16.0.
+  Versions: STRUMPACK >= 3.0.0.

 - Ginkgo (optional), used when MFEM_USE_GINKGO = YES. Note that Ginkgo needs a
  C++ compiler that supports the C++-11 standard. For additional requirements
@@ -648,16 +649,18 @@ The specific libraries and their options are:

 - ADIOS2 (optional) used when MFEM_USE_ADIOS2 = YES.
  URL: https://adios2.readthedocs.io/
+  Versions: ADIOS >= 2.5.0.

 - PUMI (optional), used when MFEM_USE_PUMI = YES.
  URL: https://scorec.rpi.edu/pumi
       https://github.com/SCOREC/core
  Options: PUMI_OPT, PUMI_LIB.
-  Versions: PUMI >= 2.2.3.
+  Versions: PUMI == 2.2.3.

 - HiOp (optional), used when MFEM_USE_HIOP = YES.
  URL: https://github.com/LLNL/hiop
  Options: HIOP_OPT, HIOP_LIB.
+  Versions: HIOP >= 0.1.

 - GSLIB (optional), used when MFEM_USE_GSLIB = YES. The gslib library must be
  built prior to the MFEM build, as follows: download gslib-1.0.5, untar it at
@@ -5,7 +5,7 @@
                      | | | | | ||  _||  __/| | | | | |
                      |_| |_| |_||_|   \___||_| |_| |_|

-                               http://mfem.org
+                               https://mfem.org

 MFEM is a modular parallel C++ library for finite element methods. Its goal is
 to enable high-performance scalable finite element discretization research and
@@ -16,8 +16,8 @@ supercomputers.

 * Copyright and licensing information can be found in files LICENSE and NOTICE.

-* The best starting point for new users interested in MFEM's features is the
-  interactive documentation in examples/README.html.
+* The best starting point for new users interested in MFEM's features is to
+  review the examples and miniapps at https://mfem.org/examples.

 * Developers interested in contributing to the library, should read the
  instructions and documentation in the CONTRIBUTING.md file.
@@ -49,17 +49,17 @@ library. Nonlinear solvers (the Newton method), eigensolvers (LOBPCG), and
 several explicit and implicit Runge-Kutta time integrators are also available.

 MFEM supports MPI-based parallelism throughout the library, and can readily be
-used as a scalable unstructured finite element problem generator. As of version
-4.0, MFEM offers initial support for GPU acceleration, and programming models,
-such as CUDA, OCCA, RAJA and OpenMP. MFEM-based applications require minimal
-changes to switch from a serial to a high-performing MPI-parallel version of the
-code, where they can take advantage of the integrated linear solvers from the
-hypre library.  Comprehensive support for other external packages, e.g. PETSc
-and SUNDIALS is also included, giving access to many additional linear and
-nonlinear solvers, preconditioners, time integrators, etc.
+used as a scalable unstructured finite element problem generator. Starting with
+version 4.0, MFEM offers support for GPU acceleration, and programming models,
+such as CUDA, HIP, OCCA, RAJA and OpenMP. MFEM-based applications require
+minimal changes to switch from a serial to a highly-performant MPI-parallel
+version of the code, where they can take advantage of the integrated linear
+solvers from the hypre library. Comprehensive support for other external
+packages, e.g. PETSc, SUNDIALS and libCEED is also included, giving access to
+additional linear and nonlinear solvers, preconditioners, time integrators, etc.

 For examples of using MFEM, see the examples/ and miniapps/ directories, as well
-as the OpenGL visualization tool GLVis which is available at http://glvis.org.
+as the OpenGL visualization tool GLVis which is available at https://glvis.org.

 MFEM is distributed under the terms of the BSD-3 license. All new contributions
 must be made under this license. See LICENSE and NOTICE for details.
@@ -35,6 +35,7 @@ set(MFEM_USE_SUPERLU @MFEM_USE_SUPERLU@)
 set(MFEM_USE_STRUMPACK @MFEM_USE_STRUMPACK@)
 set(MFEM_USE_GINKGO @MFEM_USE_GINKGO@)
 set(MFEM_USE_AMGX @MFEM_USE_AMGX@)
+set(MFEM_USE_HIOP @MFEM_USE_HIOP@)
 set(MFEM_USE_GNUTLS @MFEM_USE_GNUTLS@)
 set(MFEM_USE_GSLIB @MFEM_USE_GSLIB@)
 set(MFEM_USE_NETCDF @MFEM_USE_NETCDF@)
@@ -38,19 +38,19 @@ if(NOT ADIOS2_FOUND)
  endif()

  find_path(ADIOS2_INCLUDE_DIR adios2.h ${ADIOS2_INCLUDE_OPTS})
-  
-  # adios2 version 2.5.0 
+
+  # adios2 version 2.5.0
  find_library(ADIOS2_LIBRARY NAMES adios2 ${ADIOS2_LIBRARY_OPTS})
-  
+
  # adios2 version 2.6.0 and onwards
  if(NOT ADIOS2_LIBRARY)
-  	find_library(ADIOS2_CXX11_MPI_LIBRARY NAMES adios2_cxx11_mpi ${ADIOS2_LIBRARY_OPTS})
-  	find_library(ADIOS2_CXX11_LIBRARY NAMES adios2_cxx11 ${ADIOS2_LIBRARY_OPTS})
-  	set(ADIOS2_LIBRARY ${ADIOS2_CXX11_MPI_LIBRARY} ${ADIOS2_CXX11_LIBRARY})
-  	if(MFEM_USE_MPI)
-  	  add_definitions(-DADIOS2_USE_MPI)
-  	endif()
-  endif() 
+	find_library(ADIOS2_CXX11_MPI_LIBRARY NAMES adios2_cxx11_mpi ${ADIOS2_LIBRARY_OPTS})
+	find_library(ADIOS2_CXX11_LIBRARY NAMES adios2_cxx11 ${ADIOS2_LIBRARY_OPTS})
+	set(ADIOS2_LIBRARY ${ADIOS2_CXX11_MPI_LIBRARY} ${ADIOS2_CXX11_LIBRARY})
+	if(MFEM_USE_MPI)
+	  add_definitions(-DADIOS2_USE_MPI)
+	endif()
+  endif()

  include(FindPackageHandleStandardArgs)
  find_package_handle_standard_args(ADIOS2
@@ -235,7 +235,7 @@ endfunction(mfem_find_component)
 #   code snippets. Additionally, a list of required/optional/alternative
 #   packages (given by ${Name}_REQUIRED_PACKAGES) are searched for and added to
 #   the ${Prefix}_INCLUDE_DIRS and ${Prefix}_LIBRARIES lists. The variable
-#   ${Name}_REQUIRED_LIBRARIES can be set to spcecify any additional libraries
+#   ${Name}_REQUIRED_LIBRARIES can be set to specify any additional libraries
 #   that are needed. This function defines the following CACHE variables:
 #
 #      ${Prefix}_FOUND
@@ -733,7 +733,7 @@ function(mfem_export_mk_files)
    set(shared_link_flag "-Wl,-rpath,")
  endif()

-  # Convert Boolean vars to YES/NO without writting the values to cache
+  # Convert Boolean vars to YES/NO without writing the values to cache
  set(CONFIG_MK_BOOL_VARS MFEM_USE_MPI MFEM_USE_METIS MFEM_USE_METIS_5
      MFEM_DEBUG MFEM_USE_EXCEPTIONS MFEM_USE_ZLIB MFEM_USE_LIBUNWIND
      MFEM_USE_LAPACK MFEM_THREAD_SAFE MFEM_USE_OPENMP MFEM_USE_LEGACY_OPENMP
@@ -820,7 +820,7 @@ function(mfem_export_mk_files)
      string(REGEX REPLACE "^SCOREC::" "" libname ${pumilib})
      string(FIND "${pumilib}" ".a" staticlib)
      string(FIND "${pumilib}" ".so" sharedlib)
-      find_library(lib ${libname} PATHS ${PUMI_DIR}/lib NO_DEFUALT_PATH)
+      find_library(lib ${libname} PATHS ${PUMI_DIR}/lib NO_DEFAULT_PATH)
      if (NOT "${sharedlib}" MATCHES "-1" OR
          NOT "${staticlib}" MATCHES "-1"   )
        set(MFEM_EXT_LIBS "${pumilib} ${MFEM_EXT_LIBS}")
@@ -53,6 +53,8 @@ HIP_CXX = hipcc
 # gfx900, gfx1010, etc.
 HIP_ARCH = gfx900
 HIP_FLAGS = --amdgpu-target=$(HIP_ARCH)
+HIP_XCOMPILER =
+HIP_XLINKER   = -Wl,

 ifneq ($(NOTMAC),)
   AR      = ar
@@ -144,6 +146,16 @@ MFEM_USE_SIMD          = NO
 MFEM_USE_ADIOS2        = NO
 MFEM_USE_MKL_CPARDISO  = NO

+# MPI library compile and link flags
+# These settings are used only when building MFEM with MPI + HIP
+ifeq ($(MFEM_USE_MPI)$(MFEM_USE_HIP),YESYES)
+   # We determine MPI_DIR assuming $(MPICXX) is in $(MPI_DIR)/bin
+   MPI_DIR := $(patsubst %/,%,$(dir $(shell which $(MPICXX))))
+   MPI_DIR := $(patsubst %/,%,$(dir $(MPI_DIR)))
+   MPI_OPT = -I$(MPI_DIR)/include
+   MPI_LIB = -L$(MPI_DIR)/lib $(XLINKER)-rpath,$(MPI_DIR)/lib -lmpi
+endif
+
 # Compile and link options for zlib.
 ZLIB_DIR =
 ZLIB_OPT = $(if $(ZLIB_DIR),-I$(ZLIB_DIR)/include)
@@ -226,7 +238,7 @@ SUITESPARSE_LIB = -Wl,-rpath,$(SUITESPARSE_DIR)/lib -L$(SUITESPARSE_DIR)/lib\
 ifeq ($(MFEM_USE_SUPERLU5),YES)
   SUPERLU_DIR = @MFEM_DIR@/../SuperLU_DIST_5.1.0
   SUPERLU_OPT = -I$(SUPERLU_DIR)/include
-   SUPERLU_LIB = -Wl,-rpath,$(SUPERLU_DIR)/lib -L$(SUPERLU_DIR)/lib -lsuperlu_dist_5.1.0   
+   SUPERLU_LIB = -Wl,-rpath,$(SUPERLU_DIR)/lib -L$(SUPERLU_DIR)/lib -lsuperlu_dist_5.1.0
 else
   SUPERLU_DIR = @MFEM_DIR@/../SuperLU_DIST_6.3.1
   SUPERLU_OPT = -I$(SUPERLU_DIR)/include
@@ -48,6 +48,10 @@ groups_serial=(
   "miniapps/meshing"
   "mobius-strip.cpp klein-bottle.cpp extruder.cpp toroid.cpp
    mesh-optimizer.cpp minimal-surface.cpp"'
+'"convergence"
+   "Convergence tests:"
+   "tests/convergence"
+   "rates.cpp"'
 )
 # Parallel groups
 groups_parallel=(
@@ -81,7 +85,7 @@ groups_parallel=(
 '"convergence"
   "Convergence tests:"
   "tests/convergence"
-   "diffusion.cpp"'
+   "prates.cpp"'
 '"par-mesh-format"
   "Parallel mesh tests:"
   "tests/par-mesh-format"
@@ -118,7 +122,7 @@ groups_all=(
 '"convergence"
   "Convergence tests:"
   "tests/convergence"
-   "diffusion.cpp"'
+   "{,p}rates.cpp"'
 '"par-mesh-format"
   "Parallel mesh tests:"
   "tests/par-mesh-format"
@@ -16,7 +16,6 @@ if (DOXYGEN_FOUND)
  configure_file(${CMAKE_CURRENT_SOURCE_DIR}/CodeDocumentation.conf.in
    ${CMAKE_CURRENT_BINARY_DIR}/CodeDocumentation.conf @ONLY)

-
  add_custom_target(doc
    COMMAND ${DOXYGEN_EXECUTABLE} ${CMAKE_CURRENT_BINARY_DIR}/CodeDocumentation.conf
    COMMAND echo "<meta http-equiv=\"REFRESH\" content=\"0;URL=CodeDocumentation/html/index.html\">" > ${CMAKE_CURRENT_BINARY_DIR}/CodeDocumentation.html
@@ -32,5 +31,4 @@ if (DOXYGEN_FOUND)
    COMMENT "Removing API documentation"
    VERBATIM)

-
 endif (DOXYGEN_FOUND)
@@ -38,7 +38,7 @@ PROJECT_NAME           = "MFEM"
 # could be handy for archiving the generated documentation or if some version
 # control system is used.

-PROJECT_NUMBER         = v4.1.1
+PROJECT_NUMBER         = v4.2.0

 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a
@@ -51,7 +51,7 @@ PROJECT_BRIEF          = "Finite element discretization library"
 # pixels and the maximum width should not exceed 200 pixels. Doxygen will copy
 # the logo to the output directory.

-PROJECT_LOGO           = web/logo-small.png
+PROJECT_LOGO           = logo-small.png

 # The OUTPUT_DIRECTORY tag is used to specify the (relative or absolute) path
 # into which the generated documentation will be written. If a relative path is
@@ -766,11 +766,14 @@ INPUT                  = @MFEM_SOURCE_DIR@/doc/CodeDocumentation.dox \
                         @MFEM_SOURCE_DIR@/mesh \
                         @MFEM_SOURCE_DIR@/fem \
                         @MFEM_SOURCE_DIR@/examples \
+                         @MFEM_SOURCE_DIR@/examples/amgx \
+                         @MFEM_SOURCE_DIR@/examples/ginkgo \
+                         @MFEM_SOURCE_DIR@/examples/hiop \
                         @MFEM_SOURCE_DIR@/examples/petsc \
                         @MFEM_SOURCE_DIR@/examples/pumi \
-                         @MFEM_SOURCE_DIR@/examples/hiop \
                         @MFEM_SOURCE_DIR@/examples/sundials \
-                         @MFEM_SOURCE_DIR@/miniapps/adjoint \			 
+                         @MFEM_SOURCE_DIR@/examples/superlu \
+                         @MFEM_SOURCE_DIR@/miniapps/adjoint \
                         @MFEM_SOURCE_DIR@/miniapps/common \
                         @MFEM_SOURCE_DIR@/miniapps/electromagnetics \
                         @MFEM_SOURCE_DIR@/miniapps/gslib \
@@ -2334,7 +2337,7 @@ PLANTUML_INCLUDE_PATH  =
 # Minimum value: 0, maximum value: 10000, default value: 50.
 # This tag requires that the tag HAVE_DOT is set to YES.

-DOT_GRAPH_MAX_NODES    = 10
+DOT_GRAPH_MAX_NODES    = 50

 # The MAX_DOT_GRAPH_DEPTH tag can be used to set the maximum depth of the graphs
 # generated by dot. A depth value of 3 means that only nodes reachable from the
@@ -62,7 +62,7 @@ namespace mfem {
 * - <a class="el" href="ex9p_8cpp_source.html">Example 9p</a>: parallel Discontinuous Galerkin (DG) time-dependent advection
 * - <a class="el" href="ex10_8cpp_source.html">Example 10</a>: time-dependent implicit nonlinear elasticity
 * - <a class="el" href="ex10p_8cpp_source.html">Example 10p</a>: parallel time-dependent implicit nonlinear elasticity
- * - <a class="el" href="examples_2ex11p_8cpp_source.html">Example 11p</a>: parallel Laplace eigensolver
+ * - <a class="el" href="ex11p_8cpp_source.html">Example 11p</a>: parallel Laplace eigensolver
 * - <a class="el" href="ex12p_8cpp_source.html">Example 12p</a>: parallel linear elasticity eigensolver
 * - <a class="el" href="ex13p_8cpp_source.html">Example 13p</a>: parallel Maxwell eigensolver
 * - <a class="el" href="ex14_8cpp_source.html">Example 14</a>: Discontinuous Galerkin (DG) for the Laplace problem
@@ -90,20 +90,25 @@ namespace mfem {
 * - <a class="el" href="ex25p_8cpp_source.html">Example 25p</a>: parallel simulation of electromagnetic wave propagation using a Perfectly Matched Layer (PML)
 * - <a class="el" href="ex26_8cpp_source.html">Example 26</a>: multigrid preconditioner for the Laplace problem using nodal H1 FEM
 * - <a class="el" href="ex26p_8cpp_source.html">Example 26p</a>: parallel multigrid preconditioner for the Laplace problem using nodal H1 FEM
+ * - <a class="el" href="ex27_8cpp_source.html">Example 27</a>: boundary conditions for the Laplace problem
+ * - <a class="el" href="ex27p_8cpp_source.html">Example 27p</a>: parallel boundary conditions for the Laplace problem
 *
- * <H4>SUNDIALS Examples</H4>
+ * <H4>AmgX Examples</H4>
 * - Variants of Examples
- *   <a class="el" href="sundials_2ex9_8cpp_source.html">9</a>,
- *   <a class="el" href="sundials_2ex9p_8cpp_source.html">9p</a>,
- *   <a class="el" href="sundials_2ex10_8cpp_source.html">10</a>,
- *   <a class="el" href="sundials_2ex10p_8cpp_source.html">10p</a>,
- *   <a class="el" href="sundials_2ex16_8cpp_source.html">16</a>,
- *   and
- *   <a class="el" href="sundials_2ex16p_8cpp_source.html">16p</a>
- *   demonstrating the use of MFEM's \link sundials.hpp SUNDIALS classes\endlink
- * - CVODES adjoint examples:
- *   <a class="el" href="cvsRoberts__ASAi__dns_8cpp_source.html">serial ODE system</a>,
- *   <a class="el" href="adjoint__advection__diffusion_8cpp_source.html">parallel advection-diffusion</a>
+ *   <a class="el" href="examples_2amgx_2ex1_8cpp_source.html">1</a> and
+ *   <a class="el" href="examples_2amgx_2ex1p_8cpp_source.html">1p</a>,
+ *   demonstrating the use of MFEM's \link amgxsolver.hpp AmgX integration\endlink.
+ *
+ * <H4>Ginkgo Examples</H4>
+ * - Variants of Example
+ *   <a class="el" href="examples_2ginkgo_2ex1_8cpp_source.html">1</a>,
+ *   demonstrating the use of MFEM's \link ginkgo.hpp Ginkgo integration\endlink.
+ *
+ * <H4>HiOp Examples</H4>
+ * - Variants of Examples
+ *   <a class="el" href="hiop_2ex9_8cpp_source.html">9</a> and
+ *   <a class="el" href="hiop_2ex9p_8cpp_source.html">9p</a>,
+ *   demonstrating the use of MFEM's \link hiop.hpp HiOp integration\endlink.
 *
 * <H4>PETSc Examples</H4>
 * - Variants of Examples
@@ -115,8 +120,8 @@ namespace mfem {
 *   <a class="el" href="petsc_2ex6p_8cpp_source.html">6p</a>,
 *   <a class="el" href="petsc_2ex9p_8cpp_source.html">9p</a>,
 *   and
- *   <a class="el" href="petsc_2ex10p_8cpp_source.html">10p</a>
- *   demonstrating the use of MFEM's \link petsc.hpp PETSc classes\endlink
+ *   <a class="el" href="petsc_2ex10p_8cpp_source.html">10p</a>,
+ *   demonstrating the use of MFEM's \link petsc.hpp PETSc integration\endlink.
 *
 * <H4>PUMI Examples</H4>
 * - Variants of Examples
@@ -124,19 +129,27 @@ namespace mfem {
 *   <a class="el" href="examples_2pumi_2ex1p_8cpp_source.html">1p</a>,
 *   <a class="el" href="pumi_2ex2_8cpp_source.html">2</a>,
 *   and
- *   <a class="el" href="pumi_2ex6p_8cpp_source.html">6p</a>
- *   demonstrating the use of MFEM's \link pumi.hpp PUMI classes\endlink
+ *   <a class="el" href="pumi_2ex6p_8cpp_source.html">6p</a>,
+ *   demonstrating the use of MFEM's \link pumi.hpp PUMI integration\endlink.
 *
- * <H4>HiOp Examples</H4>
+ * <H4>SUNDIALS Examples</H4>
 * - Variants of Examples
- *   <a class="el" href="hiop_2ex9_8cpp_source.html">9</a> and
- *   <a class="el" href="hiop_2ex9p_8cpp_source.html">9p</a>,
- *   demonstrating the use of MFEM's \link hiop.hpp HiOp classes\endlink
+ *   <a class="el" href="sundials_2ex9_8cpp_source.html">9</a>,
+ *   <a class="el" href="sundials_2ex9p_8cpp_source.html">9p</a>,
+ *   <a class="el" href="sundials_2ex10_8cpp_source.html">10</a>,
+ *   <a class="el" href="sundials_2ex10p_8cpp_source.html">10p</a>,
+ *   <a class="el" href="sundials_2ex16_8cpp_source.html">16</a>,
+ *   and
+ *   <a class="el" href="sundials_2ex16p_8cpp_source.html">16p</a>,
+ *   demonstrating the use of MFEM's \link sundials.hpp SUNDIALS integration\endlink.
+ * - CVODES adjoint miniapps:
+ *   <a class="el" href="cvsRoberts__ASAi__dns_8cpp_source.html">serial ODE system</a>,
+ *   <a class="el" href="adjoint__advection__diffusion_8cpp_source.html">parallel advection-diffusion</a>.
 *
- * <H4>Ginkgo Examples</H4>
+ * <H4>SuperLU Examples</H4>
 * - Variants of Example
- *   <a class="el" href="ginkgo_2ex1_8cpp_source.html">1</a>
- *   demonstrating the use of MFEM's \link ginkgo.hpp Ginkgo classes\endlink
+ *   <a class="el" href="examples_2superlu_2ex1p_8cpp_source.html">1p</a>,
+ *   demonstrating the use of MFEM's \link superlu.hpp SuperLU integration\endlink.
 *
 * <H3>Miniapps</H3>
 * - <a class="el" href="volta_8cpp_source.html">Volta</a>: simple electrostatics simulation code
@@ -162,11 +175,11 @@ namespace mfem {
 * - <a class="el" href="lor-transfer_8cpp_source.html">LOR Transfer</a>: map functions between high-order and low-order refined spaces
 * - <a class="el" href="findpts_8cpp_source.html">Find Points</a>: evaluate grid function in physical space, <a class="el" href="findpts_8cpp_source.html">serial</a> and <a class="el" href="pfindpts_8cpp_source.html">parallel</a> versions
 * - <a class="el" href="field-diff_8cpp_source.html">Field Diff</a>: compare grid functions on different meshes
- * - <a class="el" href="field-interp_8cpp_source.html">Field Interp</a>: transfer a grid functions betwen meshes
+ * - <a class="el" href="field-interp_8cpp_source.html">Field Interp</a>: transfer a grid functions between meshes
 * - <a class="el" href="miniapps_2performance_2ex1_8cpp_source.html">HPC Example 1</a>: high-performance nodal H1 FEM for the Laplace problem
 * - <a class="el" href="miniapps_2performance_2ex1p_8cpp_source.html">HPC Example 1p</a>: high-performance parallel nodal H1 FEM for the Laplace problem
 *
- * See also the <a class="el" href="../../../examples/README.html">README</a> in the <code>examples/</code> directory
+ * See also the <a class="el" href="https://mfem.org/examples/">examples documentation</a> online.
 */

 }
@@ -5,7 +5,7 @@
                      | | | | | ||  _||  __/| | | | | |
                      |_| |_| |_||_|   \___||_| |_| |_|

-                               http://mfem.org
+                               https://mfem.org

 This directory contains Doxygen configuration files for building HTML
 documentation of the MFEM sources (including the example codes and miniapps).
@@ -21,7 +21,7 @@ create the file CodeDocumentation.html, which can be viewed in any web browser.

 Alternatively, the (pre-build) documentation can be browsed directly online at

-   http://mfem.github.io/doxygen/html/index.html
+   https://mfem.github.io/doxygen/html/index.html


 Some building considerations:
@@ -13,7 +13,6 @@ SHELL = /bin/bash
 MFEM_DIR ?= ..
 DOXYGEN_CONF = CodeDocumentation.conf

-
 # doxygen uses: graphviz, latex
 html: $(DOXYGEN_CONF)
 	@# Generate the html documentation
@@ -1,208 +0,0 @@
-body {
-    padding-top: 70px;
-}
-
-/* hack to increase height of headers to make #anchors work with the top nav-bar,
-   (note: we only want this on pages with TOC, otherwise there are other problems) */
-div.col-md-9 h1[id]::before,
-div.col-md-9 h2[id]::before,
-div.col-md-9 h3[id]::before,
-div.col-md-9 h4[id]::before,
-div.col-md-9 h5[id]::before,
-div.col-md-9 h6[id]::before {
-    content: "";
-    display: block;
-    margin-top: -75px;
-    height: 75px;
-}
-
-img {
-    max-width: 100%;
-    height: auto;
-}
-
-pre {
-    overflow-x: auto;
-}
-
-pre code {
-    overflow-wrap: normal;
-    white-space: pre;
-    font-size: 13.5px;
-}
-
-label {
-    font-weight: normal;
-}
-
-ul.nav li.main {
-    font-weight: bold;
-}
-
-div.col-md-3 {
-    padding-right: 0;
-}
-
-div.col-md-9, div.col-md-12 {
-    padding-bottom: 100px;
-}
-
-div.source-links {
-    float: right;
-}
-
-img {
-    display: block;
-    margin-left: auto;
-    margin-right: auto;
-    margin-top: 25px;
-    margin-bottom: 25px;
-}
-
-img.floatright {
-    display: block;
-    float: right;
-    margin-top: 0;
-    margin-left: 25px;
-    margin-bottom: 25px;
-}
-
-img.teamphoto {
-    box-shadow: 4px 4px 4px #999;
-}
-
-p.centered {
-    margin-left: auto;
-    margin-right: auto;
-    text-align: center;
-}
-
-/*label {
-    display: inline-block;
-    margin-bottom: 5px;
-    font-weight: normal;
-}*/
-
-/*
- * Side navigation
- *
- * Scrollspy and affixed enhanced navigation to highlight sections and secondary
- * sections of docs content.
- */
-
-/* By default it's not affixed in mobile views, so undo that */
-.bs-sidebar.affix {
-    position: static;
-}
-
-.bs-sidebar.well {
-    padding: 0;
-}
-
-/* First level of nav */
-.bs-sidenav {
-    /*margin-top: 10px;
-    margin-bottom: 10px;*/
-    padding-top:    10px;
-    padding-bottom: 10px;
-    border-radius: 5px;
-}
-
-/* All levels of nav */
-.bs-sidebar .nav > li > a {
-    display: block;
-    padding: 5px 20px;
-    z-index: 1;
-}
-.bs-sidebar .nav > li > a:hover,
-.bs-sidebar .nav > li > a:focus {
-    text-decoration: none;
-    border-right: 1px solid;
-}
-.bs-sidebar .nav > .active > a,
-.bs-sidebar .nav > .active:hover > a,
-.bs-sidebar .nav > .active:focus > a {
-    font-weight: bold;
-    background-color: transparent;
-    border-right: 1px solid;
-}
-
-/* Nav: second level (shown on .active) */
-.bs-sidebar .nav .nav {
-    display: none; /* Hide by default, but at >768px, show it */
-    margin-bottom: 8px;
-}
-.bs-sidebar .nav .nav > li > a {
-    padding-top:    3px;
-    padding-bottom: 3px;
-    padding-left: 30px;
-    font-size: 90%;
-}
-
-/* Show and affix the side nav when space allows it */
-@media (min-width: 992px) {
-    .bs-sidebar .nav > .active > ul {
-        display: block;
-    }
-    /* Widen the fixed sidebar */
-    .bs-sidebar.affix,
-    .bs-sidebar.affix-bottom {
-        width: 213px;
-    }
-    .bs-sidebar.affix {
-        position: fixed; /* Undo the static from mobile first approach */
-        top: 80px;
-    }
-    .bs-sidebar.affix-bottom {
-        position: absolute; /* Undo the static from mobile first approach */
-    }
-    .bs-sidebar.affix-bottom .bs-sidenav,
-    .bs-sidebar.affix .bs-sidenav {
-        margin-top: 0;
-        margin-bottom: 0;
-    }
-}
-@media (min-width: 1200px) {
-    /* Widen the fixed sidebar again */
-    .bs-sidebar.affix-bottom,
-    .bs-sidebar.affix {
-        width: 263px;
-    }
-}
-
-
-/* Sticky footer styles
-------------------------------------------------- */
-html {
-  position: relative;
-  min-height: 100%;
-}
-body {
-  /* Margin bottom by footer height */
-  margin-bottom: 50px;
-}
-.footer {
-  position: absolute;
-  bottom: 0;
-  width: 100%;
-  /* Set the fixed height of the footer here */
-  height: 50px;
-  background-color: #eeeeee;
-  overflow: hidden;
-}
-
-
-/* Custom page CSS
-------------------------------------------------- */
-/* Not required for template or sticky footer method. */
-
-.container .text-muted {
-  margin-top: 17px;
-  margin-bottom: 0;
-}
-.container .alignleft {
-  float: left;
-}
-.container .alignright {
-  float: right;
-}
@@ -117,19 +117,19 @@ if (MFEM_USE_SUPERLU)
    ${MPIEXEC_POSTFLAGS})
 endif()

-# Include the examples/sundials directory if SUNDIALS is enabled.
-if (MFEM_USE_SUNDIALS)
-  add_subdirectory(sundials)
+# Include the examples/amgx directory if AmgX is enabled
+if (MFEM_USE_AMGX)
+  add_subdirectory(amgx)
 endif()

-# Include the examples/sundials directory if SUNDIALS is enabled.
+# Include the examples/ginkgo directory if GINKGO is enabled.
 if (MFEM_USE_GINKGO)
  add_subdirectory(ginkgo)
 endif()

-# Include the examples/amgx directory if AmgX is enabled
-if (MFEM_USE_AMGX)
-  add_subdirectory(amgx)
+# Include the examples/hiop directory if HiOp is enabled
+if (MFEM_USE_HIOP)
+  add_subdirectory(hiop)
 endif()

 # Include the examples/petsc directory if PETSc is enabled.
@@ -142,6 +142,12 @@ if (MFEM_USE_PUMI)
  add_subdirectory(pumi)
 endif()

-if (MFEM_USE_HIOP)
-  add_subdirectory(hiop)
+# Include the examples/sundials directory if SUNDIALS is enabled.
+if (MFEM_USE_SUNDIALS)
+  add_subdirectory(sundials)
+endif()
+
+# Include the examples/superlu directory if SUPERLU is enabled.
+if (MFEM_USE_SUPERLU)
+  add_subdirectory(superlu)
 endif()
@@ -30,6 +30,10 @@
 #include <fstream>
 #include <iostream>

+#ifndef MFEM_USE_AMGX
+#error This example requires that MFEM is built with MFEM_USE_AMGX=YES
+#endif
+
 using namespace std;
 using namespace mfem;

@@ -31,6 +31,10 @@
 using namespace std;
 using namespace mfem;

+#ifndef MFEM_USE_AMGX
+#error This example requires that MFEM is built with MFEM_USE_AMGX=YES
+#endif
+
 int main(int argc, char *argv[])
 {
   // 1. Initialize MPI.
@@ -403,7 +403,6 @@ VisMan::~VisMan()
   }
 }

-
 ostream &operator<<(ostream &v, void (*f)(VisMan&))
 {
   VisMan *vp = dynamic_cast<VisMan*>(&v);
@@ -441,7 +441,6 @@ VisMan::~VisMan()
   }
 }

-
 ostream &operator<<(ostream &v, void (*f)(VisMan&))
 {
   VisMan *vp = dynamic_cast<VisMan*>(&v);
@@ -288,6 +288,7 @@ int main(int argc, char *argv[])

   GridFunction u_gf(&fespace);
   GridFunction dudt_gf(&fespace);
+
   // 6. Set the initial conditions for u. All boundaries are considered
   //    natural.
   FunctionCoefficient u_0(InitialSolution);
@@ -13,14 +13,16 @@
 // Description:  This example code solves a simple electromagnetic wave
 //               propagation problem corresponding to the second order
 //               indefinite Maxwell equation
+//
 //                  (1/mu) * curl curl E - \omega^2 * epsilon E = f
+//
 //               with a Perfectly Matched Layer (PML).
 //
 //               The example demonstrates discretization with Nedelec finite
 //               elements in 2D or 3D, as well as the use of complex-valued
 //               bilinear and linear forms. Several test problems are included,
 //               with prob = 0-3 having known exact solutions, see "On perfectly
-//               matched layers for discontinuous Petrov–Galerkin methods" by
+//               matched layers for discontinuous Petrov-Galerkin methods" by
 //               Vaziri Astaneh, Keith, Demkowicz, Comput Mech 63, 2019.
 //
 //               We recommend viewing Example 22 before viewing this example.
@@ -13,14 +13,16 @@
 // Description:  This example code solves a simple electromagnetic wave
 //               propagation problem corresponding to the second order
 //               indefinite Maxwell equation
+//
 //                  (1/mu) * curl curl E - \omega^2 * epsilon E = f
+//
 //               with a Perfectly Matched Layer (PML).
 //
 //               The example demonstrates discretization with Nedelec finite
 //               elements in 2D or 3D, as well as the use of complex-valued
 //               bilinear and linear forms. Several test problems are included,
 //               with prob = 0-3 having known exact solutions, see "On perfectly
-//               matched layers for discontinuous Petrov–Galerkin methods" by
+//               matched layers for discontinuous Petrov-Galerkin methods" by
 //               Vaziri Astaneh, Keith, Demkowicz, Comput Mech 63, 2019.
 //
 //               We recommend viewing Example 22 before viewing this example.
@@ -204,9 +204,9 @@ int main(int argc, char *argv[])
   x = 0.0;

   // 8. Create the multigrid operator using the previously created
-   //    FiniteElementSpaceHierarchy and additional boundary information. This operator
-   //    is then used to create the MultigridSolver as a preconditioner in the
-   //    iterative solver.
+   //    FiniteElementSpaceHierarchy and additional boundary information. This
+   //    operator is then used to create the MultigridSolver as a preconditioner
+   //    in the iterative solver.
   Array<int> ess_bdr(mesh->bdr_attributes.Max());
   ess_bdr = 1;

@@ -224,8 +224,8 @@ int main(int argc, char *argv[])
   // 10. Recover the solution as a finite element grid function.
   M.RecoverFineFEMSolution(X, *b, x);

-   // 11. Save the refined mesh and the solution. This output can be viewed later
-   //     using GLVis: "glvis -m refined.mesh -g sol.gf".
+   // 11. Save the refined mesh and the solution. This output can be viewed
+   //     later using GLVis: "glvis -m refined.mesh -g sol.gf".
   ofstream mesh_ofs("refined.mesh");
   mesh_ofs.precision(8);
   fespaces.GetFinestFESpace().GetMesh()->Print(mesh_ofs);
@@ -245,9 +245,9 @@ int main(int argc, char *argv[])
   x = 0.0;

   // 10. Create the multigrid operator using the previously created parallel
-   //     FiniteElementSpaceHierarchy and additional boundary information. This operator
-   //     is then used to create the MultigridSolver as a preconditioner in the
-   //     iterative solver.
+   //     FiniteElementSpaceHierarchy and additional boundary information. This
+   //     operator is then used to create the MultigridSolver as preconditioner
+   //     in the iterative solver.
   Array<int> ess_bdr(pmesh->bdr_attributes.Max());
   if (pmesh->bdr_attributes.Size())
   {
@@ -274,8 +274,8 @@ int main(int argc, char *argv[])
   //     local finite element solution on each processor.
   M->RecoverFineFEMSolution(X, *b, x);

-   // 13. Save the refined mesh and the solution in parallel. This output can
-   //     be viewed later using GLVis: "glvis -np <np> -m mesh -g sol".
+   // 13. Save the refined mesh and the solution in parallel. This output can be
+   //     viewed later using GLVis: "glvis -np <np> -m mesh -g sol".
   {
      ostringstream mesh_name, sol_name;
      mesh_name << "mesh." << setfill('0') << setw(6) << myid;
@@ -10,50 +10,50 @@
 // Description:  This example code demonstrates the use of MFEM to define a
 //               simple finite element discretization of the Laplace problem
 //               -Delta u = 0 with a variety of boundary conditions.
+//
 //               Specifically, we discretize using a FE space of the specified
-//               order using a continuous or discontinuous space.  We then
-//               apply Dirichlet, Neumann (both homogeneous and inhomogeneous),
-//               Robin, and Periodic boundary conditions on different portions
-//               of a predefined mesh.
+//               order using a continuous or discontinuous space. We then apply
+//               Dirichlet, Neumann (both homogeneous and inhomogeneous), Robin,
+//               and Periodic boundary conditions on different portions of a
+//               predefined mesh.
 //
-//               The predefined mesh consists of a rectangle with two
-//               holes removed (see below).  The narrow ends of the
-//               mesh are connected to form a Periodic boundary
-//               condition.  The lower edge (tagged with attribute 1)
-//               receives an inhomogeneous Neumann boundary condition.
-//               A Robin boundary condition is applied to upper edge
-//               (attribute 2).  The circular hole on the left
-//               (attribute 3) enforces a Dirichlet boundary
-//               condition.  Finally, a natural boundary condition, or
-//               homogeneous Neumann BC, is applied to the circular
-//               hole on the right (attribute 4).
+//               The predefined mesh consists of a rectangle with two holes
+//               removed (see below). The narrow ends of the mesh are connected
+//               to form a Periodic boundary condition. The lower edge (tagged
+//               with attribute 1) receives an inhomogeneous Neumann boundary
+//               condition. A Robin boundary condition is applied to upper edge
+//               (attribute 2). The circular hole on the left (attribute 3)
+//               enforces a Dirichlet boundary condition. Finally, a natural
+//               boundary condition, or homogeneous Neumann BC, is applied to
+//               the circular hole on the right (attribute 4).
 //
-//                  Attribute 3    ^ y  Attribute 2
-//                        \        |      /
-//                     +-----------+-----------+
-//                     |    \_     |     _     |
-//                     |    / \    |    / \    |
-//                  <--+---+---+---+---+---+---+--> x
-//                     |    \_/    |    \_/    |
-//                     |           |      \    |
-//                     +-----------+-----------+       (hole radii are
-//                          /      |        \            adjustable)
-//                  Attribute 1    v    Attribute 4
+//                    Attribute 3    ^ y  Attribute 2
+//                          \        |      /
+//                       +-----------+-----------+
+//                       |    \_     |     _     |
+//                       |    / \    |    / \    |
+//                    <--+---+---+---+---+---+---+--> x
+//                       |    \_/    |    \_/    |
+//                       |           |      \    |
+//                       +-----------+-----------+       (hole radii are
+//                            /      |        \            adjustable)
+//                    Attribute 1    v    Attribute 4
+//
+//               The boundary conditions are defined as (where u is the solution
+//               field):
 //
-//               The boundary conditions are defined as (where u is
-//               the solution field):
 //                  Dirichlet: u = d
 //                  Neumann:   n.Grad(u) = g
 //                  Robin:     n.Grad(u) + a u = b
 //
-//               The user can adjust the values of 'd', 'g', 'a', and
-//               'b' with command line options.
+//               The user can adjust the values of 'd', 'g', 'a', and 'b' with
+//               command line options.
 //
 //               This example highlights the differing implementations of
 //               boundary conditions with continuous and discontinuous Galerkin
 //               formulations of the Laplace problem.
 //
-//               We recommend viewing examples 1 and 14 before viewing this
+//               We recommend viewing Examples 1 and 14 before viewing this
 //               example.

 #include "mfem.hpp"
@@ -71,8 +71,8 @@ void n4Vec(const Vector &x, Vector &n) { n = x; n[0] -= 0.5; n /= -n.Norml2(); }
 Mesh * GenerateSerialMesh(int ref);

 // Compute the average value of alpha*n.Grad(sol) + beta*sol over the boundary
-// attributes marked in bdr_marker.  Also computes the L2 norm of
-// alpha*n.Grad(sol) + beta*sol - gamma  over the same boundary.
+// attributes marked in bdr_marker. Also computes the L2 norm of
+// alpha*n.Grad(sol) + beta*sol - gamma over the same boundary.
 double IntegrateBC(const GridFunction &sol, const Array<int> &bdr_marker,
                   double alpha, double beta, double gamma,
                   double &err);
@@ -152,9 +152,9 @@ int main(int argc, char *argv[])
   Mesh *mesh = GenerateSerialMesh(ser_ref_levels);
   int dim = mesh->Dimension();

-   // 3. Define a finite element space on the serial mesh.  Here we
-   //    use either continuous Lagrange finite elements or discontinuous
-   //    Galerkin finite elements of the specified order.
+   // 3. Define a finite element space on the serial mesh. Here we use either
+   //    continuous Lagrange finite elements or discontinuous Galerkin finite
+   //    elements of the specified order.
   FiniteElementCollection *fec =
      h1 ? (FiniteElementCollection*)new H1_FECollection(order, dim) :
      (FiniteElementCollection*)new DG_FECollection(order, dim);
@@ -162,11 +162,10 @@ int main(int argc, char *argv[])
   int size = fespace.GetTrueVSize();
   mfem::out << "Number of finite element unknowns: " << size << endl;

-   // 4. Create "marker arrays" to define the portions of the boundary
-   //    associated with each type of boundary condition.  These arrays
-   //    have an entry corresponding to each boundary attribute.
-   //    Placing a '1' in entry i marks attribute i+1 as being
-   //    active, '0' is inactive.
+   // 4. Create "marker arrays" to define the portions of boundary associated
+   //    with each type of boundary condition. These arrays have an entry
+   //    corresponding to each boundary attribute.  Placing a '1' in entry i
+   //    marks attribute i+1 as being active, '0' is inactive.
   Array<int> nbc_bdr(mesh->bdr_attributes.Max());
   Array<int> rbc_bdr(mesh->bdr_attributes.Max());
   Array<int> dbc_bdr(mesh->bdr_attributes.Max());
@@ -178,25 +177,25 @@ int main(int argc, char *argv[])
   Array<int> ess_tdof_list(0);
   if (h1 && mesh->bdr_attributes.Size())
   {
-      // For a continuous basis the linear system must be modifed to enforce
-      // an essential (Dirichlet) boundary condition.  In the DG case this is
-      // not necessary as the boundary condition will only be enforced weakly.
+      // For a continuous basis the linear system must be modified to enforce an
+      // essential (Dirichlet) boundary condition. In the DG case this is not
+      // necessary as the boundary condition will only be enforced weakly.
      fespace.GetEssentialTrueDofs(dbc_bdr, ess_tdof_list);
   }

-   // 5. Setup the various coefficients needed for the Laplace operator and
-   //    the various boundary conditions.  In general these coefficients could
-   //    be functions of position but here we use only constants.
+   // 5. Setup the various coefficients needed for the Laplace operator and the
+   //    various boundary conditions. In general these coefficients could be
+   //    functions of position but here we use only constants.
   ConstantCoefficient matCoef(mat_val);
   ConstantCoefficient dbcCoef(dbc_val);
   ConstantCoefficient nbcCoef(nbc_val);
   ConstantCoefficient rbcACoef(rbc_a_val);
   ConstantCoefficient rbcBCoef(rbc_b_val);

-   // Since the n.Grad(u) terms arise by integrating -Div(m Grad(u)) by parts
-   // we must introduce the coefficient 'm' into the boundary conditions.
-   // Therefore, in the case of the Neumann BC, we actually enforce
-   // m n.Grad(u) = m g rather than simply n.Grad(u) = g.
+   // Since the n.Grad(u) terms arise by integrating -Div(m Grad(u)) by parts we
+   // must introduce the coefficient 'm' into the boundary conditions.
+   // Therefore, in the case of the Neumann BC, we actually enforce m n.Grad(u)
+   // = m g rather than simply n.Grad(u) = g.
   ProductCoefficient m_nbcCoef(matCoef, nbcCoef);
   ProductCoefficient m_rbcACoef(matCoef, rbcACoef);
   ProductCoefficient m_rbcBCoef(matCoef, rbcBCoef);
@@ -218,7 +217,7 @@ int main(int argc, char *argv[])
   }
   else
   {
-      // Add the interfacial portion of the Lapalce operator
+      // Add the interfacial portion of the Laplace operator
      a.AddInteriorFaceIntegrator(new DGDiffusionIntegrator(matCoef,
                                                            sigma, kappa));

@@ -237,7 +236,7 @@ int main(int argc, char *argv[])

   if (h1)
   {
-      // Set the Dirchlet values in the solution vector
+      // Set the Dirichlet values in the solution vector
      u.ProjectBdrCoefficient(dbcCoef, dbc_bdr);

      // Add the desired value for n.Grad(u) on the Neumann boundary
@@ -248,7 +247,7 @@ int main(int argc, char *argv[])
   }
   else
   {
-      // Add the desired value for the Dirchlet boundary
+      // Add the desired value for the Dirichlet boundary
      b.AddBdrFaceIntegrator(new DGDirichletLFIntegrator(dbcCoef, matCoef,
                                                         sigma, kappa),
                             dbc_bdr);
@@ -292,12 +291,12 @@ int main(int argc, char *argv[])
   umf_solver.Mult(B, X);
 #endif

-   // 12. Recover the grid function corresponding to U. This is the
-   //     local finite element solution.
+   // 12. Recover the grid function corresponding to U. This is the local finite
+   //     element solution.
   a.RecoverFEMSolution(X, b, u);

-   // 13. Build a mass matrix to help solve for n.Grad(u) where 'n' is
-   //     a surface normal.
+   // 13. Build a mass matrix to help solve for n.Grad(u) where 'n' is a surface
+   //     normal.
   BilinearForm m(&fespace);
   m.AddDomainIntegrator(new MassIntegrator);
   m.Assemble();
@@ -311,8 +310,8 @@ int main(int argc, char *argv[])
             << "Verifying boundary conditions" << endl
             << "=============================" << endl;
   {
-      // Integrate the solution on the Dirichlet boundary and compare
-      // to the expected value.
+      // Integrate the solution on the Dirichlet boundary and compare to the
+      // expected value.
      double err, avg = IntegrateBC(u, dbc_bdr, 0.0, 1.0, dbc_val, err);

      bool hom_dbc = (dbc_val == 0.0);
@@ -323,8 +322,8 @@ int main(int argc, char *argv[])
                << " error " << err << endl;
   }
   {
-      // Integrate n.Grad(u) on the inhomogeneous Neumann boundary and
-      // compare to the expected value.
+      // Integrate n.Grad(u) on the inhomogeneous Neumann boundary and compare
+      // to the expected value.
      double err, avg = IntegrateBC(u, nbc_bdr, 1.0, 0.0, nbc_val, err);

      bool hom_nbc = (nbc_val == 0.0);
@@ -335,8 +334,8 @@ int main(int argc, char *argv[])
                << " error " << err << endl;
   }
   {
-      // Integrate n.Grad(u) on the homogeneous Neumann boundary and compare
-      // to the expected value of zero.
+      // Integrate n.Grad(u) on the homogeneous Neumann boundary and compare to
+      // the expected value of zero.
      Array<int> nbc0_bdr(mesh->bdr_attributes.Max());
      nbc0_bdr = 0;
      nbc0_bdr[3] = 1;
@@ -350,8 +349,8 @@ int main(int argc, char *argv[])
                << " error " << err << endl;
   }
   {
-      // Integrate n.Grad(u) + a * u on the Robin boundary and compare to
-      // the expected value.
+      // Integrate n.Grad(u) + a * u on the Robin boundary and compare to the
+      // expected value.
      double err, avg = IntegrateBC(u, rbc_bdr, 1.0, rbc_a_val, rbc_b_val, err);

      bool hom_rbc = (rbc_b_val == 0.0);
@@ -727,8 +726,8 @@ double IntegrateBC(const GridFunction &x, const Array<int> &bdr,
      avg /= nrm;
   }

-   // Compute l2 norm of the error in the boundary condition
-   // (negative quadrature weights may produce negative 'err')
+   // Compute l2 norm of the error in the boundary condition (negative
+   // quadrature weights may produce negative 'err')
   err = (err >= 0.0) ? sqrt(err) : -sqrt(-err);

   // Return the average value of alpha * n.Grad(x) + beta * x
@@ -10,50 +10,50 @@
 // Description:  This example code demonstrates the use of MFEM to define a
 //               simple finite element discretization of the Laplace problem
 //               -Delta u = 0 with a variety of boundary conditions.
+//
 //               Specifically, we discretize using a FE space of the specified
-//               order using a continuous or discontinuous space.  We then
-//               apply Dirichlet, Neumann (both homogeneous and inhomogeneous),
-//               Robin, and Periodic boundary conditions on different portions
-//               of a predefined mesh.
+//               order using a continuous or discontinuous space. We then apply
+//               Dirichlet, Neumann (both homogeneous and inhomogeneous), Robin,
+//               and Periodic boundary conditions on different portions of a
+//               predefined mesh.
 //
-//               The predefined mesh consists of a rectangle with two
-//               holes removed (see below).  The narrow ends of the
-//               mesh are connected to form a Periodic boundary
-//               condition.  The lower edge (tagged with attribute 1)
-//               receives an inhomogeneous Neumann boundary condition.
-//               A Robin boundary condition is applied to upper edge
-//               (attribute 2).  The circular hole on the left
-//               (attribute 3) enforces a Dirichlet boundary
-//               condition.  Finally, a natural boundary condition, or
-//               homogeneous Neumann BC, is applied to the circular
-//               hole on the right (attribute 4).
+//               The predefined mesh consists of a rectangle with two holes
+//               removed (see below). The narrow ends of the mesh are connected
+//               to form a Periodic boundary condition. The lower edge (tagged
+//               with attribute 1) receives an inhomogeneous Neumann boundary
+//               condition. A Robin boundary condition is applied to upper edge
+//               (attribute 2). The circular hole on the left (attribute 3)
+//               enforces a Dirichlet boundary condition. Finally, a natural
+//               boundary condition, or homogeneous Neumann BC, is applied to
+//               the circular hole on the right (attribute 4).
 //
-//                  Attribute 3    ^ y  Attribute 2
-//                        \        |      /
-//                     +-----------+-----------+
-//                     |    \_     |     _     |
-//                     |    / \    |    / \    |
-//                  <--+---+---+---+---+---+---+--> x
-//                     |    \_/    |    \_/    |
-//                     |           |      \    |
-//                     +-----------+-----------+       (hole radii are
-//                          /      |        \            adjustable)
-//                  Attribute 1    v    Attribute 4
+//                    Attribute 3    ^ y  Attribute 2
+//                          \        |      /
+//                       +-----------+-----------+
+//                       |    \_     |     _     |
+//                       |    / \    |    / \    |
+//                    <--+---+---+---+---+---+---+--> x
+//                       |    \_/    |    \_/    |
+//                       |           |      \    |
+//                       +-----------+-----------+       (hole radii are
+//                            /      |        \            adjustable)
+//                    Attribute 1    v    Attribute 4
+//
+//               The boundary conditions are defined as (where u is the solution
+//               field):
 //
-//               The boundary conditions are defined as (where u is
-//               the solution field):
 //                  Dirichlet: u = d
 //                  Neumann:   n.Grad(u) = g
 //                  Robin:     n.Grad(u) + a u = b
 //
-//               The user can adjust the values of 'd', 'g', 'a', and
-//               'b' with command line options.
+//               The user can adjust the values of 'd', 'g', 'a', and 'b' with
+//               command line options.
 //
 //               This example highlights the differing implementations of
 //               boundary conditions with continuous and discontinuous Galerkin
 //               formulations of the Laplace problem.
 //
-//               We recommend viewing examples 1 and 14 before viewing this
+//               We recommend viewing Examples 1 and 14 before viewing this
 //               example.

 #include "mfem.hpp"
@@ -71,8 +71,8 @@ void n4Vec(const Vector &x, Vector &n) { n = x; n[0] -= 0.5; n /= -n.Norml2(); }
 Mesh * GenerateSerialMesh(int ref);

 // Compute the average value of alpha*n.Grad(sol) + beta*sol over the boundary
-// attributes marked in bdr_marker.  Also computes the L2 norm of
-// alpha*n.Grad(sol) + beta*sol - gamma  over the same boundary.
+// attributes marked in bdr_marker. Also computes the L2 norm of
+// alpha*n.Grad(sol) + beta*sol - gamma over the same boundary.
 double IntegrateBC(const ParGridFunction &sol, const Array<int> &bdr_marker,
                   double alpha, double beta, double gamma,
                   double &err);
@@ -179,11 +179,10 @@ int main(int argc, char *argv[])
   HYPRE_Int size = fespace.GlobalTrueVSize();
   mfem::out << "Number of finite element unknowns: " << size << endl;

-   // 6. Create "marker arrays" to define the portions of the boundary
-   //    associated with each type of boundary condition.  These arrays
-   //    have an entry corresponding to each boundary attribute.
-   //    Placing a '1' in entry i marks attribute i+1 as being
-   //    active, '0' is inactive.
+   // 6. Create "marker arrays" to define the portions of boundary associated
+   //    with each type of boundary condition. These arrays have an entry
+   //    corresponding to each boundary attribute. Placing a '1' in entry i
+   //    marks attribute i+1 as being active, '0' is inactive.
   Array<int> nbc_bdr(pmesh.bdr_attributes.Max());
   Array<int> rbc_bdr(pmesh.bdr_attributes.Max());
   Array<int> dbc_bdr(pmesh.bdr_attributes.Max());
@@ -195,25 +194,25 @@ int main(int argc, char *argv[])
   Array<int> ess_tdof_list(0);
   if (h1 && pmesh.bdr_attributes.Size())
   {
-      // For a continuous basis the linear system must be modifed to enforce
-      // an essential (Dirichlet) boundary condition.  In the DG case this is
-      // not necessary as the boundary condition will only be enforced weakly.
+      // For a continuous basis the linear system must be modifed to enforce an
+      // essential (Dirichlet) boundary condition. In the DG case this is not
+      // necessary as the boundary condition will only be enforced weakly.
      fespace.GetEssentialTrueDofs(dbc_bdr, ess_tdof_list);
   }

-   // 7. Setup the various coefficients needed for the Laplace operator and
-   //    the various boundary conditions.  In general these coefficients could
-   //    be functions of position but here we use only constants.
+   // 7. Setup the various coefficients needed for the Laplace operator and the
+   //    various boundary conditions. In general these coefficients could be
+   //    functions of position but here we use only constants.
   ConstantCoefficient matCoef(mat_val);
   ConstantCoefficient dbcCoef(dbc_val);
   ConstantCoefficient nbcCoef(nbc_val);
   ConstantCoefficient rbcACoef(rbc_a_val);
   ConstantCoefficient rbcBCoef(rbc_b_val);

-   // Since the n.Grad(u) terms arise by integrating -Div(m Grad(u)) by parts
-   // we must introduce the coefficient 'm' into the boundary conditions.
-   // Therefore, in the case of the Neumann BC, we actually enforce
-   // m n.Grad(u) = m g rather than simply n.Grad(u) = g.
+   // Since the n.Grad(u) terms arise by integrating -Div(m Grad(u)) by parts we
+   // must introduce the coefficient 'm' into the boundary conditions.
+   // Therefore, in the case of the Neumann BC, we actually enforce m n.Grad(u)
+   // = m g rather than simply n.Grad(u) = g.
   ProductCoefficient m_nbcCoef(matCoef, nbcCoef);
   ProductCoefficient m_rbcACoef(matCoef, rbcACoef);
   ProductCoefficient m_rbcBCoef(matCoef, rbcBCoef);
@@ -235,7 +234,7 @@ int main(int argc, char *argv[])
   }
   else
   {
-      // Add the interfacial portion of the Lapalce operator
+      // Add the interfacial portion of the Laplace operator
      a.AddInteriorFaceIntegrator(new DGDiffusionIntegrator(matCoef,
                                                            sigma, kappa));

@@ -254,7 +253,7 @@ int main(int argc, char *argv[])

   if (h1)
   {
-      // Set the Dirchlet values in the solution vector
+      // Set the Dirichlet values in the solution vector
      u.ProjectBdrCoefficient(dbcCoef, dbc_bdr);

      // Add the desired value for n.Grad(u) on the Neumann boundary
@@ -265,7 +264,7 @@ int main(int argc, char *argv[])
   }
   else
   {
-      // Add the desired value for the Dirchlet boundary
+      // Add the desired value for the Dirichlet boundary
      b.AddBdrFaceIntegrator(new DGDirichletLFIntegrator(dbcCoef, matCoef,
                                                         sigma, kappa),
                             dbc_bdr);
@@ -315,8 +314,8 @@ int main(int argc, char *argv[])
   //     local finite element solution on each processor.
   a.RecoverFEMSolution(X, b, u);

-   // 14. Build a mass matrix to help solve for n.Grad(u) where 'n' is
-   //     a surface normal.
+   // 14. Build a mass matrix to help solve for n.Grad(u) where 'n' is a surface
+   //     normal.
   ParBilinearForm m(&fespace);
   m.AddDomainIntegrator(new MassIntegrator);
   m.Assemble();
@@ -330,8 +329,8 @@ int main(int argc, char *argv[])
             << "Verifying boundary conditions" << endl
             << "=============================" << endl;
   {
-      // Integrate the solution on the Dirichlet boundary and compare
-      // to the expected value.
+      // Integrate the solution on the Dirichlet boundary and compare to the
+      // expected value.
      double err, avg = IntegrateBC(u, dbc_bdr, 0.0, 1.0, dbc_val, err);

      bool hom_dbc = (dbc_val == 0.0);
@@ -342,8 +341,8 @@ int main(int argc, char *argv[])
                << " error " << err << endl;
   }
   {
-      // Integrate n.Grad(u) on the inhomogeneous Neumann boundary and
-      // compare to the expected value.
+      // Integrate n.Grad(u) on the inhomogeneous Neumann boundary and compare
+      // to the expected value.
      double err, avg = IntegrateBC(u, nbc_bdr, 1.0, 0.0, nbc_val, err);

      bool hom_nbc = (nbc_val == 0.0);
@@ -354,8 +353,8 @@ int main(int argc, char *argv[])
                << " error " << err << endl;
   }
   {
-      // Integrate n.Grad(u) on the homogeneous Neumann boundary and compare
-      // to the expected value of zero.
+      // Integrate n.Grad(u) on the homogeneous Neumann boundary and compare to
+      // the expected value of zero.
      Array<int> nbc0_bdr(pmesh.bdr_attributes.Max());
      nbc0_bdr = 0;
      nbc0_bdr[3] = 1;
@@ -369,8 +368,8 @@ int main(int argc, char *argv[])
                << " error " << err << endl;
   }
   {
-      // Integrate n.Grad(u) + a * u on the Robin boundary and compare to
-      // the expected value.
+      // Integrate n.Grad(u) + a * u on the Robin boundary and compare to the
+      // expected value.
      double err, avg = IntegrateBC(u, rbc_bdr, 1.0, rbc_a_val, rbc_b_val, err);

      bool hom_rbc = (rbc_b_val == 0.0);
@@ -381,8 +380,8 @@ int main(int argc, char *argv[])
                << " error " << err << endl;
   }

-   // 16. Save the refined mesh and the solution in parallel. This output can
-   //     be viewed later using GLVis: "glvis -np <np> -m mesh -g sol".
+   // 16. Save the refined mesh and the solution in parallel. This output can be
+   //     viewed later using GLVis: "glvis -np <np> -m mesh -g sol".
   {
      ostringstream mesh_name, sol_name;
      mesh_name << "mesh." << setfill('0') << setw(6) << mpi.WorldRank();
@@ -764,8 +763,8 @@ double IntegrateBC(const ParGridFunction &x, const Array<int> &bdr,
      glb_avg /= glb_nrm;
   }

-   // Compute l2 norm of the error in the boundary condition
-   // (negative quadrature weights may produce negative 'err')
+   // Compute l2 norm of the error in the boundary condition (negative
+   // quadrature weights may produce negative 'err')
   glb_err = (glb_err >= 0.0) ? sqrt(glb_err) : -sqrt(-glb_err);

   // Return the average value of alpha * n.Grad(x) + beta * x
@@ -19,8 +19,10 @@
 //
 // Description:  This example code solves a simple 2D/3D mixed Darcy problem
 //               corresponding to the saddle point system
+//
 //                                 k*u + grad p = f
 //                                 - div u      = g
+//
 //               with natural boundary condition -p = <given pressure>.
 //               Here, we use a given exact solution (u,p) and compute the
 //               corresponding r.h.s. (f,g).  We discretize with Raviart-Thomas
@@ -19,8 +19,10 @@
 //
 // Description:  This example code solves a simple 2D/3D mixed Darcy problem
 //               corresponding to the saddle point system
+//
 //                                 k*u + grad p = f
 //                                 - div u      = g
+//
 //               with natural boundary condition -p = <given pressure>.
 //               Here, we use a given exact solution (u,p) and compute the
 //               corresponding r.h.s. (f,g).  We discretize with Raviart-Thomas
@@ -12,7 +12,7 @@ use of MFEM for solving nonlinear constrained optimization problems, including
 features based on the HiOp, a lightweight HPC solver for nonlinear optimization
 problems.

-To use the HiOp featuress, make sure that MFEM is configured with the option
+To use the HiOp features, make sure that MFEM is configured with the option
 "MFEM_USE_HIOP = YES", see the top-level INSTALL file for details.

 We recommend comparing the original example codes with the corresponding files
@@ -52,6 +52,10 @@
 #include <fstream>
 #include <iostream>

+#ifndef MFEM_USE_HIOP
+#error This example requires that MFEM is built with MFEM_USE_HIOP=YES
+#endif
+
 using namespace std;
 using namespace mfem;

@@ -100,7 +104,7 @@ public:
   }
 };

-/// Nonlinear monotone bounded operator to test nonlinear ineq constraints.
+/// Nonlinear monotone bounded operator to test nonlinear inequality constraints
 /// Computes D(x) = tanh(sum(x_i)).
 class TanhSumOperator : public Operator
 {
@@ -124,7 +128,7 @@ public:
   }
 };

-/** Monotone and conservative a-posteriori correction for transport solutions:
+/** Monotone and conservative a posteriori correction for transport solutions:
 *  Find x that minimizes 0.5 || x - x_HO ||^2, subject to
 *  sum w_i x_i = mass,
 *  tanh(sum(x_i_min)) <= tanh(sum(x_i)) <= tanh(sum(x_i_max)),
@@ -52,6 +52,10 @@
 #include <fstream>
 #include <iostream>

+#ifndef MFEM_USE_HIOP
+#error This example requires that MFEM is built with MFEM_USE_HIOP=YES
+#endif
+
 using namespace std;
 using namespace mfem;

@@ -109,7 +113,7 @@ public:
   }
 };

-/// Nonlinear monotone bounded operator to test nonlinear ineq constraints.
+/// Nonlinear monotone bounded operator to test nonlinear inequality constraints
 /// Computes D(x) = tanh(sum(x_i)).
 class TanhSumOperator : public Operator
 {
@@ -141,7 +145,7 @@ public:
 };


-/** Monotone and conservative a-posteriori correction for transport solutions:
+/** Monotone and conservative a posteriori correction for transport solutions:
 *  Find x that minimizes 0.5 || x - x_HO ||^2, subject to
 *  sum w_i x_i = mass,
 *  tanh(sum(x_i_min)) <= tanh(sum(x_i)) <= tanh(sum(x_i_max)),
@@ -33,8 +33,14 @@ else
   EXAMPLES = $(PAR_EXAMPLES) $(SEQ_EXAMPLES)
 endif
 SUBDIRS =
-ifeq ($(MFEM_USE_SUNDIALS),YES)
-   SUBDIRS += sundials
+ifeq ($(MFEM_USE_AMGX),YES)
+   SUBDIRS += amgx
+endif
+ifeq ($(MFEM_USE_GINKGO),YES)
+   SUBDIRS += ginkgo
+endif
+ifeq ($(MFEM_USE_HIOP),YES)
+   SUBDIRS += hiop
 endif
 ifeq ($(MFEM_USE_PETSC),YES)
   SUBDIRS += petsc
@@ -42,14 +48,8 @@ endif
 ifeq ($(MFEM_USE_PUMI),YES)
   SUBDIRS += pumi
 endif
-ifeq ($(MFEM_USE_HIOP),YES)
-   SUBDIRS += hiop
-endif
-ifeq ($(MFEM_USE_GINKGO),YES)
-   SUBDIRS += ginkgo
-endif
-ifeq ($(MFEM_USE_AMGX),YES)
-   SUBDIRS += amgx
+ifeq ($(MFEM_USE_SUNDIALS),YES)
+   SUBDIRS += sundials
 endif
 ifeq ($(MFEM_USE_SUPERLU),YES)
   SUBDIRS += superlu
@@ -14,5 +14,9 @@ To build these examples, make sure that MFEM is configured with the option
 "MFEM_USE_PETSC = YES", see the top-level INSTALL file for details (version
 3.8 or higher of the PETSc dev branch is required).

+The version of Example 11 in this directory demonstrates also the use of the
+PETSc-bases SLEPc eigensolver. That example additionally requires that MFEM is
+configured with the option "MFEM_USE_SLEPC = YES".
+
 We recommend comparing the original example codes with the corresponding files
 in the current directory.
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Stowell, Mark L	6c62756b28	Fixing copyright year	2021-04-09 15:49:41 -07:00
Stowell, Mark L	96e9dc09a6	Removing partial assembly code from example	2021-04-09 15:49:26 -07:00
Stowell, Mark L	52ccc6d980	Updating other DiffusionIntegrator methods	2021-04-09 15:48:59 -07:00
Syun'ichi Shiraiwa	eef5786aa8	Merge branch 'fix_embeded_diffusion_matrix_coeff' of github.com:mfem/mfem into fix_embeded_diffusion_matrix_coeff	2021-03-28 12:32:35 -04:00
Syun'ichi Shiraiwa	017460d517	added two assertions in routines which Veselin pointed out	2021-03-28 12:31:36 -04:00
Stowell, Mark L	62668b0e81	Adding unit test	2021-03-26 17:03:46 -07:00
Syun'ichi Shiraiwa	134c3b6c7f	added a DenseMatrix with proper size	2021-03-23 18:01:15 -04:00
Tzanio Kolev	ed5604e0d3	Merge pull request #1827 from mfem/mfem-4.2-dev Final changes for the mfem-4.2 release	2020-10-30 15:30:58 -07:00
Veselin Dobrev	4bfb6b5a93	In miniapps/adjoint/cvsRoberts_ASAi_dns.cpp, call the method SetMaxNStepsB() after InitB().	2020-10-30 14:59:56 -07:00
Veselin Dobrev	ca7cf33e51	In class CVODESSolver, add a wrapper for CVodeSetMaxNumStepsB and use it in miniapps/adjoint/cvsRoberts_ASAi_dns.cpp to increase the default value.	2020-10-30 14:48:21 -07:00
Tzanio	3ff745bbd6	The mfem-4.2 release date is 10/30/20	2020-10-30 14:23:14 -07:00
Veselin Dobrev	51067ca0ae	To address an issue with SUNDIALS v5.4.0, explicitly set the maximum number of steps in miniapps/adjoint/adjoint_advection_diffusion.cpp.	2020-10-30 13:23:05 -07:00
Veselin Dobrev	eb5a60e24e	In miniapps/adjoint, remove the unnecessary dependency on 'mfem-common' from the CMake file as well.	2020-10-30 10:00:27 -07:00
Veselin Dobrev	780cf34b0e	Remove the 'mfem-common' dependency and linking from the miniapps/adjoint directory. In miniapps/adjoint/adjoint_advection_diffusion.cpp, print the usage/options messages only from rank 0. Update two package version requirements in INSTALL.	2020-10-30 08:57:14 -07:00
Veselin Dobrev	97ad695664	In HIP kernels, use MFEM_HIP_BLOCKS instead of MFEM_CUDA_BLOCKS. There is no change in the compiled code since both macros are defined to be 256.	2020-10-29 06:18:56 -07:00
Tzanio Kolev	e6cee4c66a	Merge pull request #1847 from mfem/parse-subnormals Fix GLVis issue with subnormals Apple Clang	2020-10-28 16:58:19 -07:00
Will Pazner	7fc5ce17ba	Zero subnormals in ParaView output	2020-10-28 11:48:46 -07:00
Will Pazner	62f4c49ac6	Remove conditional compilation of resetting failbit for subnormals	2020-10-28 11:42:54 -07:00
Veselin Dobrev	732c7c755d	Fix recursive header inclusion and header include warnings. Remove some copy-paste comments inside doxygen comments.	2020-10-28 07:28:41 -07:00
Will Pazner	d15a402464	Fix GLVis issue with subnormals Apple Clang Zero out subnormal numbers when writing a Vector to an output stream. When reading a Vector from an input stream with Apple Clang, ignore "out of range" errors, which can be caused by (correctly) parsing subnormal numbers.	2020-10-27 15:05:42 -07:00
Veselin Dobrev	93d6ae57a0	Fix the logic in ForallWrap to reflect exactly the allowed/enabled backends. When using 'occa-omp' enable the 'omp' backend (if MFEM was built with MFEM_USE_OPENMP=YES) unless 'raja-omp' is already enabled. If using 'ceed-cuda' enable 'cuda' only if 'raja-cuda' is not allowed. Update the documentation of Device::Configure. Ensure that at most one 'ceed-*' backend is allowed.	2020-10-26 23:33:14 -07:00
Tzanio	96b4b127ea	minor	2020-10-26 14:48:16 -07:00
Veselin Dobrev	46b8f916ba	Small edits in CHANGELOG and INSTALL.	2020-10-26 12:37:14 -07:00
Tzanio KolevandAndrew T. Barker	2f64f73cbd	Apply suggestions from code review Thanks Andrew! Co-authored-by: Andrew T. Barker <barker29@llnl.gov>	2020-10-26 08:26:09 -07:00
Tzanio	9bfe9cac44	Removed some non-ASCII characters	2020-10-26 07:19:28 -07:00
Veselin Dobrev	fa340fd6b1	In CGSolver::Mult check for convergence with <= instead of <. Adjust the "CGSolver" unit test.	2020-10-26 04:26:00 -07:00
Veselin Dobrev	acad1864a2	Remove byte order mark (BOM) from files. Add newline at the end of a file.	2020-10-26 03:15:50 -07:00
Tzanio	8fe29c6d63	Updated CHANGELOG before the mfem-4.2 release	2020-10-25 22:31:07 -07:00
Tzanio	a53d9c1299	Changes in the examples/ directory before the mfem-4.2 release	2020-10-25 17:07:24 -07:00
Tzanio	6a49025c07	Changes in the miniapps/ directory before the mfem-4.2 release	2020-10-25 15:13:00 -07:00
Veselin Dobrev	129e191776	Use 1 as the default attribute in class Element. Tweak the "Element-wise construction" mesh unit test to set the vertices in counterclockwise order and check that the default attribute is 1. Before the above changes, there was a warning from the unit test about non-positive element attributes.	2020-10-25 11:54:41 -07:00
Veselin Dobrev	65d7576530	Fix typos in the CMake file in miniapps/nurbs.	2020-10-25 00:33:29 -07:00
Tzanio	a7d05f30b5	Merge branch 'mfem-4.2-dev' of github.com:mfem/mfem into mfem-4.2-dev	2020-10-24 20:07:15 -07:00
Tzanio	a344b2051f	Changes in the fem/ directory before the mfem-4.2 release	2020-10-24 20:07:10 -07:00
Veselin Dobrev	c540ca8dec	Fix source file names in config/sample-runs.sh	2020-10-24 19:39:08 -07:00
Tzanio	410c2c8d0f	Changes in the mesh/ directory before the mfem-4.2 release	2020-10-24 19:26:22 -07:00
Tzanio	bd7fdaff4e	Merge branch 'master' into mfem-4.2-dev	2020-10-24 19:07:49 -07:00
Tzanio	d472fe16d9	Changes in the linalg/ directory before the mfem-4.2 release	2020-10-24 19:06:53 -07:00
Tzanio	89fad12237	Changes in the general/ directory before the mfem-4.2 release	2020-10-24 18:27:14 -07:00
Veselin Dobrev	54fa6fe4d6	Add support for MPI+HIP builds in makefiles. Fix a few issues with the MPI+HIP build.	2020-10-23 20:48:42 -07:00
Yohann Dudouit	651809f473	Add support for Raja > 0.12	2020-10-23 11:46:59 -07:00
Tzanio	4fb2bade1a	Merge branch 'master' into mfem-4.2-dev	2020-10-23 07:20:03 -07:00
Tzanio	748533fc93	Changes in the doc/ directory before the mfem-4.2 release	2020-10-22 21:56:22 -07:00
Tzanio	de99f19692	Changes in the config/ directory before the mfem-4.2 release	2020-10-22 19:14:38 -07:00
Tzanio	6e5e4246c1	Changes in the tests/ directory before the mfem-4.2 release	2020-10-22 19:04:41 -07:00
Tzanio	6538d4ccf5	Small changes before the mfem-4.2 release	2020-10-22 18:50:33 -07:00
Tzanio	20424d5c88	Merge branch 'paraview-bdr' into mfem-4.2-dev	2020-10-22 17:52:12 -07:00
Dylan Copeland	75c79303a2	Set operator size in AmgXSolver.	2020-10-22 16:34:01 -07:00
Tzanio	2cb0a7b8b6	Merge branch 'master' into mfem-4.2-dev	2020-10-22 12:29:48 -07:00
Tzanio	1833aa7248	Merge branch 'master' into mfem-4.2-dev	2020-10-21 18:44:37 -07:00
Tzanio	cb85fe98e7	Merge branch 'master' into mfem-4.2-dev	2020-10-21 15:36:02 -07:00
Tzanio	4832f32078	Small rewording	2020-10-21 15:35:17 -07:00
Adrien M. Bernede	d1bd2290a4	Remove .html mentions in CONTRIBUTING	2020-10-21 15:18:23 -07:00
Adrien M. Bernede	006a30c031	Fix: restoring logo	2020-10-21 15:13:14 -07:00
Adrien M. Bernede	8149aa2407	Remove mentions of .html and web	2020-10-21 15:11:30 -07:00
Adrien M. Bernede	6aa01187d7	Remove doc/web directory, now only using mfem/web repo	2020-10-21 14:36:56 -07:00
Adrien M. Bernede	89de74c545	Remove remaining html files	2020-10-21 14:10:41 -07:00
Tzanio	41b0aa5d45	Updates in INSTALL for mfem-4.2 release: - Use hypre 2.20.0 instead of 2.16.0 in the example - Mesquite is deprecated and will be removed - Versions for ADIOS and HIOP	2020-10-20 14:08:31 -07:00
Adrien M. Bernede	a209a82e08	Update MFEM version: CHANGELOG, makefile, CMakeLists.txt, doc/CodeDocumentation.conf.in	2020-10-20 13:30:38 -07:00
Tzanio	1527b58ffc	Merge branch 'tbnlf' into mfem-4.2-dev	2020-10-20 12:50:43 -07:00
Tzanio	1197dd6a31	Merge branch 'examples/artv3/amgx-ex1p' into mfem-4.2-dev	2020-10-20 12:50:33 -07:00
Tzanio	89c1638ff9	Merge branch 'yohann/test-ceed' into mfem-4.2-dev	2020-10-20 12:50:20 -07:00
Tzanio	9497f837ca	Merge branch 'nc-point-mat-index-dev' into mfem-4.2-dev	2020-10-20 12:50:08 -07:00