fixed euclid issue

Bugfixes for PetscSolvers

CHANGELOG

@@ -11,12 +11,50 @@
 Version 4.0.1 (development)
 ===========================
 
-GPU support
------------
+Improved GPU support
+--------------------
 - Added initial support for AMD GPUs based on HIP: a C++ runtime API and kernel
-  language that can run on both AMD and NVIDIA hardware. The list of current
-  backends is: "occa-cuda", "raja-cuda", "cuda", "hip", "occa-omp", "raja-omp",
-  "omp", "occa-cpu", "raja-cpu", and "cpu".
+  language that can run on both AMD and NVIDIA hardware. With this change, the
+  list of backends is: "occa-cuda", "raja-cuda", "cuda", "hip", "occa-omp",
+  "raja-omp", "omp", "occa-cpu", "raja-cpu", and "cpu".
+
+- Improved RAJA backend and multi-GPU MPI communications.
+
+Discretization improvements
+---------------------------
+- Added support for non-conforming prism AMR, including coarsening and parallel
+  load balancing. Anisotropic prism refinement is only available in the serial
+  version at the moment.
+
+Meshing improvements
+--------------------
+- The TMOP mesh optimization algorithms were extended to support r-adaptivity.
+  Target matrices can now be constructed either via a given analytical function
+  (e.g. spatial dependence of size, aspect ratio, etc., for each element) or via
+  a (Par)GridFunction specified on the original mesh.
+
+- The TMOP mesh optimization algorithms have been improved to support AMR meshes.
+
+- Added support for creating refined versions of periodic meshes, making use of
+  the new L2ElementRestriction class. This class also allows for computing
+  geometric factors on periodic meshes using partial assembly.
+
+- Improved element numbering after uniform mesh refinement.
+
+New and updated examples and miniapps
+-------------------------------------
+- The mesh-optimizer and pmesh-optimizer miniapps have been updated to
+  demonstrate the new r-adaptivity capabilities of TMOP.
+
+- The (p)mesh-optimizer miniapp has been updated to demonstrate mesh
+  optimization for an AMR mesh.
+
+Miscellaneous
+-------------
+- Upgraded the SUNDIALS interface to utilize SUNDIALS version 5.0.  This
+  necessitated a complete rework of the interface and requires changes at
+  the application level.  Example usage of this new interface can be found
+  in the examples/sundials directory.
 
 
 Version 4.0, released on May 24, 2019
@@ -60,7 +98,6 @@ GPU support
 
 Discretization improvements
 ---------------------------
-
 - Partial assembled finite element operators are now available in the core
   library, based on the new classes PABilinearFormExtension, ElementRestriction,
   DofToQuad and GeometricFactors (associated with the classes BilinearForm,

CMakeLists.txt

@@ -254,7 +254,7 @@ endif()
 
 # Axom/Sidre
 if (MFEM_USE_SIDRE)
-  find_package(Axom REQUIRED Sidre SLIC axom_utils)
+  find_package(Axom REQUIRED Axom)
 endif()
 
 # PUMI

INSTALL

@@ -383,11 +383,11 @@ MFEM_USE_MPFR = YES/NO
    see below.
 
 MFEM_USE_SIDRE = YES/NO
-   Sidre is a component of LLNL's axom project, http://goo.gl/cZyJdn, that
-   provides an HDF5-based file format for visualization or restart capability
-   following the Conduit (https://github.com/LLNL/conduit) mesh blueprint
-   specification. When enabled, this option requires installation of HDF5 (see
-   also MFEM_USE_NETCDF), Conduit and LLNL's axom project.
+   Sidre is a component of LLNL's axom project, https://github.com/LLNL/axom,
+   that provides an HDF5-based file format for visualization or restart
+   capability following the Conduit (https://github.com/LLNL/conduit) mesh
+   blueprint specification. When enabled, this option requires installation of
+   HDF5 (see also MFEM_USE_NETCDF), Conduit and LLNL's axom project.
 
 MFEM_USE_CONDUIT = YES/NO
    Enables support for converting MFEM Mesh and Grid Function objects to and
@@ -488,6 +488,7 @@ The specific libraries and their options are:
 - SUNDIALS (optional), used when MFEM_USE_SUNDIALS = YES.
   Beginning with MFEM v3.3, SUNDIALS v2.7.0 is supported.
   Beginning with MFEM v3.3.2, SUNDIALS v3.0.0 is also supported.
+  Beginning with MFEM v4.1, only SUNDIALS v5.0.0+ is supported.
   If MFEM_USE_MPI is enabled, we expect that SUNDIALS is built with support for
   both MPI and hypre.
   URL: http://computation.llnl.gov/projects/sundials/sundials-software
@@ -542,7 +543,8 @@ The specific libraries and their options are:
   Options: PETSC_OPT, PETSC_LIB.
 
 - Sidre (optional), part of LLNL's axom project, used when MFEM_USE_SIDRE = YES.
-  URL: http://goo.gl/cZyJdn (axom, to be released)
+  Starting with MFEM v4.1, Axom version 0.3.1 or later is required.
+  URL: https://github.com/LLNL/axom
        https://github.com/LLNL/conduit (Conduit)
        https://support.hdfgroup.org/HDF5 (HDF5)
   Options: SIDRE_OPT, SIDRE_LIB.
@@ -570,6 +572,7 @@ The specific libraries and their options are:
   Options: OCCA_DIR, OCCA_OPT, OCCA_LIB.
 
 - RAJA, used when MFEM_USE_RAJA = YES.
+  Beginning with MFEM v4.1, only RAJA v0.10.0+ is supported.
   URL: https://github.com/LLNL/RAJA
   Options: RAJA_DIR, RAJA_OPT, RAJA_LIB.
 
@@ -712,6 +715,7 @@ MFEM_USE_PUMI
 MFEM_USE_CUDA
 MFEM_USE_OCCA
 MFEM_USE_RAJA
+MFEM_USE_SIDRE
 
 The following options are CMake specific:
 
@@ -760,6 +764,7 @@ The CMake build system adds auto-detection for the following packages/libraries:
  - PUMI
  - OCCA
  - RAJA
+ - AXOM - Used when MFEM_USE_SIDRE is enabled
 
 The following built-in CMake packages are also used:
 

config/cmake/modules/FindAxom.cmake

@@ -18,6 +18,4 @@ include(MfemCmakeUtilities)
 # Note: components are enabled based on the find_package() parameters.
 mfem_find_package(Axom AXOM AXOM_DIR "include" "" "lib" ""
   "Paths to headers required by Axom." "Libraries required by Axom."
-  ADD_COMPONENT Sidre "include" sidre/sidre.hpp "lib" sidre
-  ADD_COMPONENT SLIC "include" slic/slic.hpp "lib" slic
-  ADD_COMPONENT axom_utils "include" axom_utils/Utilities.hpp "lib" axom_utils)
+  ADD_COMPONENT Axom "include" axom/config.hpp "lib" axom)

config/defaults.cmake

@@ -81,7 +81,7 @@ set(METIS_DIR "${MFEM_DIR}/../metis-4.0" CACHE PATH "Path to the METIS library."
 
 set(LIBUNWIND_DIR "" CACHE PATH "Path to Libunwind.")
 
-set(SUNDIALS_DIR "${MFEM_DIR}/../sundials-3.0.0" CACHE PATH
+set(SUNDIALS_DIR "${MFEM_DIR}/../sundials-5.0.0/instdir" CACHE PATH
     "Path to the SUNDIALS library.")
 # The following may be necessary, if SUNDIALS was built with KLU:
 # set(SUNDIALS_REQUIRED_PACKAGES "SuiteSparse/KLU/AMD/BTF/COLAMD/config"
@@ -154,7 +154,7 @@ set(CONDUIT_DIR "${MFEM_DIR}/../conduit" CACHE PATH
 
 set(AXOM_DIR "${MFEM_DIR}/../axom" CACHE PATH "Path to the Axom library.")
 # May need to add "Boost" as requirement.
-set(Axom_REQUIRED_PACKAGES "Conduit/relay" CACHE STRING
+set(Axom_REQUIRED_PACKAGES "Conduit/relay/blueprint" CACHE STRING
     "Additional packages required by Axom.")
 
 set(PUMI_DIR "${MFEM_DIR}/../pumi-2.1.0" CACHE STRING

config/defaults.mk

@@ -182,9 +182,9 @@ OPENMP_LIB =
 POSIX_CLOCKS_LIB = -lrt
 
 # SUNDIALS library configuration
-SUNDIALS_DIR = @MFEM_DIR@/../sundials-3.0.0
+SUNDIALS_DIR = @MFEM_DIR@/../sundials-5.0.0/instdir
 SUNDIALS_OPT = -I$(SUNDIALS_DIR)/include
-SUNDIALS_LIB = -Wl,-rpath,$(SUNDIALS_DIR)/lib -L$(SUNDIALS_DIR)/lib\
+SUNDIALS_LIB = -Wl,-rpath,$(SUNDIALS_DIR)/lib64 -L$(SUNDIALS_DIR)/lib64\
  -lsundials_arkode -lsundials_cvode -lsundials_nvecserial -lsundials_kinsol
 
 ifeq ($(MFEM_USE_MPI),YES)
@@ -209,7 +209,7 @@ SUITESPARSE_LIB = -Wl,-rpath,$(SUITESPARSE_DIR)/lib -L$(SUITESPARSE_DIR)/lib\
 # SuperLU library configuration
 SUPERLU_DIR = @MFEM_DIR@/../SuperLU_DIST_5.1.0
 SUPERLU_OPT = -I$(SUPERLU_DIR)/SRC
-SUPERLU_LIB = -Wl,-rpath,$(SUPERLU_DIR)/SRC -L$(SUPERLU_DIR)/SRC -lsuperlu_dist
+SUPERLU_LIB = -Wl,-rpath,$(SUPERLU_DIR)/lib -L$(SUPERLU_DIR)/lib -lsuperlu_dist_5.1.0
 
 # SCOTCH library configuration (required by STRUMPACK <= v2.1.0, optional in
 # STRUMPACK >= v2.2.0)
@@ -299,7 +299,7 @@ SIDRE_LIB = \
    -Wl,-rpath,$(SIDRE_DIR)/lib -L$(SIDRE_DIR)/lib \
    -Wl,-rpath,$(CONDUIT_DIR)/lib -L$(CONDUIT_DIR)/lib \
    -Wl,-rpath,$(HDF5_DIR)/lib -L$(HDF5_DIR)/lib \
-   -lsidre -lslic -laxom_utils -lconduit -lconduit_relay -lhdf5 $(ZLIB_LIB) -ldl
+   -laxom -lconduit -lconduit_relay -lconduit_blueprint -lhdf5 $(ZLIB_LIB) -ldl
 
 # PUMI
 # Note that PUMI_DIR is needed -- it is used to check for gmi_sim.h

examples/ex12p.cpp

@@ -5,11 +5,11 @@
 // Sample runs:
 //    mpirun -np 4 ex12p -m ../data/beam-tri.mesh
 //    mpirun -np 4 ex12p -m ../data/beam-quad.mesh
-//    mpirun -np 4 ex12p -m ../data/beam-tet.mesh -s 79 -n 10 -o 2 -elast
-//    mpirun -np 4 ex12p -m ../data/beam-hex.mesh -s 3876
-//    mpirun -np 4 ex12p -m ../data/beam-wedge.mesh -s 79
+//    mpirun -np 4 ex12p -m ../data/beam-tet.mesh -s 462 -n 10 -o 2 -elast
+//    mpirun -np 4 ex12p -m ../data/beam-hex.mesh -s 3878
+//    mpirun -np 4 ex12p -m ../data/beam-wedge.mesh -s 81
 //    mpirun -np 4 ex12p -m ../data/beam-tri.mesh -s 3876 -o 2 -sys
-//    mpirun -np 4 ex12p -m ../data/beam-quad.mesh -s 4526 -n 6 -o 3 -elast
+//    mpirun -np 4 ex12p -m ../data/beam-quad.mesh -s 4544 -n 6 -o 3 -elast
 //    mpirun -np 4 ex12p -m ../data/beam-quad-nurbs.mesh
 //    mpirun -np 4 ex12p -m ../data/beam-hex-nurbs.mesh
 //

examples/ex17.cpp

@@ -252,7 +252,7 @@ int main(int argc, char *argv[])
    }
    else
    {
-      GMRES(A, M, B, X, 3, 5000, 50, rtol*rtol, 0.0);
+      GMRES(A, M, B, X, 3, 5000, 100, rtol*rtol, 0.0);
    }
 #else
    // 11. If MFEM was compiled with SuiteSparse, use UMFPACK to solve the system.

examples/ex3p.cpp

@@ -102,8 +102,7 @@ int main(int argc, char *argv[])
    //    'ref_levels' to be the largest number that gives a final mesh with no
    //    more than 1,000 elements.
    {
-      int ref_levels =
-         (int)floor(log(1000./mesh->GetNE())/log(2.)/dim);
+      int ref_levels = (int)floor(log(1000./mesh->GetNE())/log(2.)/dim);
       for (int l = 0; l < ref_levels; l++)
       {
          mesh->UniformRefinement();

examples/sundials/CMakeLists.txt

@@ -42,12 +42,12 @@ add_mfem_examples(SUNDIALS_EXAMPLES_SRCS ${PFX} "" test_sundials)
 #   ctest -R sundials
 
 # Command line options for the tests.
-# Example 9: test explicit CVODE time stepping
-set(EX9_COMMON_OPTS -m ../../data/periodic-hexagon.mesh -p 0 -s 11)
+# Example 9: test CVODE with CV_ADAMS (non-stiff implicit) time stepping
+set(EX9_COMMON_OPTS -m ../../data/periodic-hexagon.mesh -p 0 -s 7)
 set(EX9_TEST_OPTS ${EX9_COMMON_OPTS}  -r  2 -dt 0.0018 -vs 25)
 set(EX9P_TEST_OPTS ${EX9_COMMON_OPTS} -rp 1 -dt 0.0009 -vs 50)
-# Example 10: test implicit CVODE time stepping
-set(EX10_COMMON_OPTS -m ../../data/beam-quad.mesh -o 2 -s 5 -dt 0.15 -vs 10)
+# Example 10: test CVODE with CV_BDF (stiff implicit) time stepping
+set(EX10_COMMON_OPTS -m ../../data/beam-quad.mesh -o 2 -s 5 -dt 0.15 -tf 6 -vs 10)
 set(EX10_TEST_OPTS ${EX10_COMMON_OPTS}  -r  2)
 set(EX10P_TEST_OPTS ${EX10_COMMON_OPTS} -rp 1)
 # Example 16: use the default options

examples/sundials/ex10.cpp

@@ -4,16 +4,16 @@
 // Compile with: make ex10
 //
 // Sample runs:
-//    ex10 -m ../../data/beam-quad.mesh -r 2 -o 2 -s  5 -dt 0.15 -vs 10
-//    ex10 -m ../../data/beam-tri.mesh  -r 2 -o 2 -s  7 -dt 0.3  -vs 5
-//    ex10 -m ../../data/beam-hex.mesh  -r 1 -o 2 -s  5 -dt 0.2  -vs 5
+//    ex10 -m ../../data/beam-quad.mesh -r 2 -o 2 -s 12 -dt 0.15 -vs 10
+//    ex10 -m ../../data/beam-tri.mesh  -r 2 -o 2 -s 16 -dt 0.3  -vs 5
+//    ex10 -m ../../data/beam-hex.mesh  -r 1 -o 2 -s 12 -dt 0.2  -vs 5
 //    ex10 -m ../../data/beam-tri.mesh  -r 2 -o 2 -s  2 -dt 3 -nls kinsol
 //    ex10 -m ../../data/beam-quad.mesh -r 2 -o 2 -s  2 -dt 3 -nls kinsol
 //    ex10 -m ../../data/beam-hex.mesh  -r 1 -o 2 -s  2 -dt 3 -nls kinsol
-//    ex10 -m ../../data/beam-quad.mesh -r 2 -o 2 -s 15 -dt 5e-3 -vs 60
-//    ex10 -m ../../data/beam-tri.mesh  -r 2 -o 2 -s 16 -dt 0.01 -vs 30
-//    ex10 -m ../../data/beam-hex.mesh  -r 1 -o 2 -s 15 -dt 0.01 -vs 30
-//    ex10 -m ../../data/beam-quad-amr.mesh -r 2 -o 2 -s 5 -dt 0.15 -vs 10
+//    ex10 -m ../../data/beam-quad.mesh -r 2 -o 2 -s 14 -dt 0.15 -vs 10
+//    ex10 -m ../../data/beam-tri.mesh  -r 2 -o 2 -s 17 -dt 0.01 -vs 30
+//    ex10 -m ../../data/beam-hex.mesh  -r 1 -o 2 -s 14 -dt 0.15 -vs 10
+//    ex10 -m ../../data/beam-quad-amr.mesh -r 2 -o 2 -s 12 -dt 0.15 -vs 10
 //
 // Description:  This examples solves a time dependent nonlinear elasticity
 //               problem of the form dv/dt = H(x) + S v, dx/dt = v, where H is a
@@ -53,7 +53,6 @@ using namespace std;
 using namespace mfem;
 
 class ReducedSystemOperator;
-class SundialsJacSolver;
 
 /** After spatial discretization, the hyperelastic model can be written as a
  *  system of ODEs:
@@ -92,12 +91,17 @@ protected:
 
    mutable Vector z; // auxiliary vector
 
+   SparseMatrix *grad_H;
+   SparseMatrix *Jacobian;
+
+   double saved_gamma; // saved gamma value from implicit setup
+
 public:
    /// Solver type to use in the ImplicitSolve() method, used by SDIRK methods.
    enum NonlinearSolverType
    {
       NEWTON = 0, ///< Use MFEM's plain NewtonSolver
-      KINSOL = 1  ///< Use SUNDIALS' KINSOL (through MFEM's class KinSolver)
+      KINSOL = 1  ///< Use SUNDIALS' KINSOL (through MFEM's class KINSolver)
    };
 
    HyperelasticOperator(FiniteElementSpace &f, Array<int> &ess_bdr,
@@ -106,15 +110,41 @@ public:
 
    /// Compute the right-hand side of the ODE system.
    virtual void Mult(const Vector &vx, Vector &dvx_dt) const;
+
    /** Solve the Backward-Euler equation: k = f(x + dt*k, t), for the unknown k.
        This is the only requirement for high-order SDIRK implicit integration.*/
    virtual void ImplicitSolve(const double dt, const Vector &x, Vector &k);
 
-   /** Connect the Jacobian linear system solver (SundialsJacSolver) used by
-       SUNDIALS' CVODE and ARKODE time integrators to the internal objects
-       created by HyperelasticOperator. This method is called by the InitSystem
-       method of SundialsJacSolver. */
-   void InitSundialsJacSolver(SundialsJacSolver &sjsolv);
+
+   /// Custom Jacobian system solver for the SUNDIALS time integrators.
+   /** For the ODE system represented by HyperelasticOperator
+
+          M dv/dt = -(H(x) + S*v)
+          dx/dt = v,
+
+       this class facilitates the solution of linear systems of the form
+
+           (M + γS) yv + γJ yx = M bv,   J=(dH/dx)(x)
+                - γ yv +    yx =   bx
+
+       for given bv, bx, x, and γ = GetTimeStep(). */
+
+   /** Linear solve applicable to the SUNDIALS format.
+       Solves (Mass - dt J) y = Mass b, where in our case:
+       Mass = | M  0 |  J = | -S  -grad_H |  y = | v_hat |  b = | b_v |
+              | 0  I |      |  I     0    |      | x_hat |      | b_x |
+       The result replaces the rhs b.
+       We substitute x_hat = b_x + dt v_hat and solve
+       (M + dt S + dt^2 grad_H) v_hat = M b_v - dt grad_H b_x. */
+
+   /** Setup the linear system. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSetup(const Vector &y, const Vector &fy,
+                                int jok, int *jcur, double gamma);
+
+   /** Solve the linear system. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSolve(const Vector &b, Vector &x, double tol);
 
    double ElasticEnergy(const Vector &x) const;
    double KineticEnergy(const Vector &v) const;
@@ -152,53 +182,6 @@ public:
    virtual ~ReducedSystemOperator();
 };
 
-/// Custom Jacobian system solver for the SUNDIALS time integrators.
-/** For the ODE system represented by HyperelasticOperator
-
-        M dv/dt = -(H(x) + S*v)
-          dx/dt = v,
-
-    this class facilitates the solution of linear systems of the form
-
-        (M + γS) yv + γJ yx = M bv,   J=(dH/dx)(x)
-             - γ yv +    yx =   bx
-
-    for given bv, bx, x, and γ = GetTimeStep(). */
-class SundialsJacSolver : public SundialsODELinearSolver
-{
-private:
-   BilinearForm *M, *S;
-   NonlinearForm *H;
-   SparseMatrix *grad_H, *Jacobian;
-   Solver *J_solver;
-
-public:
-   SundialsJacSolver()
-      : M(), S(), H(), grad_H(), Jacobian(), J_solver() { }
-
-   /// Connect the solver to the objects created inside HyperelasticOperator.
-   void SetOperators(BilinearForm &M_, BilinearForm &S_,
-                     NonlinearForm &H_, Solver &solver)
-   {
-      M = &M_; S = &S_; H = &H_; J_solver = &solver;
-   }
-
-   /** Linear solve applicable to the SUNDIALS format.
-       Solves (Mass - dt J) y = Mass b, where in our case:
-       Mass = | M  0 |  J = | -S  -grad_H |  y = | v_hat |  b = | b_v |
-              | 0  I |      |  I     0    |      | x_hat |      | b_x |
-       The result replaces the rhs b.
-       We substitute x_hat = b_x + dt v_hat and solve
-       (M + dt S + dt^2 grad_H) v_hat = M b_v - dt grad_H b_x. */
-   int InitSystem(void *sundials_mem);
-   int SetupSystem(void *sundials_mem, int conv_fail,
-                   const Vector &y_pred, const Vector &f_pred, int &jac_cur,
-                   Vector &v_temp1, Vector &v_temp2, Vector &v_temp3);
-   int SolveSystem(void *sundials_mem, Vector &b, const Vector &weight,
-                   const Vector &y_cur, const Vector &f_cur);
-   int FreeSystem(void *sundials_mem);
-};
-
 
 /** Function representing the elastic energy density for the given hyperelastic
     model+deformation. Used in HyperelasticOperator::GetElasticEnergyDensity. */
@@ -243,6 +226,12 @@ int main(int argc, char *argv[])
 
    // Relative and absolute tolerances for CVODE and ARKODE.
    const double reltol = 1e-1, abstol = 1e-1;
+   // Since this example uses the loose tolerances defined above, it is
+   // necessary to lower the linear solver tolerance for CVODE which is relative
+   // to the above tolerances.
+   const double cvode_eps_lin = 1e-4;
+   // Similarly, the nonlinear tolerance for ARKODE needs to be tightened.
+   const double arkode_eps_nonlin = 1e-6;
 
    OptionsParser args(argc, argv);
    args.AddOption(&mesh_file, "-m", "--mesh",
@@ -252,15 +241,24 @@ int main(int argc, char *argv[])
    args.AddOption(&order, "-o", "--order",
                   "Order (degree) of the finite elements.");
    args.AddOption(&ode_solver_type, "-s", "--ode-solver",
-                  "ODE solver: 1 - Backward Euler, 2 - SDIRK2, 3 - SDIRK3,\n\t"
-                  "            4 - CVODE implicit, approximate Jacobian,\n\t"
-                  "            5 - CVODE implicit, specified Jacobian,\n\t"
-                  "            6 - ARKODE implicit, approximate Jacobian,\n\t"
-                  "            7 - ARKODE implicit, specified Jacobian,\n\t"
-                  "            11 - Forward Euler, 12 - RK2,\n\t"
-                  "            13 - RK3 SSP, 14 - RK4,\n\t"
-                  "            15 - CVODE (adaptive order) explicit,\n\t"
-                  "            16 - ARKODE default (4th order) explicit.");
+                  "ODE solver:\n\t"
+                  "1  - Backward Euler,\n\t"
+                  "2  - SDIRK2, L-stable\n\t"
+                  "3  - SDIRK3, L-stable\n\t"
+                  "4  - Implicit Midpoint,\n\t"
+                  "5  - SDIRK2, A-stable,\n\t"
+                  "6  - SDIRK3, A-stable,\n\t"
+                  "7  - Forward Euler,\n\t"
+                  "8  - RK2,\n\t"
+                  "9  - RK3 SSP,\n\t"
+                  "10 - RK4,\n\t"
+                  "11 - CVODE implicit BDF, approximate Jacobian,\n\t"
+                  "12 - CVODE implicit BDF, specified Jacobian,\n\t"
+                  "13 - CVODE implicit ADAMS, approximate Jacobian,\n\t"
+                  "14 - CVODE implicit ADAMS, specified Jacobian,\n\t"
+                  "15 - ARKODE implicit, approximate Jacobian,\n\t"
+                  "16 - ARKODE implicit, specified Jacobian,\n\t"
+                  "17 - ARKODE explicit, 4th order.");
    args.AddOption(&nls, "-nls", "--nonlinear-solver",
                   "Nonlinear systems solver: "
                   "\"newton\" (plain Newton) or \"kinsol\" (KINSOL).");
@@ -287,72 +285,19 @@ int main(int argc, char *argv[])
    }
    args.PrintOptions(cout);
 
+   // check for vaild ODE solver option
+   if (ode_solver_type < 1 || ode_solver_type > 17)
+   {
+      cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
+      return 1;
+   }
+
    // 2. Read the mesh from the given mesh file. We can handle triangular,
    //    quadrilateral, tetrahedral and hexahedral meshes with the same code.
    Mesh *mesh = new Mesh(mesh_file, 1, 1);
    int dim = mesh->Dimension();
 
-   // 3. Define the ODE solver used for time integration. Several implicit
-   //    singly diagonal implicit Runge-Kutta (SDIRK) methods, as well as
-   //    explicit Runge-Kutta methods are available.
-   ODESolver *ode_solver;
-   CVODESolver *cvode = NULL;
-   ARKODESolver *arkode = NULL;
-   SundialsJacSolver *sjsolver = NULL;
-   switch (ode_solver_type)
-   {
-      // Implicit L-stable methods
-      case 1: ode_solver = new BackwardEulerSolver; break;
-      case 2: ode_solver = new SDIRK23Solver(2); break;
-      case 3: ode_solver = new SDIRK33Solver; break;
-      case 4:
-      case 5:
-         cvode = new CVODESolver(CV_BDF, CV_NEWTON);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         if (ode_solver_type == 5)
-         {
-            sjsolver = new SundialsJacSolver;
-            cvode->SetLinearSolver(*sjsolver);
-         }
-         ode_solver = cvode; break;
-      case 6:
-      case 7:
-         arkode = new ARKODESolver(ARKODESolver::IMPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         if (ode_solver_type == 7)
-         {
-            // Custom Jacobian inversion.
-            sjsolver = new SundialsJacSolver;
-            arkode->SetLinearSolver(*sjsolver);
-         }
-         ode_solver = arkode; break;
-      // Explicit methods
-      case 11: ode_solver = new ForwardEulerSolver; break;
-      case 12: ode_solver = new RK2Solver(0.5); break; // midpoint method
-      case 13: ode_solver = new RK3SSPSolver; break;
-      case 14: ode_solver = new RK4Solver; break;
-      case 15:
-         cvode = new CVODESolver(CV_ADAMS, CV_FUNCTIONAL);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 16:
-         arkode = new ARKODESolver(ARKODESolver::IMPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         ode_solver = arkode; break;
-      // Implicit A-stable methods (not L-stable)
-      case 22: ode_solver = new ImplicitMidpointSolver; break;
-      case 23: ode_solver = new SDIRK23Solver; break;
-      case 24: ode_solver = new SDIRK34Solver; break;
-      default:
-         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
-         delete mesh;
-         return 3;
-   }
-
+   // 3. Setup the nonlinear solver
    map<string,HyperelasticOperator::NonlinearSolverType> nls_map;
    nls_map["newton"] = HyperelasticOperator::NEWTON;
    nls_map["kinsol"] = HyperelasticOperator::KINSOL;
@@ -439,11 +384,82 @@ int main(int argc, char *argv[])
    cout << "initial kinetic energy (KE) = " << ke0 << endl;
    cout << "initial   total energy (TE) = " << (ee0 + ke0) << endl;
 
+   // 8. Define the ODE solver used for time integration. Several implicit
+   //    singly diagonal implicit Runge-Kutta (SDIRK) methods, as well as
+   //    explicit Runge-Kutta methods are available.
    double t = 0.0;
    oper.SetTime(t);
-   ode_solver->Init(oper);
 
-   // 8. Perform time-integration (looping over the time iterations, ti, with a
+   ODESolver *ode_solver = NULL;
+   CVODESolver *cvode = NULL;
+   ARKStepSolver *arkode = NULL;
+   switch (ode_solver_type)
+   {
+      // Implicit L-stable methods
+      case 1:  ode_solver = new BackwardEulerSolver; break;
+      case 2:  ode_solver = new SDIRK23Solver(2); break;
+      case 3:  ode_solver = new SDIRK33Solver; break;
+      // Implicit A-stable methods (not L-stable)
+      case 4:  ode_solver = new ImplicitMidpointSolver; break;
+      case 5:  ode_solver = new SDIRK23Solver; break;
+      case 6:  ode_solver = new SDIRK34Solver; break;
+      // Explicit methods
+      case 7:  ode_solver = new ForwardEulerSolver; break;
+      case 8:  ode_solver = new RK2Solver(0.5); break; // midpoint method
+      case 9:  ode_solver = new RK3SSPSolver; break;
+      case 10: ode_solver = new RK4Solver; break;
+      // CVODE BDF
+      case 11:
+      case 12:
+         cvode = new CVODESolver(CV_BDF);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         CVodeSetEpsLin(cvode->GetMem(), cvode_eps_lin);
+         cvode->SetMaxStep(dt);
+         if (ode_solver_type == 11)
+         {
+            cvode->UseSundialsLinearSolver();
+         }
+         ode_solver = cvode; break;
+      // CVODE Adams
+      case 13:
+      case 14:
+         cvode = new CVODESolver(CV_ADAMS);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         CVodeSetEpsLin(cvode->GetMem(), cvode_eps_lin);
+         cvode->SetMaxStep(dt);
+         if (ode_solver_type == 13)
+         {
+            cvode->UseSundialsLinearSolver();
+         }
+         ode_solver = cvode; break;
+      // ARKStep Implicit methods
+      case 15:
+      case 16:
+         arkode = new ARKStepSolver(ARKStepSolver::IMPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         ARKStepSetNonlinConvCoef(arkode->GetMem(), arkode_eps_nonlin);
+         arkode->SetMaxStep(dt);
+         if (ode_solver_type == 15)
+         {
+            arkode->UseSundialsLinearSolver();
+         }
+         ode_solver = arkode; break;
+      // ARKStep Explicit methods
+      case 17:
+         arkode = new ARKStepSolver(ARKStepSolver::EXPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         ode_solver = arkode; break;
+   }
+
+   // Initialize MFEM integrators, SUNDIALS integrators are initialized above
+   if (ode_solver_type < 11) { ode_solver->Init(oper); }
+
+   // 9. Perform time-integration (looping over the time iterations, ti, with a
    //    time-step dt).
    bool last_step = false;
    for (int ti = 1; !last_step; ti++)
@@ -478,7 +494,7 @@ int main(int argc, char *argv[])
       }
    }
 
-   // 9. Save the displaced mesh, the velocity and elastic energy.
+   // 10. Save the displaced mesh, the velocity and elastic energy.
    {
       v.SetFromTrueVector(); x.SetFromTrueVector();
       GridFunction *nodes = &x;
@@ -497,9 +513,8 @@ int main(int argc, char *argv[])
       w.Save(ee_ofs);
    }
 
-   // 10. Free the used memory.
+   // 11. Free the used memory.
    delete ode_solver;
-   delete sjsolver;
    delete mesh;
 
    return 0;
@@ -579,81 +594,14 @@ ReducedSystemOperator::~ReducedSystemOperator()
 }
 
 
-int SundialsJacSolver::InitSystem(void *sundials_mem)
-{
-   TimeDependentOperator *td_oper = GetTimeDependentOperator(sundials_mem);
-   HyperelasticOperator *he_oper;
-
-   // During development, we use dynamic_cast<> to ensure the setup is correct:
-   he_oper = dynamic_cast<HyperelasticOperator*>(td_oper);
-   MFEM_VERIFY(he_oper, "operator is not HyperelasticOperator");
-
-   // When the implementation is finalized, we can switch to static_cast<>:
-   // he_oper = static_cast<HyperelasticOperator*>(td_oper);
-
-   he_oper->InitSundialsJacSolver(*this);
-   return 0;
-}
-
-int SundialsJacSolver::SetupSystem(void *sundials_mem, int conv_fail,
-                                   const Vector &y_pred, const Vector &f_pred,
-                                   int &jac_cur, Vector &v_temp1,
-                                   Vector &v_temp2, Vector &v_temp3)
-{
-   int sc = y_pred.Size() / 2;
-   const Vector x(y_pred.GetData() + sc, sc);
-   double dt = GetTimeStep(sundials_mem);
-
-   // J = M + dt*(S + dt*grad(H))
-   delete Jacobian;
-   Jacobian = Add(1.0, M->SpMat(), dt, S->SpMat());
-   grad_H = dynamic_cast<SparseMatrix *>(&H->GetGradient(x));
-   Jacobian->Add(dt * dt, *grad_H);
-
-   J_solver->SetOperator(*Jacobian);
-
-   jac_cur = 1;
-   return 0;
-}
-
-int SundialsJacSolver::SolveSystem(void *sundials_mem, Vector &b,
-                                   const Vector &weight, const Vector &y_cur,
-                                   const Vector &f_cur)
-{
-   int sc = b.Size() / 2;
-   // Vector x(y_cur.GetData() + sc, sc);
-   Vector b_v(b.GetData() +  0, sc);
-   Vector b_x(b.GetData() + sc, sc);
-   Vector rhs(sc);
-   double dt = GetTimeStep(sundials_mem);
-
-   // rhs = M b_v - dt*grad(H) b_x
-   grad_H->Mult(b_x, rhs);
-   rhs *= -dt;
-   M->AddMult(b_v, rhs);
-
-   J_solver->iterative_mode = false;
-   J_solver->Mult(rhs, b_v);
-
-   b_x.Add(dt, b_v);
-
-   return 0;
-}
-
-int SundialsJacSolver::FreeSystem(void *sundials_mem)
-{
-   delete Jacobian;
-   return 0;
-}
-
-
 HyperelasticOperator::HyperelasticOperator(FiniteElementSpace &f,
                                            Array<int> &ess_bdr, double visc,
                                            double mu, double K,
                                            NonlinearSolverType nls_type)
    : TimeDependentOperator(2*f.GetTrueVSize(), 0.0), fespace(f),
      M(&fespace), S(&fespace), H(&fespace),
-     viscosity(visc), z(height/2)
+     viscosity(visc), z(height/2),
+     grad_H(NULL), Jacobian(NULL)
 {
    const double rel_tol = 1e-8;
    const int skip_zero_entries = 0;
@@ -702,23 +650,24 @@ HyperelasticOperator::HyperelasticOperator(FiniteElementSpace &f,
 
    if (nls_type == KINSOL)
    {
-      KinSolver *kinsolver = new KinSolver(KIN_NONE, true);
-      kinsolver->SetMaxSetupCalls(4);
+      KINSolver *kinsolver = new KINSolver(KIN_NONE, true);
       newton_solver = kinsolver;
+      newton_solver->SetOperator(*reduced_oper);
       newton_solver->SetMaxIter(200);
       newton_solver->SetRelTol(rel_tol);
       newton_solver->SetPrintLevel(0);
+      kinsolver->SetMaxSetupCalls(4);
    }
    else
    {
       newton_solver = new NewtonSolver();
+      newton_solver->SetOperator(*reduced_oper);
       newton_solver->SetMaxIter(10);
       newton_solver->SetRelTol(rel_tol);
       newton_solver->SetPrintLevel(-1);
    }
    newton_solver->SetSolver(*J_solver);
    newton_solver->iterative_mode = false;
-   newton_solver->SetOperator(*reduced_oper);
 }
 
 void HyperelasticOperator::Mult(const Vector &vx, Vector &dvx_dt) const
@@ -768,9 +717,53 @@ void HyperelasticOperator::ImplicitSolve(const double dt,
    add(v, dt, dv_dt, dx_dt);
 }
 
-void HyperelasticOperator::InitSundialsJacSolver(SundialsJacSolver &sjsolv)
+int HyperelasticOperator::SUNImplicitSetup(const Vector &y,
+                                           const Vector &fy, int jok, int *jcur,
+                                           double gamma)
 {
-   sjsolv.SetOperators(M, S, H, *J_solver);
+   int sc = y.Size() / 2;
+   const Vector x(y.GetData() + sc, sc);
+
+   // J = M + dt*(S + dt*grad(H))
+   if (Jacobian) { delete Jacobian; }
+   Jacobian = Add(1.0, M.SpMat(), gamma, S.SpMat());
+   grad_H = dynamic_cast<SparseMatrix *>(&H.GetGradient(x));
+   Jacobian->Add(gamma * gamma, *grad_H);
+
+   // Set Jacobian solve operator
+   J_solver->SetOperator(*Jacobian);
+
+   // Indicate that the Jacobian was updated
+   *jcur = 1;
+
+   // Save gamma for use in solve
+   saved_gamma = gamma;
+
+   // Return success
+   return 0;
+}
+
+int HyperelasticOperator::SUNImplicitSolve(const Vector &b, Vector &x,
+                                           double tol)
+{
+   int sc = b.Size() / 2;
+   Vector b_v(b.GetData() +  0, sc);
+   Vector b_x(b.GetData() + sc, sc);
+   Vector x_v(x.GetData() +  0, sc);
+   Vector x_x(x.GetData() + sc, sc);
+   Vector rhs(sc);
+
+   // rhs = M b_v - dt*grad(H) b_x
+   grad_H->Mult(b_x, rhs);
+   rhs *= -saved_gamma;
+   M.AddMult(b_v, rhs);
+
+   J_solver->iterative_mode = false;
+   J_solver->Mult(rhs, x_v);
+
+   add(b_x, saved_gamma, x_v, x_x);
+
+   return 0;
 }
 
 double HyperelasticOperator::ElasticEnergy(const Vector &x) const
@@ -792,6 +785,7 @@ void HyperelasticOperator::GetElasticEnergyDensity(
 
 HyperelasticOperator::~HyperelasticOperator()
 {
+   delete Jacobian;
    delete newton_solver;
    delete J_solver;
    delete J_prec;

examples/sundials/ex10p.cpp

@@ -4,16 +4,16 @@
 // Compile with: make ex10p
 //
 // Sample runs:
-//    mpirun -np 4 ex10p -m ../../data/beam-quad.mesh -rp 1 -o 2 -s  5 -dt 0.15 -vs 10
-//    mpirun -np 4 ex10p -m ../../data/beam-tri.mesh  -rp 1 -o 2 -s  7 -dt 0.25  -vs 10
-//    mpirun -np 4 ex10p -m ../../data/beam-hex.mesh  -rp 0 -o 2 -s  5 -dt 0.15  -vs 10
+//    mpirun -np 4 ex10p -m ../../data/beam-quad.mesh -rp 1 -o 2 -s 12 -dt 0.15 -vs 10
+//    mpirun -np 4 ex10p -m ../../data/beam-tri.mesh  -rp 1 -o 2 -s 16 -dt 0.25 -vs 10
+//    mpirun -np 4 ex10p -m ../../data/beam-hex.mesh  -rp 0 -o 2 -s 12 -dt 0.15 -vs 10
 //    mpirun -np 4 ex10p -m ../../data/beam-tri.mesh  -rp 1 -o 2 -s  2 -dt 3 -nls kinsol
 //    mpirun -np 4 ex10p -m ../../data/beam-quad.mesh -rp 1 -o 2 -s  2 -dt 3 -nls kinsol
 //    mpirun -np 4 ex10p -m ../../data/beam-hex.mesh  -rs 1 -o 2 -s  2 -dt 3 -nls kinsol
-//    mpirun -np 4 ex10p -m ../../data/beam-quad.mesh -rp 1 -o 2 -s 15 -dt 3e-3 -vs 120
-//    mpirun -np 4 ex10p -m ../../data/beam-tri.mesh  -rp 1 -o 2 -s 16 -dt 5e-3 -vs 60
-//    mpirun -np 4 ex10p -m ../../data/beam-hex.mesh  -rp 0 -o 2 -s 15 -dt 5e-3 -vs 60
-//    mpirun -np 4 ex10p -m ../../data/beam-quad-amr.mesh -rp 1 -o 2 -s 5 -dt 0.15 -vs 10
+//    mpirun -np 4 ex10p -m ../../data/beam-quad.mesh -rp 1 -o 2 -s 14 -dt 0.15 -vs 10
+//    mpirun -np 4 ex10p -m ../../data/beam-tri.mesh  -rp 1 -o 2 -s 17 -dt 5e-3 -vs 60
+//    mpirun -np 4 ex10p -m ../../data/beam-hex.mesh  -rp 0 -o 2 -s 14 -dt 0.15 -vs 10
+//    mpirun -np 4 ex10p -m ../../data/beam-quad-amr.mesh -rp 1 -o 2 -s 12 -dt 0.15 -vs 10
 //
 // Description:  This examples solves a time dependent nonlinear elasticity
 //               problem of the form dv/dt = H(x) + S v, dx/dt = v, where H is a
@@ -53,7 +53,6 @@ using namespace std;
 using namespace mfem;
 
 class ReducedSystemOperator;
-class SundialsJacSolver;
 
 /** After spatial discretization, the hyperelastic model can be written as a
  *  system of ODEs:
@@ -94,12 +93,17 @@ protected:
 
    mutable Vector z; // auxiliary vector
 
+   const SparseMatrix *local_grad_H;
+   HypreParMatrix *Jacobian;
+
+   double saved_gamma; // saved gamma value from implicit setup
+
 public:
    /// Solver type to use in the ImplicitSolve() method, used by SDIRK methods.
    enum NonlinearSolverType
    {
       NEWTON = 0, ///< Use MFEM's plain NewtonSolver
-      KINSOL = 1  ///< Use SUNDIALS' KINSOL (through MFEM's class KinSolver)
+      KINSOL = 1  ///< Use SUNDIALS' KINSOL (through MFEM's class KINSolver)
    };
 
    HyperelasticOperator(ParFiniteElementSpace &f, Array<int> &ess_bdr,
@@ -108,15 +112,41 @@ public:
 
    /// Compute the right-hand side of the ODE system.
    virtual void Mult(const Vector &vx, Vector &dvx_dt) const;
+
    /** Solve the Backward-Euler equation: k = f(x + dt*k, t), for the unknown k.
        This is the only requirement for high-order SDIRK implicit integration.*/
    virtual void ImplicitSolve(const double dt, const Vector &x, Vector &k);
 
-   /** Connect the Jacobian linear system solver (SundialsJacSolver) used by
-       SUNDIALS' CVODE and ARKODE time integrators to the internal objects
-       created by HyperelasticOperator. This method is called by the InitSystem
-       method of SundialsJacSolver. */
-   void InitSundialsJacSolver(SundialsJacSolver &sjsolv);
+
+   /// Custom Jacobian system solver for the SUNDIALS time integrators.
+   /** For the ODE system represented by HyperelasticOperator
+
+          M dv/dt = -(H(x) + S*v)
+          dx/dt = v,
+
+       this class facilitates the solution of linear systems of the form
+
+           (M + γS) yv + γJ yx = M bv,   J=(dH/dx)(x)
+                - γ yv +    yx =   bx
+
+       for given bv, bx, x, and γ = GetTimeStep(). */
+
+   /** Linear solve applicable to the SUNDIALS format.
+       Solves (Mass - dt J) y = Mass b, where in our case:
+       Mass = | M  0 |  J = | -S  -grad_H |  y = | v_hat |  b = | b_v |
+              | 0  I |      |  I     0    |      | x_hat |      | b_x |
+       The result replaces the rhs b.
+       We substitute x_hat = b_x + dt v_hat and solve
+       (M + dt S + dt^2 grad_H) v_hat = M b_v - dt grad_H b_x. */
+
+   /** Setup the linear system. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSetup(const Vector &y, const Vector &fy,
+                                int jok, int *jcur, double gamma);
+
+   /** Solve the linear system. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSolve(const Vector &b, Vector &x, double tol);
 
    double ElasticEnergy(const ParGridFunction &x) const;
    double KineticEnergy(const ParGridFunction &v) const;
@@ -157,57 +187,6 @@ public:
    virtual ~ReducedSystemOperator();
 };
 
-/// Custom Jacobian system solver for the SUNDIALS time integrators.
-/** For the ODE system represented by HyperelasticOperator
-
-        M dv/dt = -(H(x) + S*v)
-          dx/dt = v,
-
-    this class facilitates the solution of linear systems of the form
-
-        (M + γS) yv + γJ yx = M bv,   J=(dH/dx)(x)
-             - γ yv +    yx =   bx
-
-    for given bv, bx, x, and γ = GetTimeStep(). */
-class SundialsJacSolver : public SundialsODELinearSolver
-{
-private:
-   ParBilinearForm *M, *S;
-   ParNonlinearForm *H;
-   const SparseMatrix *local_grad_H;
-   HypreParMatrix *Jacobian;
-   Solver *J_solver;
-   const Array<int> *ess_tdof_list;
-
-public:
-   SundialsJacSolver()
-      : M(), S(), H(), local_grad_H(), Jacobian(), J_solver() { }
-
-   /// Connect the solver to the objects created inside HyperelasticOperator.
-   void SetOperators(ParBilinearForm &M_, ParBilinearForm &S_,
-                     ParNonlinearForm &H_, Solver &solver,
-                     const Array<int> &ess_tdof_list_)
-   {
-      M = &M_; S = &S_; H = &H_; J_solver = &solver;
-      ess_tdof_list = &ess_tdof_list_;
-   }
-
-   /** Linear solve applicable to the SUNDIALS format.
-       Solves (Mass - dt J) y = Mass b, where in our case:
-       Mass = | M  0 |  J = | -S  -grad_H |  y = | v_hat |  b = | b_v |
-              | 0  I |      |  I     0    |      | x_hat |      | b_x |
-       The result replaces the rhs b.
-       We substitute x_hat = b_x + dt v_hat and solve
-       (M + dt S + dt^2 grad_H) v_hat = M b_v - dt grad_H b_x. */
-   int InitSystem(void *sundials_mem);
-   int SetupSystem(void *sundials_mem, int conv_fail,
-                   const Vector &y_pred, const Vector &f_pred, int &jac_cur,
-                   Vector &v_temp1, Vector &v_temp2, Vector &v_temp3);
-   int SolveSystem(void *sundials_mem, Vector &b, const Vector &weight,
-                   const Vector &y_cur, const Vector &f_cur);
-   int FreeSystem(void *sundials_mem);
-};
-
 
 /** Function representing the elastic energy density for the given hyperelastic
     model+deformation. Used in HyperelasticOperator::GetElasticEnergyDensity. */
@@ -259,6 +238,12 @@ int main(int argc, char *argv[])
 
    // Relative and absolute tolerances for CVODE and ARKODE.
    const double reltol = 1e-1, abstol = 1e-1;
+   // Since this example uses the loose tolerances defined above, it is
+   // necessary to lower the linear solver tolerance for CVODE which is relative
+   // to the above tolerances.
+   const double cvode_eps_lin = 1e-4;
+   // Similarly, the nonlinear tolerance for ARKODE needs to be tightened.
+   const double arkode_eps_nonlin = 1e-6;
 
    OptionsParser args(argc, argv);
    args.AddOption(&mesh_file, "-m", "--mesh",
@@ -270,15 +255,24 @@ int main(int argc, char *argv[])
    args.AddOption(&order, "-o", "--order",
                   "Order (degree) of the finite elements.");
    args.AddOption(&ode_solver_type, "-s", "--ode-solver",
-                  "ODE solver: 1 - Backward Euler, 2 - SDIRK2, 3 - SDIRK3,\n\t"
-                  "            4 - CVODE implicit, approximate Jacobian,\n\t"
-                  "            5 - CVODE implicit, specified Jacobian,\n\t"
-                  "            6 - ARKODE implicit, approximate Jacobian,\n\t"
-                  "            7 - ARKODE implicit, specified Jacobian,\n\t"
-                  "            11 - Forward Euler, 12 - RK2,\n\t"
-                  "            13 - RK3 SSP, 14 - RK4,\n\t"
-                  "            15 - CVODE (adaptive order) explicit,\n\t"
-                  "            16 - ARKODE default (4th order) explicit.");
+                  "ODE solver:\n\t"
+                  "1  - Backward Euler,\n\t"
+                  "2  - SDIRK2, L-stable\n\t"
+                  "3  - SDIRK3, L-stable\n\t"
+                  "4  - Implicit Midpoint,\n\t"
+                  "5  - SDIRK2, A-stable,\n\t"
+                  "6  - SDIRK3, A-stable,\n\t"
+                  "7  - Forward Euler,\n\t"
+                  "8  - RK2,\n\t"
+                  "9  - RK3 SSP,\n\t"
+                  "10 - RK4,\n\t"
+                  "11 - CVODE implicit BDF, approximate Jacobian,\n\t"
+                  "12 - CVODE implicit BDF, specified Jacobian,\n\t"
+                  "13 - CVODE implicit ADAMS, approximate Jacobian,\n\t"
+                  "14 - CVODE implicit ADAMS, specified Jacobian,\n\t"
+                  "15 - ARKODE implicit, approximate Jacobian,\n\t"
+                  "16 - ARKODE implicit, specified Jacobian,\n\t"
+                  "17 - ARKODE explicit, 4th order.");
    args.AddOption(&nls, "-nls", "--nonlinear-solver",
                   "Nonlinear systems solver: "
                   "\"newton\" (plain Newton) or \"kinsol\" (KINSOL).");
@@ -312,76 +306,24 @@ int main(int argc, char *argv[])
       args.PrintOptions(cout);
    }
 
+   // check for vaild ODE solver option
+   if (ode_solver_type < 1 || ode_solver_type > 17)
+   {
+      if (myid == 0)
+      {
+         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
+      }
+      MPI_Finalize();
+      return 1;
+   }
+
    // 3. Read the serial mesh from the given mesh file on all processors. We can
    //    handle triangular, quadrilateral, tetrahedral and hexahedral meshes
    //    with the same code.
    Mesh *mesh = new Mesh(mesh_file, 1, 1);
    int dim = mesh->Dimension();
 
-   // 4. Define the ODE solver used for time integration. Several implicit
-   //    singly diagonal implicit Runge-Kutta (SDIRK) methods, as well as
-   //    explicit Runge-Kutta methods are available.
-   ODESolver *ode_solver;
-   CVODESolver *cvode = NULL;
-   ARKODESolver *arkode = NULL;
-   SundialsJacSolver *sjsolver = NULL;
-   switch (ode_solver_type)
-   {
-      // Implicit L-stable methods
-      case 1: ode_solver = new BackwardEulerSolver; break;
-      case 2: ode_solver = new SDIRK23Solver(2); break;
-      case 3: ode_solver = new SDIRK33Solver; break;
-      case 4:
-      case 5:
-         cvode = new CVODESolver(MPI_COMM_WORLD, CV_BDF, CV_NEWTON);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         if (ode_solver_type == 5)
-         {
-            sjsolver = new SundialsJacSolver;
-            cvode->SetLinearSolver(*sjsolver); // Custom Jacobian inversion.
-         }
-         ode_solver = cvode; break;
-      case 6:
-      case 7:
-         arkode = new ARKODESolver(MPI_COMM_WORLD, ARKODESolver::IMPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         if (ode_solver_type == 7)
-         {
-            sjsolver = new SundialsJacSolver;
-            arkode->SetLinearSolver(*sjsolver); // Custom Jacobian inversion.
-         }
-         ode_solver = arkode; break;
-      // Explicit methods
-      case 11: ode_solver = new ForwardEulerSolver; break;
-      case 12: ode_solver = new RK2Solver(0.5); break; // midpoint method
-      case 13: ode_solver = new RK3SSPSolver; break;
-      case 14: ode_solver = new RK4Solver; break;
-      case 15:
-         cvode = new CVODESolver(MPI_COMM_WORLD, CV_ADAMS, CV_FUNCTIONAL);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 16:
-         arkode = new ARKODESolver(MPI_COMM_WORLD, ARKODESolver::EXPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         ode_solver = arkode; break;
-      // Implicit A-stable methods (not L-stable)
-      case 22: ode_solver = new ImplicitMidpointSolver; break;
-      case 23: ode_solver = new SDIRK23Solver; break;
-      case 24: ode_solver = new SDIRK34Solver; break;
-      default:
-         if (myid == 0)
-         {
-            cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
-         }
-         delete mesh;
-         MPI_Finalize();
-         return 3;
-   }
-
+   // 4. Nonlinear solver
    map<string,HyperelasticOperator::NonlinearSolverType> nls_map;
    nls_map["newton"] = HyperelasticOperator::NEWTON;
    nls_map["kinsol"] = HyperelasticOperator::KINSOL;
@@ -391,7 +333,6 @@ int main(int argc, char *argv[])
       {
          cout << "Unknown type of nonlinear solver: " << nls << endl;
       }
-      delete ode_solver;
       delete mesh;
       MPI_Finalize();
       return 4;
@@ -495,11 +436,82 @@ int main(int argc, char *argv[])
       cout << "initial   total energy (TE) = " << (ee0 + ke0) << endl;
    }
 
+   // 10. Define the ODE solver used for time integration. Several implicit
+   //     singly diagonal implicit Runge-Kutta (SDIRK) methods, as well as
+   //     explicit Runge-Kutta methods are available.
    double t = 0.0;
    oper.SetTime(t);
-   ode_solver->Init(oper);
 
-   // 10. Perform time-integration
+   ODESolver *ode_solver = NULL;
+   CVODESolver *cvode = NULL;
+   ARKStepSolver *arkode = NULL;
+   switch (ode_solver_type)
+   {
+      // Implicit L-stable methods
+      case 1:  ode_solver = new BackwardEulerSolver; break;
+      case 2:  ode_solver = new SDIRK23Solver(2); break;
+      case 3:  ode_solver = new SDIRK33Solver; break;
+      // Implicit A-stable methods (not L-stable)
+      case 4:  ode_solver = new ImplicitMidpointSolver; break;
+      case 5:  ode_solver = new SDIRK23Solver; break;
+      case 6:  ode_solver = new SDIRK34Solver; break;
+      // Explicit methods
+      case 7:  ode_solver = new ForwardEulerSolver; break;
+      case 8:  ode_solver = new RK2Solver(0.5); break; // midpoint method
+      case 9:  ode_solver = new RK3SSPSolver; break;
+      case 10: ode_solver = new RK4Solver; break;
+      // CVODE BDF
+      case 11:
+      case 12:
+         cvode = new CVODESolver(MPI_COMM_WORLD, CV_BDF);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         CVodeSetEpsLin(cvode->GetMem(), cvode_eps_lin);
+         cvode->SetMaxStep(dt);
+         if (ode_solver_type == 11)
+         {
+            cvode->UseSundialsLinearSolver();
+         }
+         ode_solver = cvode; break;
+      // CVODE Adams
+      case 13:
+      case 14:
+         cvode = new CVODESolver(MPI_COMM_WORLD, CV_ADAMS);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         CVodeSetEpsLin(cvode->GetMem(), cvode_eps_lin);
+         cvode->SetMaxStep(dt);
+         if (ode_solver_type == 13)
+         {
+            cvode->UseSundialsLinearSolver();
+         }
+         ode_solver = cvode; break;
+      // ARKStep Implicit methods
+      case 15:
+      case 16:
+         arkode = new ARKStepSolver(MPI_COMM_WORLD, ARKStepSolver::IMPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         ARKStepSetNonlinConvCoef(arkode->GetMem(), arkode_eps_nonlin);
+         arkode->SetMaxStep(dt);
+         if (ode_solver_type == 15)
+         {
+            arkode->UseSundialsLinearSolver();
+         }
+         ode_solver = arkode; break;
+      // ARKStep Explicit methods
+      case 17:
+         arkode = new ARKStepSolver(MPI_COMM_WORLD, ARKStepSolver::EXPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         ode_solver = arkode; break;
+   }
+
+   // Initialize MFEM integrators, SUNDIALS integrators are initialized above
+   if (ode_solver_type < 11) { ode_solver->Init(oper); }
+
+   // 11. Perform time-integration
    //     (looping over the time iterations, ti, with a time-step dt).
    bool last_step = false;
    for (int ti = 1; !last_step; ti++)
@@ -538,7 +550,7 @@ int main(int argc, char *argv[])
       }
    }
 
-   // 11. Save the displaced mesh, the velocity and elastic energy.
+   // 12. Save the displaced mesh, the velocity and elastic energy.
    {
       v_gf.SetFromTrueVector(); x_gf.SetFromTrueVector();
       GridFunction *nodes = &x_gf;
@@ -563,9 +575,8 @@ int main(int argc, char *argv[])
       w_gf.Save(ee_ofs);
    }
 
-   // 12. Free the used memory.
+   // 13. Free the used memory.
    delete ode_solver;
-   delete sjsolver;
    delete pmesh;
 
    MPI_Finalize();
@@ -653,92 +664,14 @@ ReducedSystemOperator::~ReducedSystemOperator()
 }
 
 
-int SundialsJacSolver::InitSystem(void *sundials_mem)
-{
-   TimeDependentOperator *td_oper = GetTimeDependentOperator(sundials_mem);
-   HyperelasticOperator *he_oper;
-
-   // During development, we use dynamic_cast<> to ensure the setup is correct:
-   he_oper = dynamic_cast<HyperelasticOperator*>(td_oper);
-   MFEM_VERIFY(he_oper, "operator is not HyperelasticOperator");
-
-   // When the implementation is finalized, we can switch to static_cast<>:
-   // he_oper = static_cast<HyperelasticOperator*>(td_oper);
-
-   he_oper->InitSundialsJacSolver(*this);
-   return 0;
-}
-
-int SundialsJacSolver::SetupSystem(void *sundials_mem, int conv_fail,
-                                   const Vector &y_pred, const Vector &f_pred,
-                                   int &jac_cur, Vector &v_temp1,
-                                   Vector &v_temp2, Vector &v_temp3)
-{
-   int sc = y_pred.Size() / 2;
-   const Vector x(y_pred.GetData() + sc, sc);
-   double dt = GetTimeStep(sundials_mem);
-
-   // J = M + dt*(S + dt*grad(H))
-   delete Jacobian;
-   SparseMatrix *localJ = Add(1.0, M->SpMat(), dt, S->SpMat());
-   local_grad_H = &H->GetLocalGradient(x);
-   localJ->Add(dt*dt, *local_grad_H);
-   Jacobian = M->ParallelAssemble(localJ);
-   delete localJ;
-   HypreParMatrix *Je = Jacobian->EliminateRowsCols(*ess_tdof_list);
-   delete Je;
-
-   J_solver->SetOperator(*Jacobian);
-
-   jac_cur = 1;
-   return 0;
-}
-
-int SundialsJacSolver::SolveSystem(void *sundials_mem, Vector &b,
-                                   const Vector &weight, const Vector &y_cur,
-                                   const Vector &f_cur)
-{
-   int sc = b.Size() / 2;
-   ParFiniteElementSpace *fes = H->ParFESpace();
-   // Vector x(y_cur.GetData() + sc, sc);
-   Vector b_v(b.GetData() +  0, sc);
-   Vector b_x(b.GetData() + sc, sc);
-   Vector rhs(sc);
-   double dt = GetTimeStep(sundials_mem);
-
-   // We can assume that b_v and b_x have zeros at essential tdofs.
-
-   // rhs = M b_v - dt*grad(H) b_x
-   ParGridFunction lb_x(fes), lrhs(fes);
-   lb_x.Distribute(b_x);
-   local_grad_H->Mult(lb_x, lrhs);
-   lrhs.ParallelAssemble(rhs);
-   rhs *= -dt;
-   M->TrueAddMult(b_v, rhs);
-   rhs.SetSubVector(*ess_tdof_list, 0.0);
-
-   J_solver->iterative_mode = false;
-   J_solver->Mult(rhs, b_v);
-
-   b_x.Add(dt, b_v);
-
-   return 0;
-}
-
-int SundialsJacSolver::FreeSystem(void *sundials_mem)
-{
-   delete Jacobian;
-   return 0;
-}
-
-
 HyperelasticOperator::HyperelasticOperator(ParFiniteElementSpace &f,
                                            Array<int> &ess_bdr, double visc,
                                            double mu, double K,
                                            NonlinearSolverType nls_type)
    : TimeDependentOperator(2*f.TrueVSize(), 0.0), fespace(f),
      M(&fespace), S(&fespace), H(&fespace),
-     viscosity(visc), M_solver(f.GetComm()), z(height/2)
+     viscosity(visc), M_solver(f.GetComm()), z(height/2),
+     local_grad_H(NULL), Jacobian(NULL)
 {
    const double rel_tol = 1e-8;
    const int skip_zero_entries = 0;
@@ -788,23 +721,24 @@ HyperelasticOperator::HyperelasticOperator(ParFiniteElementSpace &f,
 
    if (nls_type == KINSOL)
    {
-      KinSolver *kinsolver = new KinSolver(f.GetComm(), KIN_NONE, true);
-      kinsolver->SetMaxSetupCalls(4);
+      KINSolver *kinsolver = new KINSolver(f.GetComm(), KIN_NONE, true);
       newton_solver = kinsolver;
+      newton_solver->SetOperator(*reduced_oper);
       newton_solver->SetMaxIter(200);
       newton_solver->SetRelTol(rel_tol);
       newton_solver->SetPrintLevel(0);
+      kinsolver->SetMaxSetupCalls(4);
    }
    else
    {
       newton_solver = new NewtonSolver(f.GetComm());
+      newton_solver->SetOperator(*reduced_oper);
       newton_solver->SetMaxIter(10);
       newton_solver->SetRelTol(rel_tol);
       newton_solver->SetPrintLevel(-1);
    }
    newton_solver->SetSolver(*J_solver);
    newton_solver->iterative_mode = false;
-   newton_solver->SetOperator(*reduced_oper);
 }
 
 void HyperelasticOperator::Mult(const Vector &vx, Vector &dvx_dt) const
@@ -858,9 +792,64 @@ void HyperelasticOperator::ImplicitSolve(const double dt,
    add(v, dt, dv_dt, dx_dt);
 }
 
-void HyperelasticOperator::InitSundialsJacSolver(SundialsJacSolver &sjsolv)
+int HyperelasticOperator::SUNImplicitSetup(const Vector &y,
+                                           const Vector &fy, int jok, int *jcur,
+                                           double gamma)
 {
-   sjsolv.SetOperators(M, S, H, *J_solver, ess_tdof_list);
+   int sc = y.Size() / 2;
+   const Vector x(y.GetData() + sc, sc);
+
+   // J = M + dt*(S + dt*grad(H))
+   if (Jacobian) { delete Jacobian; }
+   SparseMatrix *localJ = Add(1.0, M.SpMat(), gamma, S.SpMat());
+   local_grad_H = &H.GetLocalGradient(x);
+   localJ->Add(gamma*gamma, *local_grad_H);
+   Jacobian = M.ParallelAssemble(localJ);
+   delete localJ;
+   HypreParMatrix *Je = Jacobian->EliminateRowsCols(ess_tdof_list);
+   delete Je;
+
+   // Set Jacobian solve operator
+   J_solver->SetOperator(*Jacobian);
+
+   // Indicate that the Jacobian was updated
+   *jcur = 1;
+
+   // Save gamma for use in solve
+   saved_gamma = gamma;
+
+   // Return success
+   return 0;
+}
+
+int HyperelasticOperator::SUNImplicitSolve(const Vector &b, Vector &x,
+                                           double tol)
+{
+   int sc = b.Size() / 2;
+   ParFiniteElementSpace *fes = H.ParFESpace();
+   Vector b_v(b.GetData() +  0, sc);
+   Vector b_x(b.GetData() + sc, sc);
+   Vector x_v(x.GetData() +  0, sc);
+   Vector x_x(x.GetData() + sc, sc);
+   Vector rhs(sc);
+
+   // We can assume that b_v and b_x have zeros at essential tdofs.
+
+   // rhs = M b_v - dt*grad(H) b_x
+   ParGridFunction lb_x(fes), lrhs(fes);
+   lb_x.Distribute(b_x);
+   local_grad_H->Mult(lb_x, lrhs);
+   lrhs.ParallelAssemble(rhs);
+   rhs *= -saved_gamma;
+   M.TrueAddMult(b_v, rhs);
+   rhs.SetSubVector(ess_tdof_list, 0.0);
+
+   J_solver->iterative_mode = false;
+   J_solver->Mult(rhs, x_v);
+
+   add(b_x, saved_gamma, x_v, x_x);
+
+   return 0;
 }
 
 double HyperelasticOperator::ElasticEnergy(const ParGridFunction &x) const
@@ -886,6 +875,7 @@ void HyperelasticOperator::GetElasticEnergyDensity(
 
 HyperelasticOperator::~HyperelasticOperator()
 {
+   delete Jacobian;
    delete newton_solver;
    delete J_solver;
    delete J_prec;

examples/sundials/ex16.cpp

@@ -7,9 +7,9 @@
 //               ex16 -m ../../data/inline-tri.mesh
 //               ex16 -m ../../data/disc-nurbs.mesh -tf 2
 //               ex16 -s 12 -a 0.0 -k 1.0
-//               ex16 -s 1 -a 1.0 -k 0.0 -dt 1e-4 -tf 5e-2 -vs 25
-//               ex16 -s 2 -a 0.5 -k 0.5 -o 4 -dt 1e-4 -tf 2e-2 -vs 25
-//               ex16 -s 3 -dt 1.0e-4 -tf 4.0e-2 -vs 40
+//               ex16 -s 8 -a 1.0 -k 0.0 -dt 1e-4 -tf 5e-2 -vs 25
+//               ex16 -s 9 -a 0.5 -k 0.5 -o 4 -dt 1e-4 -tf 2e-2 -vs 25
+//               ex16 -s 10 -dt 1.0e-4 -tf 4.0e-2 -vs 40
 //               ex16 -m ../../data/fichera-q2.mesh
 //               ex16 -m ../../data/escher.mesh
 //               ex16 -m ../../data/beam-tet.mesh -tf 10 -dt 0.1
@@ -58,7 +58,6 @@ protected:
 
    SparseMatrix Mmat, Kmat;
    SparseMatrix *T; // T = M + dt K
-   double current_dt;
 
    CGSolver M_solver; // Krylov solver for inverting the mass matrix M
    DSmoother M_prec;  // Preconditioner for the mass matrix M
@@ -75,13 +74,30 @@ public:
                       const Vector &u);
 
    virtual void Mult(const Vector &u, Vector &du_dt) const;
+
    /** Solve the Backward-Euler equation: k = f(u + dt*k, t), for the unknown k.
        This is the only requirement for high-order SDIRK implicit integration.*/
    virtual void ImplicitSolve(const double dt, const Vector &u, Vector &k);
 
-   /** Solve the system (M + dt K) y = M b. The result y replaces the input b.
-       This method is used by the implicit SUNDIALS solvers. */
-   void SundialsSolve(const double dt, Vector &b);
+   /// Custom Jacobian system solver for the SUNDIALS time integrators.
+   /** For the ODE system represented by ConductionOperator
+
+           M du/dt = -K(u),
+
+       this class facilitates the solution of linear systems of the form
+
+           (M + γK) y = M b,
+
+       for given b, u (not used), and γ = GetTimeStep(). */
+
+   /** Setup the system (M + dt K) x = M b. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSetup(const Vector &x, const Vector &fx,
+                                int jok, int *jcur, double gamma);
+
+   /** Solve the system (M + dt K) x = M b. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSolve(const Vector &b, Vector &x, double tol);
 
    /// Update the diffusion BilinearForm K using the given true-dof vector `u`.
    void SetParameters(const Vector &u);
@@ -89,33 +105,6 @@ public:
    virtual ~ConductionOperator();
 };
 
-/// Custom Jacobian system solver for the SUNDIALS time integrators.
-/** For the ODE system represented by ConductionOperator
-
-        M du/dt = -K(u),
-
-    this class facilitates the solution of linear systems of the form
-
-        (M + γK) y = M b,
-
-    for given b, u (not used), and γ = GetTimeStep(). */
-class SundialsJacSolver : public SundialsODELinearSolver
-{
-private:
-   ConductionOperator *oper;
-
-public:
-   SundialsJacSolver() : oper(NULL) { }
-
-   int InitSystem(void *sundials_mem);
-   int SetupSystem(void *sundials_mem, int conv_fail,
-                   const Vector &y_pred, const Vector &f_pred, int &jac_cur,
-                   Vector &v_temp1, Vector &v_temp2, Vector &v_temp3);
-   int SolveSystem(void *sundials_mem, Vector &b, const Vector &weight,
-                   const Vector &y_cur, const Vector &f_cur);
-   int FreeSystem(void *sundials_mem);
-};
-
 double InitialTemperature(const Vector &x);
 
 int main(int argc, char *argv[])
@@ -124,7 +113,7 @@ int main(int argc, char *argv[])
    const char *mesh_file = "../../data/star.mesh";
    int ref_levels = 2;
    int order = 2;
-   int ode_solver_type = 11; // 11 = CVODE implicit
+   int ode_solver_type = 9; // CVODE implicit BDF
    double t_final = 0.5;
    double dt = 1.0e-2;
    double alpha = 1.0e-2;
@@ -147,12 +136,19 @@ int main(int argc, char *argv[])
    args.AddOption(&order, "-o", "--order",
                   "Order (degree) of the finite elements.");
    args.AddOption(&ode_solver_type, "-s", "--ode-solver",
-                  "ODE solver:\n"
-                  "\t 1/11 - CVODE (explicit/implicit),\n"
-                  "\t 2/12 - ARKODE (default explicit/implicit),\n"
-                  "\t 3 - ARKODE (Fehlberg-6-4-5)\n"
-                  "\t 4 - Forward Euler, 5 - RK2, 6 - RK3 SSP, 7 - RK4,\n"
-                  "\t 8 - Backward Euler, 9 - SDIRK23, 10 - SDIRK33.");
+                  "ODE solver:\n\t"
+                  "1  - Forward Euler,\n\t"
+                  "2  - RK2,\n\t"
+                  "3  - RK3 SSP,\n\t"
+                  "4  - RK4,\n\t"
+                  "5  - Backward Euler,\n\t"
+                  "6  - SDIRK 2,\n\t"
+                  "7  - SDIRK 3,\n\t"
+                  "8  - CVODE (implicit Adams),\n\t"
+                  "9  - CVODE (implicit BDF),\n\t"
+                  "10 - ARKODE (default explicit),\n\t"
+                  "11 - ARKODE (explicit Fehlberg-6-4-5),\n\t"
+                  "12 - ARKODE (default impicit).");
    args.AddOption(&t_final, "-tf", "--t-final",
                   "Final time; start time is 0.");
    args.AddOption(&dt, "-dt", "--time-step",
@@ -175,6 +171,11 @@ int main(int argc, char *argv[])
       args.PrintUsage(cout);
       return 1;
    }
+   if (ode_solver_type < 1 || ode_solver_type > 12)
+   {
+      cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
+      return 3;
+   }
    args.PrintOptions(cout);
 
    // 2. Read the mesh from the given mesh file. We can handle triangular,
@@ -182,61 +183,7 @@ int main(int argc, char *argv[])
    Mesh *mesh = new Mesh(mesh_file, 1, 1);
    int dim = mesh->Dimension();
 
-   // 3. Define the ODE solver used for time integration. Several
-   // SUNDIALS solvers are available, as well as included both
-   // explicit and implicit MFEM ODE solvers.
-   ODESolver *ode_solver = NULL;
-   CVODESolver *cvode = NULL;
-   ARKODESolver *arkode = NULL;
-   SundialsJacSolver sun_solver; // Used by the implicit SUNDIALS ode solvers.
-   switch (ode_solver_type)
-   {
-      // SUNDIALS solvers
-      case 1:
-         cvode = new CVODESolver(CV_ADAMS, CV_FUNCTIONAL);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 11:
-         cvode = new CVODESolver(CV_BDF, CV_NEWTON);
-         cvode->SetLinearSolver(sun_solver);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 2:
-      case 3:
-         arkode = new ARKODESolver(ARKODESolver::EXPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         if (ode_solver_type == 3) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
-         ode_solver = arkode; break;
-      case 12:
-         arkode = new ARKODESolver(ARKODESolver::IMPLICIT);
-         arkode->SetLinearSolver(sun_solver);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         ode_solver = arkode; break;
-      // Other MFEM explicit methods
-      case 4: ode_solver = new ForwardEulerSolver; break;
-      case 5: ode_solver = new RK2Solver(0.5); break; // midpoint method
-      case 6: ode_solver = new RK3SSPSolver; break;
-      case 7: ode_solver = new RK4Solver; break;
-      // MFEM implicit L-stable methods
-      case 8:  ode_solver = new BackwardEulerSolver; break;
-      case 9:  ode_solver = new SDIRK23Solver(2); break;
-      case 10: ode_solver = new SDIRK33Solver; break;
-      default:
-         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
-         delete mesh;
-         return 3;
-   }
-
-   // Since we want to update the diffusion coefficient after every time step,
-   // we need to use the "one-step" mode of the SUNDIALS solvers.
-   if (cvode) { cvode->SetStepMode(CV_ONE_STEP); }
-   if (arkode) { arkode->SetStepMode(ARK_ONE_STEP); }
-
-   // 4. Refine the mesh to increase the resolution. In this example we do
+   // 3. Refine the mesh to increase the resolution. In this example we do
    //    'ref_levels' of uniform refinement, where 'ref_levels' is a
    //    command-line parameter.
    for (int lev = 0; lev < ref_levels; lev++)
@@ -244,7 +191,7 @@ int main(int argc, char *argv[])
       mesh->UniformRefinement();
    }
 
-   // 5. Define the vector finite element space representing the current and the
+   // 4. Define the vector finite element space representing the current and the
    //    initial temperature, u_ref.
    H1_FECollection fe_coll(order, dim);
    FiniteElementSpace fespace(mesh, &fe_coll);
@@ -254,14 +201,14 @@ int main(int argc, char *argv[])
 
    GridFunction u_gf(&fespace);
 
-   // 6. Set the initial conditions for u. All boundaries are considered
+   // 5. Set the initial conditions for u. All boundaries are considered
    //    natural.
    FunctionCoefficient u_0(InitialTemperature);
    u_gf.ProjectCoefficient(u_0);
    Vector u;
    u_gf.GetTrueDofs(u);
 
-   // 7. Initialize the conduction operator and the visualization.
+   // 6. Initialize the conduction operator and the visualization.
    ConductionOperator oper(fespace, alpha, kappa, u);
 
    u_gf.SetFromTrueDofs(u);
@@ -307,13 +254,65 @@ int main(int argc, char *argv[])
       }
    }
 
+   // 7. Define the ODE solver used for time integration.
+   double t = 0.0;
+   ODESolver *ode_solver = NULL;
+   CVODESolver *cvode = NULL;
+   ARKStepSolver *arkode = NULL;
+   switch (ode_solver_type)
+   {
+      // MFEM explicit methods
+      case 1: ode_solver = new ForwardEulerSolver; break;
+      case 2: ode_solver = new RK2Solver(0.5); break; // midpoint method
+      case 3: ode_solver = new RK3SSPSolver; break;
+      case 4: ode_solver = new RK4Solver; break;
+      // MFEM implicit L-stable methods
+      case 5: ode_solver = new BackwardEulerSolver; break;
+      case 6: ode_solver = new SDIRK23Solver(2); break;
+      case 7: ode_solver = new SDIRK33Solver; break;
+      // CVODE
+      case 8:
+         cvode = new CVODESolver(CV_ADAMS);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         cvode->SetMaxStep(dt);
+         ode_solver = cvode; break;
+      case 9:
+         cvode = new CVODESolver(CV_BDF);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         cvode->SetMaxStep(dt);
+         ode_solver = cvode; break;
+      // ARKODE
+      case 10:
+      case 11:
+         arkode = new ARKStepSolver(ARKStepSolver::EXPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         if (ode_solver_type == 11) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
+         ode_solver = arkode; break;
+      case 12:
+         arkode = new ARKStepSolver(ARKStepSolver::IMPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         ode_solver = arkode; break;
+   }
+
+   // Initialize MFEM integrators, SUNDIALS integrators are initialized above
+   if (ode_solver_type < 8) { ode_solver->Init(oper); }
+
+   // Since we want to update the diffusion coefficient after every time step,
+   // we need to use the "one-step" mode of the SUNDIALS solvers.
+   if (cvode) { cvode->SetStepMode(CV_ONE_STEP); }
+   if (arkode) { arkode->SetStepMode(ARK_ONE_STEP); }
+
    // 8. Perform time-integration (looping over the time iterations, ti, with a
    //    time-step dt).
    cout << "Integrating the ODE ..." << endl;
    tic_toc.Clear();
    tic_toc.Start();
-   ode_solver->Init(oper);
-   double t = 0.0;
 
    bool last_step = false;
    for (int ti = 1; !last_step; ti++)
@@ -371,7 +370,7 @@ int main(int argc, char *argv[])
 ConductionOperator::ConductionOperator(FiniteElementSpace &f, double al,
                                        double kap, const Vector &u)
    : TimeDependentOperator(f.GetTrueVSize(), 0.0), fespace(f), M(NULL), K(NULL),
-     T(NULL), current_dt(0.0), z(height)
+     T(NULL), z(height)
 {
    const double rel_tol = 1e-8;
 
@@ -417,32 +416,14 @@ void ConductionOperator::ImplicitSolve(const double dt,
    // Solve the equation:
    //    du_dt = M^{-1}*[-K(u + dt*du_dt)]
    // for du_dt
-   if (!T)
-   {
-      T = Add(1.0, Mmat, dt, Kmat);
-      current_dt = dt;
-      T_solver.SetOperator(*T);
-   }
-   MFEM_VERIFY(dt == current_dt, ""); // SDIRK methods use the same dt
+   if (T) { delete T; }
+   T = Add(1.0, Mmat, dt, Kmat);
+   T_solver.SetOperator(*T);
    Kmat.Mult(u, z);
    z.Neg();
    T_solver.Mult(z, du_dt);
 }
 
-void ConductionOperator::SundialsSolve(const double dt, Vector &b)
-{
-   // Solve the system (M + dt K) y = M b. The result y replaces the input b.
-   if (!T || dt != current_dt)
-   {
-      delete T;
-      T = Add(1.0, Mmat, dt, Kmat);
-      current_dt = dt;
-      T_solver.SetOperator(*T);
-   }
-   Mmat.Mult(b, z);
-   T_solver.Mult(z, b);
-}
-
 void ConductionOperator::SetParameters(const Vector &u)
 {
    GridFunction u_alpha_gf(&fespace);
@@ -460,8 +441,26 @@ void ConductionOperator::SetParameters(const Vector &u)
    K->AddDomainIntegrator(new DiffusionIntegrator(u_coeff));
    K->Assemble();
    K->FormSystemMatrix(ess_tdof_list, Kmat);
-   delete T;
-   T = NULL; // re-compute T on the next ImplicitSolve or SundialsSolve
+}
+
+int ConductionOperator::SUNImplicitSetup(const Vector &x,
+                                         const Vector &fx, int jok, int *jcur,
+                                         double gamma)
+{
+   // Setup the ODE Jacobian T = M + gamma K.
+   if (T) { delete T; }
+   T = Add(1.0, Mmat, gamma, Kmat);
+   T_solver.SetOperator(*T);
+   *jcur = 1;
+   return (0);
+}
+
+int ConductionOperator::SUNImplicitSolve(const Vector &b, Vector &x, double tol)
+{
+   // Solve the system A x = z => (M - gamma K) x = M b.
+   Mmat.Mult(b, z);
+   T_solver.Mult(z, x);
+   return (0);
 }
 
 ConductionOperator::~ConductionOperator()
@@ -471,46 +470,6 @@ ConductionOperator::~ConductionOperator()
    delete K;
 }
 
-
-int SundialsJacSolver::InitSystem(void *sundials_mem)
-{
-   TimeDependentOperator *td_oper = GetTimeDependentOperator(sundials_mem);
-
-   // During development, we use dynamic_cast<> to ensure the setup is correct:
-   oper = dynamic_cast<ConductionOperator*>(td_oper);
-   MFEM_VERIFY(oper, "operator is not ConductionOperator");
-
-   // When the implementation is finalized, we can switch to static_cast<>:
-   // oper = static_cast<ConductionOperator*>(td_oper);
-
-   return 0;
-}
-
-int SundialsJacSolver::SetupSystem(void *sundials_mem, int conv_fail,
-                                   const Vector &y_pred, const Vector &f_pred,
-                                   int &jac_cur, Vector &v_temp1,
-                                   Vector &v_temp2, Vector &v_temp3)
-{
-   jac_cur = 1;
-
-   return 0;
-}
-
-int SundialsJacSolver::SolveSystem(void *sundials_mem, Vector &b,
-                                   const Vector &weight, const Vector &y_cur,
-                                   const Vector &f_cur)
-{
-   oper->SundialsSolve(GetTimeStep(sundials_mem), b);
-
-   return 0;
-}
-
-int SundialsJacSolver::FreeSystem(void *sundials_mem)
-{
-   return 0;
-}
-
-
 double InitialTemperature(const Vector &x)
 {
    if (x.Norml2() < 0.5)

examples/sundials/ex16p.cpp

@@ -8,9 +8,9 @@
 //     mpirun -np 4 ex16p -m ../../data/inline-tri.mesh
 //     mpirun -np 4 ex16p -m ../../data/disc-nurbs.mesh -tf 2
 //     mpirun -np 4 ex16p -s 12 -a 0.0 -k 1.0
-//     mpirun -np 4 ex16p -s 1 -a 1.0 -k 0.0 -dt 4e-6 -tf 2e-2 -vs 50
-//     mpirun -np 8 ex16p -s 2 -a 0.5 -k 0.5 -o 4 -dt 8e-6 -tf 2e-2 -vs 50
-//     mpirun -np 4 ex16p -s 3 -dt 2.0e-4 -tf 4.0e-2
+//     mpirun -np 4 ex16p -s 8 -a 1.0 -k 0.0 -dt 4e-6 -tf 2e-2 -vs 50
+//     mpirun -np 8 ex16p -s 9 -a 0.5 -k 0.5 -o 4 -dt 8e-6 -tf 2e-2 -vs 50
+//     mpirun -np 4 ex16p -s 10 -dt 2.0e-4 -tf 4.0e-2
 //     mpirun -np 16 ex16p -m ../../data/fichera-q2.mesh
 //     mpirun -np 16 ex16p -m ../../data/escher-p2.mesh
 //     mpirun -np 8 ex16p -m ../../data/beam-tet.mesh -tf 10 -dt 0.1
@@ -77,13 +77,19 @@ public:
                       const Vector &u);
 
    virtual void Mult(const Vector &u, Vector &du_dt) const;
+
    /** Solve the Backward-Euler equation: k = f(u + dt*k, t), for the unknown k.
        This is the only requirement for high-order SDIRK implicit integration.*/
    virtual void ImplicitSolve(const double dt, const Vector &u, Vector &k);
 
-   /** Solve the system (M + dt K) y = M b. The result y replaces the input b.
-       This method is used by the implicit SUNDIALS solvers. */
-   void SundialsSolve(const double dt, Vector &b);
+   /** Setup the system (M + dt K) x = M b. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSetup(const Vector &x, const Vector &fx,
+                                int jok, int *jcur, double gamma);
+
+   /** Solve the system (M + dt K) x = M b. This method is used by the implicit
+       SUNDIALS solvers. */
+   virtual int SUNImplicitSolve(const Vector &b, Vector &x, double tol);
 
    /// Update the diffusion BilinearForm K using the given true-dof vector `u`.
    void SetParameters(const Vector &u);
@@ -91,33 +97,6 @@ public:
    virtual ~ConductionOperator();
 };
 
-/// Custom Jacobian system solver for the SUNDIALS time integrators.
-/** For the ODE system represented by ConductionOperator
-
-        M du/dt = -K(u),
-
-    this class facilitates the solution of linear systems of the form
-
-        (M + γK) y = M b,
-
-    for given b, u (not used), and γ = GetTimeStep(). */
-class SundialsJacSolver : public SundialsODELinearSolver
-{
-private:
-   ConductionOperator *oper;
-
-public:
-   SundialsJacSolver() : oper(NULL) { }
-
-   int InitSystem(void *sundials_mem);
-   int SetupSystem(void *sundials_mem, int conv_fail,
-                   const Vector &y_pred, const Vector &f_pred, int &jac_cur,
-                   Vector &v_temp1, Vector &v_temp2, Vector &v_temp3);
-   int SolveSystem(void *sundials_mem, Vector &b, const Vector &weight,
-                   const Vector &y_cur, const Vector &f_cur);
-   int FreeSystem(void *sundials_mem);
-};
-
 double InitialTemperature(const Vector &x);
 
 int main(int argc, char *argv[])
@@ -133,7 +112,7 @@ int main(int argc, char *argv[])
    int ser_ref_levels = 2;
    int par_ref_levels = 1;
    int order = 2;
-   int ode_solver_type = 11; // 11 = CVODE implicit
+   int ode_solver_type = 9; // CVODE implicit BDF
    double t_final = 0.5;
    double dt = 1.0e-2;
    double alpha = 1.0e-2;
@@ -158,12 +137,19 @@ int main(int argc, char *argv[])
    args.AddOption(&order, "-o", "--order",
                   "Order (degree) of the finite elements.");
    args.AddOption(&ode_solver_type, "-s", "--ode-solver",
-                  "ODE solver:\n"
-                  "\t 1/11 - CVODE (explicit/implicit),\n"
-                  "\t 2/12 - ARKODE (default explicit/implicit),\n"
-                  "\t 3 - ARKODE (Fehlberg-6-4-5)\n"
-                  "\t 4 - Forward Euler, 5 - RK2, 6 - RK3 SSP, 7 - RK4,\n"
-                  "\t 8 - Backward Euler, 9 - SDIRK23, 10 - SDIRK33.");
+                  "ODE solver:\n\t"
+                  "1  - Forward Euler,\n\t"
+                  "2  - RK2,\n\t"
+                  "3  - RK3 SSP,\n\t"
+                  "4  - RK4,\n\t"
+                  "5  - Backward Euler,\n\t"
+                  "6  - SDIRK 2,\n\t"
+                  "7  - SDIRK 3,\n\t"
+                  "8  - CVODE (implicit Adams),\n\t"
+                  "9  - CVODE (implicit BDF),\n\t"
+                  "10 - ARKODE (default explicit),\n\t"
+                  "11 - ARKODE (explicit Fehlberg-6-4-5),\n\t"
+                  "12 - ARKODE (default impicit).");
    args.AddOption(&t_final, "-tf", "--t-final",
                   "Final time; start time is 0.");
    args.AddOption(&dt, "-dt", "--time-step",
@@ -193,67 +179,24 @@ int main(int argc, char *argv[])
       args.PrintOptions(cout);
    }
 
+   // check for vaild ODE solver option
+   if (ode_solver_type < 1 || ode_solver_type > 12)
+   {
+      if (myid == 0)
+      {
+         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
+      }
+      MPI_Finalize();
+      return 1;
+   }
+
    // 3. Read the serial mesh from the given mesh file on all processors. We can
    //    handle triangular, quadrilateral, tetrahedral and hexahedral meshes
    //    with the same code.
    Mesh *mesh = new Mesh(mesh_file, 1, 1);
    int dim = mesh->Dimension();
 
-   // 4. Define the ODE solver used for time integration. Several
-   // SUNDIALS solvers are available, as well as included both
-   // explicit and implicit MFEM ODE solvers.
-   ODESolver *ode_solver = NULL;
-   CVODESolver *cvode = NULL;
-   ARKODESolver *arkode = NULL;
-   SundialsJacSolver sun_solver; // Used by the implicit SUNDIALS ode solvers.
-   switch (ode_solver_type)
-   {
-      // SUNDIALS solvers
-      case 1:
-         cvode = new CVODESolver(MPI_COMM_WORLD, CV_ADAMS, CV_FUNCTIONAL);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 11:
-         cvode = new CVODESolver(MPI_COMM_WORLD, CV_BDF, CV_NEWTON);
-         cvode->SetLinearSolver(sun_solver);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 2:
-      case 3:
-         arkode = new ARKODESolver(MPI_COMM_WORLD, ARKODESolver::EXPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         if (ode_solver_type == 3) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
-         ode_solver = arkode; break;
-      case 12:
-         arkode = new ARKODESolver(MPI_COMM_WORLD, ARKODESolver::IMPLICIT);
-         arkode->SetLinearSolver(sun_solver);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         ode_solver = arkode; break;
-      // Other MFEM explicit methods
-      case 4: ode_solver = new ForwardEulerSolver; break;
-      case 5: ode_solver = new RK2Solver(0.5); break; // midpoint method
-      case 6: ode_solver = new RK3SSPSolver; break;
-      case 7: ode_solver = new RK4Solver; break;
-      // MFEM implicit L-stable methods
-      case 8:  ode_solver = new BackwardEulerSolver; break;
-      case 9:  ode_solver = new SDIRK23Solver(2); break;
-      case 10: ode_solver = new SDIRK33Solver; break;
-      default:
-         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
-         delete mesh;
-         return 3;
-   }
-
-   // Since we want to update the diffusion coefficient after every time step,
-   // we need to use the "one-step" mode of the SUNDIALS solvers.
-   if (cvode) { cvode->SetStepMode(CV_ONE_STEP); }
-   if (arkode) { arkode->SetStepMode(ARK_ONE_STEP); }
-
-   // 5. Refine the mesh in serial to increase the resolution. In this example
+   // 4. Refine the mesh in serial to increase the resolution. In this example
    //    we do 'ser_ref_levels' of uniform refinement, where 'ser_ref_levels' is
    //    a command-line parameter.
    for (int lev = 0; lev < ser_ref_levels; lev++)
@@ -261,7 +204,7 @@ int main(int argc, char *argv[])
       mesh->UniformRefinement();
    }
 
-   // 6. Define a parallel mesh by a partitioning of the serial mesh. Refine
+   // 5. Define a parallel mesh by a partitioning of the serial mesh. Refine
    //    this mesh further in parallel to increase the resolution. Once the
    //    parallel mesh is defined, the serial mesh can be deleted.
    ParMesh *pmesh = new ParMesh(MPI_COMM_WORLD, *mesh);
@@ -271,7 +214,7 @@ int main(int argc, char *argv[])
       pmesh->UniformRefinement();
    }
 
-   // 7. Define the vector finite element space representing the current and the
+   // 6. Define the vector finite element space representing the current and the
    //    initial temperature, u_ref.
    H1_FECollection fe_coll(order, dim);
    ParFiniteElementSpace fespace(pmesh, &fe_coll);
@@ -284,14 +227,14 @@ int main(int argc, char *argv[])
 
    ParGridFunction u_gf(&fespace);
 
-   // 8. Set the initial conditions for u. All boundaries are considered
+   // 7. Set the initial conditions for u. All boundaries are considered
    //    natural.
    FunctionCoefficient u_0(InitialTemperature);
    u_gf.ProjectCoefficient(u_0);
    Vector u;
    u_gf.GetTrueDofs(u);
 
-   // 9. Initialize the conduction operator and the VisIt visualization.
+   // 8. Initialize the conduction operator and the VisIt visualization.
    ConductionOperator oper(fespace, alpha, kappa, u);
 
    u_gf.SetFromTrueDofs(u);
@@ -350,6 +293,60 @@ int main(int argc, char *argv[])
       }
    }
 
+   // 9. Define the ODE solver used for time integration.
+   double t = 0.0;
+   ODESolver *ode_solver = NULL;
+   CVODESolver *cvode = NULL;
+   ARKStepSolver *arkode = NULL;
+   switch (ode_solver_type)
+   {
+      // MFEM explicit methods
+      case 1: ode_solver = new ForwardEulerSolver; break;
+      case 2: ode_solver = new RK2Solver(0.5); break; // midpoint method
+      case 3: ode_solver = new RK3SSPSolver; break;
+      case 4: ode_solver = new RK4Solver; break;
+      // MFEM implicit L-stable methods
+      case 5: ode_solver = new BackwardEulerSolver; break;
+      case 6: ode_solver = new SDIRK23Solver(2); break;
+      case 7: ode_solver = new SDIRK33Solver; break;
+      // CVODE
+      case 8:
+         cvode = new CVODESolver(MPI_COMM_WORLD, CV_ADAMS);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         cvode->SetMaxStep(dt);
+         ode_solver = cvode; break;
+      case 9:
+         cvode = new CVODESolver(MPI_COMM_WORLD, CV_BDF);
+         cvode->Init(oper);
+         cvode->SetSStolerances(reltol, abstol);
+         cvode->SetMaxStep(dt);
+         ode_solver = cvode; break;
+      // ARKODE
+      case 10:
+      case 11:
+         arkode = new ARKStepSolver(MPI_COMM_WORLD, ARKStepSolver::EXPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         if (ode_solver_type == 11) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
+         ode_solver = arkode; break;
+      case 12:
+         arkode = new ARKStepSolver(MPI_COMM_WORLD, ARKStepSolver::IMPLICIT);
+         arkode->Init(oper);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         ode_solver = arkode; break;
+   }
+
+   // Initialize MFEM integrators, SUNDIALS integrators are initialized above
+   if (ode_solver_type < 8) { ode_solver->Init(oper); }
+
+   // Since we want to update the diffusion coefficient after every time step,
+   // we need to use the "one-step" mode of the SUNDIALS solvers.
+   if (cvode) { cvode->SetStepMode(CV_ONE_STEP); }
+   if (arkode) { arkode->SetStepMode(ARK_ONE_STEP); }
+
    // 10. Perform time-integration (looping over the time iterations, ti, with a
    //     time-step dt).
    if (myid == 0)
@@ -358,8 +355,6 @@ int main(int argc, char *argv[])
    }
    tic_toc.Clear();
    tic_toc.Start();
-   ode_solver->Init(oper);
-   double t = 0.0;
 
    bool last_step = false;
    for (int ti = 1; !last_step; ti++)
@@ -428,7 +423,7 @@ int main(int argc, char *argv[])
 ConductionOperator::ConductionOperator(ParFiniteElementSpace &f, double al,
                                        double kap, const Vector &u)
    : TimeDependentOperator(f.GetTrueVSize(), 0.0), fespace(f), M(NULL), K(NULL),
-     T(NULL), current_dt(0.0),
+     T(NULL),
      M_solver(f.GetComm()), T_solver(f.GetComm()), z(height)
 {
    const double rel_tol = 1e-8;
@@ -476,30 +471,32 @@ void ConductionOperator::ImplicitSolve(const double dt,
    // Solve the equation:
    //    du_dt = M^{-1}*[-K(u + dt*du_dt)]
    // for du_dt
-   if (!T)
-   {
-      T = Add(1.0, Mmat, dt, Kmat);
-      current_dt = dt;
-      T_solver.SetOperator(*T);
-   }
-   MFEM_VERIFY(dt == current_dt, ""); // SDIRK methods use the same dt
+   if (T) { delete T; }
+   T = Add(1.0, Mmat, dt, Kmat);
+   T_solver.SetOperator(*T);
    Kmat.Mult(u, z);
    z.Neg();
    T_solver.Mult(z, du_dt);
 }
 
-void ConductionOperator::SundialsSolve(const double dt, Vector &b)
+int ConductionOperator::SUNImplicitSetup(const Vector &x,
+                                         const Vector &fx, int jok, int *jcur,
+                                         double gamma)
 {
-   // Solve the system (M + dt K) y = M b. The result y replaces the input b.
-   if (!T || dt != current_dt)
-   {
-      delete T;
-      T = Add(1.0, Mmat, dt, Kmat);
-      current_dt = dt;
-      T_solver.SetOperator(*T);
-   }
+   // Setup the ODE Jacobian T = M + gamma K.
+   if (T) { delete T; }
+   T = Add(1.0, Mmat, gamma, Kmat);
+   T_solver.SetOperator(*T);
+   *jcur = 1;
+   return (0);
+}
+
+int ConductionOperator::SUNImplicitSolve(const Vector &b, Vector &x, double tol)
+{
+   // Solve the system A x = z => (M - gamma K) x = M b.
    Mmat.Mult(b, z);
-   T_solver.Mult(z, b);
+   T_solver.Mult(z, x);
+   return (0);
 }
 
 void ConductionOperator::SetParameters(const Vector &u)
@@ -519,8 +516,6 @@ void ConductionOperator::SetParameters(const Vector &u)
    K->AddDomainIntegrator(new DiffusionIntegrator(u_coeff));
    K->Assemble(0); // keep sparsity pattern of M and K the same
    K->FormSystemMatrix(ess_tdof_list, Kmat);
-   delete T;
-   T = NULL; // re-compute T on the next ImplicitSolve or SundialsSolve
 }
 
 ConductionOperator::~ConductionOperator()
@@ -530,46 +525,6 @@ ConductionOperator::~ConductionOperator()
    delete K;
 }
 
-
-int SundialsJacSolver::InitSystem(void *sundials_mem)
-{
-   TimeDependentOperator *td_oper = GetTimeDependentOperator(sundials_mem);
-
-   // During development, we use dynamic_cast<> to ensure the setup is correct:
-   oper = dynamic_cast<ConductionOperator*>(td_oper);
-   MFEM_VERIFY(oper, "operator is not ConductionOperator");
-
-   // When the implementation is finalized, we can switch to static_cast<>:
-   // oper = static_cast<ConductionOperator*>(td_oper);
-
-   return 0;
-}
-
-int SundialsJacSolver::SetupSystem(void *sundials_mem, int conv_fail,
-                                   const Vector &y_pred, const Vector &f_pred,
-                                   int &jac_cur, Vector &v_temp1,
-                                   Vector &v_temp2, Vector &v_temp3)
-{
-   jac_cur = 1;
-
-   return 0;
-}
-
-int SundialsJacSolver::SolveSystem(void *sundials_mem, Vector &b,
-                                   const Vector &weight, const Vector &y_cur,
-                                   const Vector &f_cur)
-{
-   oper->SundialsSolve(GetTimeStep(sundials_mem), b);
-
-   return 0;
-}
-
-int SundialsJacSolver::FreeSystem(void *sundials_mem)
-{
-   return 0;
-}
-
-
 double InitialTemperature(const Vector &x)
 {
    if (x.Norml2() < 0.5)

examples/sundials/ex9.cpp

@@ -4,14 +4,14 @@
 // Compile with: make ex9
 //
 // Sample runs:
-//    ex9 -m ../../data/periodic-segment.mesh -p 0 -r 2 -s 11 -dt 0.005
-//    ex9 -m ../../data/periodic-square.mesh  -p 1 -r 2 -s 12 -dt 0.005 -tf 9
-//    ex9 -m ../../data/periodic-hexagon.mesh -p 0 -r 2 -s 11 -dt 0.0018 -vs 25
-//    ex9 -m ../../data/periodic-hexagon.mesh -p 0 -r 2 -s 13 -dt 0.01 -vs 15
-//    ex9 -m ../../data/amr-quad.mesh         -p 1 -r 2 -s 13 -dt 0.002 -tf 9
-//    ex9 -m ../../data/star-q3.mesh          -p 1 -r 2 -s 13 -dt 0.005 -tf 9
-//    ex9 -m ../../data/disc-nurbs.mesh       -p 1 -r 3 -s 11 -dt 0.005 -tf 9
-//    ex9 -m ../../data/periodic-cube.mesh    -p 0 -r 2 -s 12 -dt 0.02 -tf 8 -o 2
+//    ex9 -m ../../data/periodic-segment.mesh -p 0 -r 2 -s 7 -dt 0.005
+//    ex9 -m ../../data/periodic-square.mesh  -p 1 -r 2 -s 8 -dt 0.005 -tf 9
+//    ex9 -m ../../data/periodic-hexagon.mesh -p 0 -r 2 -s 7 -dt 0.0018 -vs 25
+//    ex9 -m ../../data/periodic-hexagon.mesh -p 0 -r 2 -s 9 -dt 0.01 -vs 15
+//    ex9 -m ../../data/amr-quad.mesh         -p 1 -r 2 -s 9 -dt 0.002 -tf 9
+//    ex9 -m ../../data/star-q3.mesh          -p 1 -r 2 -s 9 -dt 0.005 -tf 9
+//    ex9 -m ../../data/disc-nurbs.mesh       -p 1 -r 3 -s 7 -dt 0.005 -tf 9
+//    ex9 -m ../../data/periodic-cube.mesh    -p 0 -r 2 -s 8 -dt 0.02 -tf 8 -o 2
 //
 // Description:  This example code solves the time-dependent advection equation
 //               du/dt + v.grad(u) = 0, where v is a given fluid velocity, and
@@ -109,11 +109,15 @@ int main(int argc, char *argv[])
    args.AddOption(&order, "-o", "--order",
                   "Order (degree) of the finite elements.");
    args.AddOption(&ode_solver_type, "-s", "--ode-solver",
-                  "ODE solver: 1 - Forward Euler,\n\t"
-                  "            2 - RK2 SSP, 3 - RK3 SSP, 4 - RK4, 6 - RK6,\n\t"
-                  "            11 - CVODE (adaptive order) explicit,\n\t"
-                  "            12 - ARKODE default (4th order) explicit,\n\t"
-                  "            13 - ARKODE RK8.");
+                  "ODE solver:\n\t"
+                  "1 - Forward Euler,\n\t"
+                  "2 - RK2 SSP,\n\t"
+                  "3 - RK3 SSP,\n\t"
+                  "4 - RK4,\n\t"
+                  "6 - RK6,\n\t"
+                  "7 - CVODE (adaptive order implicit Adams),\n\t"
+                  "8 - ARKODE default (4th order) explicit,\n\t"
+                  "9 - ARKODE RK8.");
    args.AddOption(&t_final, "-tf", "--t-final",
                   "Final time; start time is 0.");
    args.AddOption(&dt, "-dt", "--time-step",
@@ -135,65 +139,41 @@ int main(int argc, char *argv[])
       args.PrintUsage(cout);
       return 1;
    }
+   // check for vaild ODE solver option
+   if (ode_solver_type < 1 || ode_solver_type > 9)
+   {
+      cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
+      return 3;
+   }
    args.PrintOptions(cout);
 
    // 2. Read the mesh from the given mesh file. We can handle geometrically
    //    periodic meshes in this code.
-   Mesh *mesh = new Mesh(mesh_file, 1, 1);
-   int dim = mesh->Dimension();
+   Mesh mesh(mesh_file, 1, 1);
+   int dim = mesh.Dimension();
 
-   // 3. Define the ODE solver used for time integration. Several explicit
-   //    Runge-Kutta methods are available.
-   ODESolver *ode_solver = NULL;
-   CVODESolver *cvode = NULL;
-   ARKODESolver *arkode = NULL;
-   switch (ode_solver_type)
-   {
-      case 1: ode_solver = new ForwardEulerSolver; break;
-      case 2: ode_solver = new RK2Solver(1.0); break;
-      case 3: ode_solver = new RK3SSPSolver; break;
-      case 4: ode_solver = new RK4Solver; break;
-      case 6: ode_solver = new RK6Solver; break;
-      case 11:
-         cvode = new CVODESolver(CV_ADAMS, CV_FUNCTIONAL);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 12:
-      case 13:
-         arkode = new ARKODESolver(ARKODESolver::EXPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         if (ode_solver_type == 13) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
-         ode_solver = arkode; break;
-      default:
-         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
-         delete mesh;
-         return 3;
-   }
-
-   // 4. Refine the mesh to increase the resolution. In this example we do
+   // 3. Refine the mesh to increase the resolution. In this example we do
    //    'ref_levels' of uniform refinement, where 'ref_levels' is a
    //    command-line parameter. If the mesh is of NURBS type, we convert it to
    //    a (piecewise-polynomial) high-order mesh.
    for (int lev = 0; lev < ref_levels; lev++)
    {
-      mesh->UniformRefinement();
+      mesh.UniformRefinement();
    }
-   if (mesh->NURBSext)
+   if (mesh.NURBSext)
    {
-      mesh->SetCurvature(max(order, 1));
+      mesh.SetCurvature(max(order, 1));
    }
-   mesh->GetBoundingBox(bb_min, bb_max, max(order, 1));
+   mesh.GetBoundingBox(bb_min, bb_max, max(order, 1));
 
-   // 5. Define the discontinuous DG finite element space of the given
+   // 4. Define the discontinuous DG finite element space of the given
    //    polynomial order on the refined mesh.
    DG_FECollection fec(order, dim);
-   FiniteElementSpace fes(mesh, &fec);
+   FiniteElementSpace fes(&mesh, &fec);
 
    cout << "Number of unknowns: " << fes.GetVSize() << endl;
 
-   // 6. Set up and assemble the bilinear and linear forms corresponding to the
+   // 5. Set up and assemble the bilinear and linear forms corresponding to the
    //    DG discretization. The DGTraceIntegrator involves integrals over mesh
    //    interior faces.
    VectorFunctionCoefficient velocity(dim, velocity_function);
@@ -220,7 +200,7 @@ int main(int argc, char *argv[])
    k.Finalize(skip_zeros);
    b.Assemble();
 
-   // 7. Define the initial conditions, save the corresponding grid function to
+   // 6. Define the initial conditions, save the corresponding grid function to
    //    a file and (optionally) save data in the VisIt format and initialize
    //    GLVis visualization.
    GridFunction u(&fes);
@@ -229,7 +209,7 @@ int main(int argc, char *argv[])
    {
       ofstream omesh("ex9.mesh");
       omesh.precision(precision);
-      mesh->Print(omesh);
+      mesh.Print(omesh);
       ofstream osol("ex9-init.gf");
       osol.precision(precision);
       u.Save(osol);
@@ -243,14 +223,14 @@ int main(int argc, char *argv[])
       if (binary)
       {
 #ifdef MFEM_USE_SIDRE
-         dc = new SidreDataCollection("Example9", mesh);
+         dc = new SidreDataCollection("Example9", &mesh);
 #else
          MFEM_ABORT("Must build with MFEM_USE_SIDRE=YES for binary output.");
 #endif
       }
       else
       {
-         dc = new VisItDataCollection("Example9", mesh);
+         dc = new VisItDataCollection("Example9", &mesh);
          dc->SetPrecision(precision);
       }
       dc->RegisterField("solution", &u);
@@ -275,7 +255,7 @@ int main(int argc, char *argv[])
       else
       {
          sout.precision(precision);
-         sout << "solution\n" << *mesh << u;
+         sout << "solution\n" << mesh << u;
          sout << "pause\n";
          sout << flush;
          cout << "GLVis visualization paused."
@@ -283,15 +263,46 @@ int main(int argc, char *argv[])
       }
    }
 
-   // 8. Define the time-dependent evolution operator describing the ODE
-   //    right-hand side, and perform time-integration (looping over the time
-   //    iterations, ti, with a time-step dt).
+   // 7. Define the time-dependent evolution operator describing the ODE
+   //    right-hand side, and define the ODE solver used for time integration.
    FE_Evolution adv(m.SpMat(), k.SpMat(), b);
 
    double t = 0.0;
    adv.SetTime(t);
-   ode_solver->Init(adv);
 
+   // Create the time integrator
+   ODESolver *ode_solver = NULL;
+   CVODESolver *cvode = NULL;
+   ARKStepSolver *arkode = NULL;
+   switch (ode_solver_type)
+   {
+      case 1: ode_solver = new ForwardEulerSolver; break;
+      case 2: ode_solver = new RK2Solver(1.0); break;
+      case 3: ode_solver = new RK3SSPSolver; break;
+      case 4: ode_solver = new RK4Solver; break;
+      case 6: ode_solver = new RK6Solver; break;
+      case 7:
+         cvode = new CVODESolver(CV_ADAMS);
+         cvode->Init(adv);
+         cvode->SetSStolerances(reltol, abstol);
+         cvode->SetMaxStep(dt);
+         cvode->UseSundialsLinearSolver();
+         ode_solver = cvode; break;
+      case 8:
+      case 9:
+         arkode = new ARKStepSolver(ARKStepSolver::EXPLICIT);
+         arkode->Init(adv);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         if (ode_solver_type == 9) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
+         ode_solver = arkode; break;
+   }
+
+   // Initialize MFEM integrators, SUNDIALS integrators are initialized above
+   if (ode_solver_type < 7) { ode_solver->Init(adv); }
+
+   // 8. Perform time-integration (looping over the time iterations, ti,
+   //    with a time-step dt).
    bool done = false;
    for (int ti = 0; !done; )
    {
@@ -309,7 +320,7 @@ int main(int argc, char *argv[])
 
          if (visualization)
          {
-            sout << "solution\n" << *mesh << u << flush;
+            sout << "solution\n" << mesh << u << flush;
          }
 
          if (visit)

examples/sundials/ex9p.cpp

@@ -4,14 +4,14 @@
 // Compile with: make ex9p
 //
 // Sample runs:
-//    mpirun -np 4 ex9p -m ../../data/periodic-segment.mesh -p 1 -rp 1 -s 11 -dt 0.0025
-//    mpirun -np 4 ex9p -m ../../data/periodic-square.mesh  -p 1 -rp 1 -s 12 -dt 0.0025 -tf 9
-//    mpirun -np 4 ex9p -m ../../data/periodic-hexagon.mesh -p 0 -rp 1 -s 11 -dt 0.0009 -vs 25
-//    mpirun -np 4 ex9p -m ../../data/periodic-hexagon.mesh -p 0 -rp 1 -s 13 -dt 0.005 -vs 15
-//    mpirun -np 4 ex9p -m ../../data/amr-quad.mesh         -p 1 -rp 1 -s 13 -dt 0.001 -tf 9
-//    mpirun -np 4 ex9p -m ../../data/star-q3.mesh          -p 1 -rp 1 -s 13 -dt 0.0025 -tf 9
-//    mpirun -np 4 ex9p -m ../../data/disc-nurbs.mesh       -p 1 -rp 2 -s 11 -dt 0.0025 -tf 9
-//    mpirun -np 4 ex9p -m ../../data/periodic-cube.mesh    -p 0 -rp 1 -s 12 -dt 0.01 -tf 8 -o 2
+//    mpirun -np 4 ex9p -m ../../data/periodic-segment.mesh -p 1 -rp 1 -s 7 -dt 0.0025
+//    mpirun -np 4 ex9p -m ../../data/periodic-square.mesh  -p 1 -rp 1 -s 8 -dt 0.0025 -tf 9
+//    mpirun -np 4 ex9p -m ../../data/periodic-hexagon.mesh -p 0 -rp 1 -s 7 -dt 0.0009 -vs 25
+//    mpirun -np 4 ex9p -m ../../data/periodic-hexagon.mesh -p 0 -rp 1 -s 9 -dt 0.005 -vs 15
+//    mpirun -np 4 ex9p -m ../../data/amr-quad.mesh         -p 1 -rp 1 -s 9 -dt 0.001 -tf 9
+//    mpirun -np 4 ex9p -m ../../data/star-q3.mesh          -p 1 -rp 1 -s 9 -dt 0.0025 -tf 9
+//    mpirun -np 4 ex9p -m ../../data/disc-nurbs.mesh       -p 1 -rp 2 -s 7 -dt 0.0025 -tf 9
+//    mpirun -np 4 ex9p -m ../../data/periodic-cube.mesh    -p 0 -rp 1 -s 8 -dt 0.01 -tf 8 -o 2
 //
 // Description:  This example code solves the time-dependent advection equation
 //               du/dt + v.grad(u) = 0, where v is a given fluid velocity, and
@@ -117,11 +117,15 @@ int main(int argc, char *argv[])
    args.AddOption(&order, "-o", "--order",
                   "Order (degree) of the finite elements.");
    args.AddOption(&ode_solver_type, "-s", "--ode-solver",
-                  "ODE solver: 1 - Forward Euler,\n\t"
-                  "            2 - RK2 SSP, 3 - RK3 SSP, 4 - RK4, 6 - RK6,\n\t"
-                  "            11 - CVODE (adaptive order) explicit,\n\t"
-                  "            12 - ARKODE default (4th order) explicit,\n\t"
-                  "            13 - ARKODE RK8.");
+                  "ODE solver:\n\t"
+                  "1 - Forward Euler,\n\t"
+                  "2 - RK2 SSP,\n\t"
+                  "3 - RK3 SSP,\n\t"
+                  "4 - RK4,\n\t"
+                  "6 - RK6,\n\t"
+                  "7 - CVODE (adaptive order implicit Adams),\n\t"
+                  "8 - ARKODE default (4th order) explicit,\n\t"
+                  "9 - ARKODE RK8.");
    args.AddOption(&t_final, "-tf", "--t-final",
                   "Final time; start time is 0.");
    args.AddOption(&dt, "-dt", "--time-step",
@@ -151,47 +155,23 @@ int main(int argc, char *argv[])
    {
       args.PrintOptions(cout);
    }
+   // check for vaild ODE solver option
+   if (ode_solver_type < 1 || ode_solver_type > 9)
+   {
+      if (myid == 0)
+      {
+         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
+      }
+      MPI_Finalize();
+      return 3;
+   }
 
    // 3. Read the serial mesh from the given mesh file on all processors. We can
    //    handle geometrically periodic meshes in this code.
    Mesh *mesh = new Mesh(mesh_file, 1, 1);
    int dim = mesh->Dimension();
 
-   // 4. Define the ODE solver used for time integration. Several explicit
-   //    Runge-Kutta methods are available.
-   ODESolver *ode_solver = NULL;
-   CVODESolver *cvode = NULL;
-   ARKODESolver *arkode = NULL;
-   switch (ode_solver_type)
-   {
-      case 1: ode_solver = new ForwardEulerSolver; break;
-      case 2: ode_solver = new RK2Solver(1.0); break;
-      case 3: ode_solver = new RK3SSPSolver; break;
-      case 4: ode_solver = new RK4Solver; break;
-      case 6: ode_solver = new RK6Solver; break;
-      case 11:
-         cvode = new CVODESolver(MPI_COMM_WORLD, CV_ADAMS, CV_FUNCTIONAL);
-         cvode->SetSStolerances(reltol, abstol);
-         cvode->SetMaxStep(dt);
-         ode_solver = cvode; break;
-      case 12:
-      case 13:
-         arkode = new ARKODESolver(MPI_COMM_WORLD, ARKODESolver::EXPLICIT);
-         arkode->SetSStolerances(reltol, abstol);
-         arkode->SetMaxStep(dt);
-         if (ode_solver_type == 13) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
-         ode_solver = arkode; break;
-      default:
-         if (myid == 0)
-         {
-            cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
-         }
-         delete mesh;
-         MPI_Finalize();
-         return 3;
-   }
-
-   // 5. Refine the mesh in serial to increase the resolution. In this example
+   // 4. Refine the mesh in serial to increase the resolution. In this example
    //    we do 'ser_ref_levels' of uniform refinement, where 'ser_ref_levels' is
    //    a command-line parameter. If the mesh is of NURBS type, we convert it
    //    to a (piecewise-polynomial) high-order mesh.
@@ -205,7 +185,7 @@ int main(int argc, char *argv[])
    }
    mesh->GetBoundingBox(bb_min, bb_max, max(order, 1));
 
-   // 6. Define the parallel mesh by a partitioning of the serial mesh. Refine
+   // 5. Define the parallel mesh by a partitioning of the serial mesh. Refine
    //    this mesh further in parallel to increase the resolution. Once the
    //    parallel mesh is defined, the serial mesh can be deleted.
    ParMesh *pmesh = new ParMesh(MPI_COMM_WORLD, *mesh);
@@ -215,7 +195,7 @@ int main(int argc, char *argv[])
       pmesh->UniformRefinement();
    }
 
-   // 7. Define the parallel discontinuous DG finite element space on the
+   // 6. Define the parallel discontinuous DG finite element space on the
    //    parallel refined mesh of the given polynomial order.
    DG_FECollection fec(order, dim);
    ParFiniteElementSpace *fes = new ParFiniteElementSpace(pmesh, &fec);
@@ -226,7 +206,7 @@ int main(int argc, char *argv[])
       cout << "Number of unknowns: " << global_vSize << endl;
    }
 
-   // 8. Set up and assemble the parallel bilinear and linear forms (and the
+   // 7. Set up and assemble the parallel bilinear and linear forms (and the
    //    parallel hypre matrices) corresponding to the DG discretization. The
    //    DGTraceIntegrator involves integrals over mesh interior faces.
    VectorFunctionCoefficient velocity(dim, velocity_function);
@@ -257,7 +237,7 @@ int main(int argc, char *argv[])
    HypreParMatrix *K = k->ParallelAssemble();
    HypreParVector *B = b->ParallelAssemble();
 
-   // 9. Define the initial conditions, save the corresponding grid function to
+   // 8. Define the initial conditions, save the corresponding grid function to
    //    a file and (optionally) save data in the VisIt format and initialize
    //    GLVis visualization.
    ParGridFunction *u = new ParGridFunction(fes);
@@ -330,15 +310,46 @@ int main(int argc, char *argv[])
       }
    }
 
-   // 10. Define the time-dependent evolution operator describing the ODE
-   //     right-hand side, and perform time-integration (looping over the time
-   //     iterations, ti, with a time-step dt).
+   // 9. Define the time-dependent evolution operator describing the ODE
+   //    right-hand side, and define the ODE solver used for time integration.
    FE_Evolution adv(*M, *K, *B);
 
    double t = 0.0;
    adv.SetTime(t);
-   ode_solver->Init(adv);
 
+   // Create the time integrator
+   ODESolver *ode_solver = NULL;
+   CVODESolver *cvode = NULL;
+   ARKStepSolver *arkode = NULL;
+   switch (ode_solver_type)
+   {
+      case 1: ode_solver = new ForwardEulerSolver; break;
+      case 2: ode_solver = new RK2Solver(1.0); break;
+      case 3: ode_solver = new RK3SSPSolver; break;
+      case 4: ode_solver = new RK4Solver; break;
+      case 6: ode_solver = new RK6Solver; break;
+      case 7:
+         cvode = new CVODESolver(MPI_COMM_WORLD, CV_ADAMS);
+         cvode->Init(adv);
+         cvode->SetSStolerances(reltol, abstol);
+         cvode->SetMaxStep(dt);
+         cvode->UseSundialsLinearSolver();
+         ode_solver = cvode; break;
+      case 8:
+      case 9:
+         arkode = new ARKStepSolver(MPI_COMM_WORLD, ARKStepSolver::EXPLICIT);
+         arkode->Init(adv);
+         arkode->SetSStolerances(reltol, abstol);
+         arkode->SetMaxStep(dt);
+         if (ode_solver_type == 9) { arkode->SetERKTableNum(FEHLBERG_13_7_8); }
+         ode_solver = arkode; break;
+   }
+
+   // Initialize MFEM integrators, SUNDIALS integrators are initialized above
+   if (ode_solver_type < 7) { ode_solver->Init(adv); }
+
+   // 10. Perform time-integration (looping over the time iterations, ti,
+   //     with a time-step dt).
    bool done = false;
    for (int ti = 0; !done; )
    {

examples/sundials/makefile

@@ -60,15 +60,15 @@ PARALLEL_NAME := Parallel SUNDIALS example
 	@$(call mfem-test,$<,, $(SERIAL_NAME))
 
 # Testing: Specific execution options:
-# Example 9: test explicit CVODE time stepping
-EX9_COMMON_ARGS := -m ../../data/periodic-hexagon.mesh -p 0 -s 11
+# Example 9: test CVODE with CV_ADAMS (non-stiff implicit) time stepping
+EX9_COMMON_ARGS := -m ../../data/periodic-hexagon.mesh -p 0 -s 7
 EX9_ARGS  := $(EX9_COMMON_ARGS) -r  2 -dt 0.0018 -vs 25
 EX9P_ARGS := $(EX9_COMMON_ARGS) -rp 1 -dt 0.0009 -vs 50
 ex9-test-seq: ex9
 	@$(call mfem-test,$<,, $(SERIAL_NAME),$(EX9_ARGS))
 ex9p-test-par: ex9p
 	@$(call mfem-test,$<, $(RUN_MPI), $(PARALLEL_NAME),$(EX9P_ARGS))
-# Example 10: test implicit CVODE time stepping
+# Example 10: test CVODE with CV_BDF (stiff implicit) time stepping
 EX10_COMMON_ARGS := -m ../../data/beam-quad.mesh -o 2 -s 5 -dt 0.15 -tf 6 -vs 10
 EX10_ARGS  := $(EX10_COMMON_ARGS) -r  2
 EX10P_ARGS := $(EX10_COMMON_ARGS) -rp 1

fem/CMakeLists.txt

@@ -32,6 +32,7 @@ set(SRCS
   nonlininteg.cpp
   staticcond.cpp
   tmop.cpp
+  tmop_tools.cpp
   )
 
 set(HDRS
@@ -64,6 +65,7 @@ set(HDRS
   tfespace.hpp
   tintrules.hpp
   tmop.hpp
+  tmop_tools.hpp
   )
 
 if (MFEM_USE_SIDRE)

fem/bilininteg_diffusion.cpp

@@ -27,7 +27,7 @@ static void OccaPADiffusionSetup2D(const int D1D,
                                    const int NE,
                                    const Array<double> &W,
                                    const Vector &J,
-                                   const double COEFF,
+                                   const Vector &C,
                                    Vector &op)
 {
    occa::properties props;
@@ -35,7 +35,9 @@ static void OccaPADiffusionSetup2D(const int D1D,
    props["defines/Q1D"] = Q1D;
    const occa::memory o_W = OccaMemoryRead(W.GetMemory(), W.Size());
    const occa::memory o_J = OccaMemoryRead(J.GetMemory(), J.Size());
+   const occa::memory o_C = OccaMemoryRead(C.GetMemory(), C.Size());
    occa::memory o_op = OccaMemoryWrite(op.GetMemory(), op.Size());
+   const bool const_c = C.Size() == 1;
    const occa_id_t id = std::make_pair(D1D,Q1D);
    static occa_kernel_t OccaDiffSetup2D_ker;
    if (OccaDiffSetup2D_ker.find(id) == OccaDiffSetup2D_ker.end())
@@ -45,7 +47,7 @@ static void OccaPADiffusionSetup2D(const int D1D,
                                      "DiffusionSetup2D", props);
       OccaDiffSetup2D_ker.emplace(id, DiffusionSetup2D);
    }
-   OccaDiffSetup2D_ker.at(id)(NE, o_W, o_J, COEFF, o_op);
+   OccaDiffSetup2D_ker.at(id)(NE, o_W, o_J, o_C, o_op, const_c);
 }
 
 static void OccaPADiffusionSetup3D(const int D1D,
@@ -53,7 +55,7 @@ static void OccaPADiffusionSetup3D(const int D1D,
                                    const int NE,
                                    const Array<double> &W,
                                    const Vector &J,
-                                   const double COEFF,
+                                   const Vector &C,
                                    Vector &op)
 {
    occa::properties props;
@@ -61,7 +63,9 @@ static void OccaPADiffusionSetup3D(const int D1D,
    props["defines/Q1D"] = Q1D;
    const occa::memory o_W = OccaMemoryRead(W.GetMemory(), W.Size());
    const occa::memory o_J = OccaMemoryRead(J.GetMemory(), J.Size());
+   const occa::memory o_C = OccaMemoryRead(C.GetMemory(), C.Size());
    occa::memory o_op = OccaMemoryWrite(op.GetMemory(), op.Size());
+   const bool const_c = C.Size() == 1;
    const occa_id_t id = std::make_pair(D1D,Q1D);
    static occa_kernel_t OccaDiffSetup3D_ker;
    if (OccaDiffSetup3D_ker.find(id) == OccaDiffSetup3D_ker.end())
@@ -71,7 +75,7 @@ static void OccaPADiffusionSetup3D(const int D1D,
                                      "DiffusionSetup3D", props);
       OccaDiffSetup3D_ker.emplace(id, DiffusionSetup3D);
    }
-   OccaDiffSetup3D_ker.at(id)(NE, o_W, o_J, COEFF, o_op);
+   OccaDiffSetup3D_ker.at(id)(NE, o_W, o_J, o_C, o_op, const_c);
 }
 #endif // MFEM_USE_OCCA
 
@@ -80,14 +84,16 @@ static void PADiffusionSetup2D(const int Q1D,
                                const int NE,
                                const Array<double> &w,
                                const Vector &j,
-                               const double COEFF,
-                               Vector &op)
+                               const Vector &c,
+                               Vector &d)
 {
    const int NQ = Q1D*Q1D;
+   const bool const_c = c.Size() == 1;
    auto W = w.Read();
 
    auto J = Reshape(j.Read(), NQ, 2, 2, NE);
-   auto y = Reshape(op.Write(), NQ, 3, NE);
+   auto C = const_c ? Reshape(c.Read(), 1, 1) : Reshape(c.Read(), NQ, NE);
+   auto D = Reshape(d.Write(), NQ, 3, NE);
 
    MFEM_FORALL(e, NE,
    {
@@ -97,10 +103,11 @@ static void PADiffusionSetup2D(const int Q1D,
          const double J21 = J(q,1,0,e);
          const double J12 = J(q,0,1,e);
          const double J22 = J(q,1,1,e);
-         const double c_detJ = W[q] * COEFF / ((J11*J22)-(J21*J12));
-         y(q,0,e) =  c_detJ * (J12*J12 + J22*J22); // 1,1
-         y(q,1,e) = -c_detJ * (J12*J11 + J22*J21); // 1,2
-         y(q,2,e) =  c_detJ * (J11*J11 + J21*J21); // 2,2
+         const double coeff = const_c ? C(0,0) : C(q,e);
+         const double c_detJ = W[q] * coeff / ((J11*J22)-(J21*J12));
+         D(q,0,e) =  c_detJ * (J12*J12 + J22*J22); // 1,1
+         D(q,1,e) = -c_detJ * (J12*J11 + J22*J21); // 1,2
+         D(q,2,e) =  c_detJ * (J11*J11 + J21*J21); // 2,2
       }
    });
 }
@@ -110,13 +117,15 @@ static void PADiffusionSetup3D(const int Q1D,
                                const int NE,
                                const Array<double> &w,
                                const Vector &j,
-                               const double COEFF,
-                               Vector &op)
+                               const Vector &c,
+                               Vector &d)
 {
    const int NQ = Q1D*Q1D*Q1D;
+   const bool const_c = c.Size() == 1;
    auto W = w.Read();
    auto J = Reshape(j.Read(), NQ, 3, 3, NE);
-   auto y = Reshape(op.Write(), NQ, 6, NE);
+   auto C = const_c ? Reshape(c.Read(), 1, 1) : Reshape(c.Read(), NQ, NE);
+   auto D = Reshape(d.Write(), NQ, 6, NE);
    MFEM_FORALL(e, NE,
    {
       for (int q = 0; q < NQ; ++q)
@@ -133,7 +142,8 @@ static void PADiffusionSetup3D(const int Q1D,
          const double detJ = J11 * (J22 * J33 - J32 * J23) -
          /* */               J21 * (J12 * J33 - J32 * J13) +
          /* */               J31 * (J12 * J23 - J22 * J13);
-         const double c_detJ = W[q] * COEFF / detJ;
+         const double coeff = const_c ? C(0,0) : C(q,e);
+         const double c_detJ = W[q] * coeff / detJ;
          // adj(J)
          const double A11 = (J22 * J33) - (J23 * J32);
          const double A12 = (J32 * J13) - (J12 * J33);
@@ -145,12 +155,12 @@ static void PADiffusionSetup3D(const int Q1D,
          const double A32 = (J31 * J12) - (J11 * J32);
          const double A33 = (J11 * J22) - (J12 * J21);
          // detJ J^{-1} J^{-T} = (1/detJ) adj(J) adj(J)^T
-         y(q,0,e) = c_detJ * (A11*A11 + A12*A12 + A13*A13); // 1,1
-         y(q,1,e) = c_detJ * (A11*A21 + A12*A22 + A13*A23); // 2,1
-         y(q,2,e) = c_detJ * (A11*A31 + A12*A32 + A13*A33); // 3,1
-         y(q,3,e) = c_detJ * (A21*A21 + A22*A22 + A23*A23); // 2,2
-         y(q,4,e) = c_detJ * (A21*A31 + A22*A32 + A23*A33); // 3,2
-         y(q,5,e) = c_detJ * (A31*A31 + A32*A32 + A33*A33); // 3,3
+         D(q,0,e) = c_detJ * (A11*A11 + A12*A12 + A13*A13); // 1,1
+         D(q,1,e) = c_detJ * (A11*A21 + A12*A22 + A13*A23); // 2,1
+         D(q,2,e) = c_detJ * (A11*A31 + A12*A32 + A13*A33); // 3,1
+         D(q,3,e) = c_detJ * (A21*A21 + A22*A22 + A23*A23); // 2,2
+         D(q,4,e) = c_detJ * (A21*A31 + A22*A32 + A23*A33); // 3,2
+         D(q,5,e) = c_detJ * (A31*A31 + A32*A32 + A33*A33); // 3,3
       }
    });
 }
@@ -161,8 +171,8 @@ static void PADiffusionSetup(const int dim,
                              const int NE,
                              const Array<double> &W,
                              const Vector &J,
-                             const double COEFF,
-                             Vector &op)
+                             const Vector &C,
+                             Vector &D)
 {
    if (dim == 1) { MFEM_ABORT("dim==1 not supported in PADiffusionSetup"); }
    if (dim == 2)
@@ -170,22 +180,22 @@ static void PADiffusionSetup(const int dim,
 #ifdef MFEM_USE_OCCA
       if (DeviceCanUseOcca())
       {
-         OccaPADiffusionSetup2D(D1D, Q1D, NE, W, J, COEFF, op);
+         OccaPADiffusionSetup2D(D1D, Q1D, NE, W, J, C, D);
          return;
       }
 #endif // MFEM_USE_OCCA
-      PADiffusionSetup2D(Q1D, NE, W, J, COEFF, op);
+      PADiffusionSetup2D(Q1D, NE, W, J, C, D);
    }
    if (dim == 3)
    {
 #ifdef MFEM_USE_OCCA
       if (DeviceCanUseOcca())
       {
-         OccaPADiffusionSetup3D(D1D, Q1D, NE, W, J, COEFF, op);
+         OccaPADiffusionSetup3D(D1D, Q1D, NE, W, J, C, D);
          return;
       }
 #endif // MFEM_USE_OCCA
-      PADiffusionSetup3D(Q1D, NE, W, J, COEFF, op);
+      PADiffusionSetup3D(Q1D, NE, W, J, C, D);
    }
 }
 
@@ -205,11 +215,32 @@ void DiffusionIntegrator::AssemblePA(const FiniteElementSpace &fes)
    dofs1D = maps->ndof;
    quad1D = maps->nqpt;
    pa_data.SetSize(symmDims * nq * ne, Device::GetMemoryType());
-   ConstantCoefficient *cQ = dynamic_cast<ConstantCoefficient*>(Q);
-   MFEM_VERIFY(cQ != NULL, "only ConstantCoefficient is supported!");
-   const double coeff = cQ->constant;
-   PADiffusionSetup(dim, dofs1D, quad1D, ne, ir->GetWeights(), geom->J,
-                    coeff, pa_data);
+   Vector coeff;
+   if (Q == nullptr)
+   {
+      coeff.SetSize(1);
+      coeff(0) = 1.0;
+   }
+   else if (ConstantCoefficient* cQ = dynamic_cast<ConstantCoefficient*>(Q))
+   {
+      coeff.SetSize(1);
+      coeff(0) = cQ->constant;
+   }
+   else
+   {
+      coeff.SetSize(nq * ne);
+      auto C = Reshape(coeff.Write(), nq, ne);
+      for (int e = 0; e < ne; ++e)
+      {
+         ElementTransformation& T = *fes.GetElementTransformation(e);
+         for (int q = 0; q < nq; ++q)
+         {
+            C(q,e) = Q->Eval(T, ir->IntPoint(q));
+         }
+      }
+   }
+   PADiffusionSetup(dim, dofs1D, quad1D, ne, ir->GetWeights(), geom->J, coeff,
+                    pa_data);
 }
 
 #ifdef MFEM_USE_OCCA
@@ -1053,40 +1084,7 @@ static void PADiffusionApply(const int dim,
       MFEM_ABORT("OCCA PADiffusionApply unknown kernel!");
    }
 #endif // MFEM_USE_OCCA
-
-   if (Device::Allows(Backend::RAJA_CUDA))
-   {
-      if (dim == 2)
-      {
-         switch ((D1D << 4 ) | Q1D)
-         {
-            case 0x22: return PADiffusionApply2D<2,2>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x33: return PADiffusionApply2D<3,3>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x44: return PADiffusionApply2D<4,4>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x55: return PADiffusionApply2D<5,5>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x66: return PADiffusionApply2D<6,6>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x77: return PADiffusionApply2D<7,7>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x88: return PADiffusionApply2D<8,8>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x99: return PADiffusionApply2D<9,9>(NE,B,G,Bt,Gt,op,x,y);
-            default:   return PADiffusionApply2D(NE,B,G,Bt,Gt,op,x,y,D1D,Q1D);
-         }
-      }
-      if (dim == 3)
-      {
-         switch ((D1D << 4 ) | Q1D)
-         {
-            case 0x23: return PADiffusionApply3D<2,3>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x34: return PADiffusionApply3D<3,4>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x45: return PADiffusionApply3D<4,5>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x56: return PADiffusionApply3D<5,6>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x67: return PADiffusionApply3D<6,7>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x78: return PADiffusionApply3D<7,8>(NE,B,G,Bt,Gt,op,x,y);
-            case 0x89: return PADiffusionApply3D<8,9>(NE,B,G,Bt,Gt,op,x,y);
-            default:   return PADiffusionApply3D(NE,B,G,Bt,Gt,op,x,y,D1D,Q1D);
-         }
-      }
-   }
-   else if (dim == 2)
+   if (dim == 2)
    {
       switch ((D1D << 4 ) | Q1D)
       {

fem/bilininteg_mass.cpp

@@ -38,24 +38,40 @@ void MassIntegrator::AssemblePA(const FiniteElementSpace &fes)
    dofs1D = maps->ndof;
    quad1D = maps->nqpt;
    pa_data.SetSize(ne*nq, Device::GetMemoryType());
-   ConstantCoefficient *const_coeff = dynamic_cast<ConstantCoefficient*>(Q);
-   // TODO: other types of coefficients ...
+   Vector coeff;
+   if (Q == nullptr)
+   {
+      coeff.SetSize(1);
+      coeff(0) = 1.0;
+   }
+   else if (ConstantCoefficient* cQ = dynamic_cast<ConstantCoefficient*>(Q))
+   {
+      coeff.SetSize(1);
+      coeff(0) = cQ->constant;
+   }
+   else
+   {
+      coeff.SetSize(nq * ne);
+      auto C = Reshape(coeff.Write(), nq, ne);
+      for (int e = 0; e < ne; ++e)
+      {
+         ElementTransformation& T = *fes.GetElementTransformation(e);
+         for (int q = 0; q < nq; ++q)
+         {
+            C(q,e) = Q->Eval(T, ir->IntPoint(q));
+         }
+      }
+   }
    if (dim==1) { MFEM_ABORT("Not supported yet... stay tuned!"); }
    if (dim==2)
    {
-      double constant = 0.0;
-      if (const_coeff)
-      {
-         constant = const_coeff->constant;
-      }
-      else
-      {
-         MFEM_ABORT("Coefficient type not supported");
-      }
       const int NE = ne;
       const int NQ = nq;
+      const bool const_c = coeff.Size() == 1;
       auto w = ir->GetWeights().Read();
       auto J = Reshape(geom->J.Read(), NQ,2,2,NE);
+      auto C =
+         const_c ? Reshape(coeff.Read(), 1,1) : Reshape(coeff.Read(), NQ,NE);
       auto v = Reshape(pa_data.Write(), NQ, NE);
       MFEM_FORALL(e, NE,
       {
@@ -66,25 +82,20 @@ void MassIntegrator::AssemblePA(const FiniteElementSpace &fes)
             const double J21 = J(q,0,1,e);
             const double J22 = J(q,1,1,e);
             const double detJ = (J11*J22)-(J21*J12);
-            v(q,e) =  w[q] * constant * detJ;
+            const double coeff = const_c ? C(0,0) : C(q,e);
+            v(q,e) =  w[q] * coeff * detJ;
          }
       });
    }
    if (dim==3)
    {
-      double constant = 0.0;
-      if (const_coeff)
-      {
-         constant = const_coeff->constant;
-      }
-      else
-      {
-         MFEM_ABORT("Coefficient type not supported");
-      }
       const int NE = ne;
       const int NQ = nq;
+      const bool const_c = coeff.Size() == 1;
       auto W = ir->GetWeights().Read();
       auto J = Reshape(geom->J.Read(), NQ,3,3,NE);
+      auto C =
+         const_c ? Reshape(coeff.Read(), 1,1) : Reshape(coeff.Read(), NQ,NE);
       auto v = Reshape(pa_data.Write(), NQ,NE);
       MFEM_FORALL(e, NE,
       {
@@ -96,7 +107,8 @@ void MassIntegrator::AssemblePA(const FiniteElementSpace &fes)
             const double detJ = J11 * (J22 * J33 - J32 * J23) -
             /* */               J21 * (J12 * J33 - J32 * J13) +
             /* */               J31 * (J12 * J23 - J22 * J13);
-            v(q,e) = W[q] * constant * detJ;
+            const double coeff = const_c ? C(0,0) : C(q,e);
+            v(q,e) = W[q] * coeff * detJ;
          }
       });
    }
@@ -749,42 +761,9 @@ static void PAMassApply(const int dim,
       MFEM_ABORT("OCCA PA Mass Apply unknown kernel!");
    }
 #endif // MFEM_USE_OCCA
-
-   if (Device::Allows(Backend::RAJA_CUDA))
+   if (dim == 2)
    {
-      if (dim == 2)
-      {
-         switch ((D1D << 4 ) | Q1D)
-         {
-            case 0x22: return PAMassApply2D<2,2>(NE, B, Bt, op, x, y);
-            case 0x33: return PAMassApply2D<3,3>(NE, B, Bt, op, x, y);
-            case 0x44: return PAMassApply2D<4,4>(NE, B, Bt, op, x, y);
-            case 0x55: return PAMassApply2D<5,5>(NE, B, Bt, op, x, y);
-            case 0x66: return PAMassApply2D<6,6>(NE, B, Bt, op, x, y);
-            case 0x77: return PAMassApply2D<7,7>(NE, B, Bt, op, x, y);
-            case 0x88: return PAMassApply2D<8,8>(NE, B, Bt, op, x, y);
-            case 0x99: return PAMassApply2D<9,9>(NE, B, Bt, op, x, y);
-            default:   return PAMassApply2D(NE, B, Bt, op, x, y, D1D, Q1D);
-         }
-      }
-      if (dim == 3)
-      {
-         switch ((D1D << 4 ) | Q1D)
-         {
-            case 0x23: return PAMassApply3D<2,3>(NE, B, Bt, op, x, y);
-            case 0x34: return PAMassApply3D<3,4>(NE, B, Bt, op, x, y);
-            case 0x45: return PAMassApply3D<4,5>(NE, B, Bt, op, x, y);
-            case 0x56: return PAMassApply3D<5,6>(NE, B, Bt, op, x, y);
-            case 0x67: return PAMassApply3D<6,7>(NE, B, Bt, op, x, y);
-            case 0x78: return PAMassApply3D<7,8>(NE, B, Bt, op, x, y);
-            case 0x89: return PAMassApply3D<8,9>(NE, B, Bt, op, x, y);
-            default:   return PAMassApply3D(NE, B, Bt, op, x, y, D1D, Q1D);
-         }
-      }
-   }
-   else if (dim == 2)
-   {
-      switch ((D1D << 4 ) | Q1D)
+      switch ((D1D << 4) | Q1D)
       {
          case 0x22: return SmemPAMassApply2D<2,2,16>(NE, B, Bt, op, x, y);
          case 0x33: return SmemPAMassApply2D<3,3,16>(NE, B, Bt, op, x, y);
@@ -799,7 +778,7 @@ static void PAMassApply(const int dim,
    }
    else if (dim == 3)
    {
-      switch ((D1D << 4 ) | Q1D)
+      switch ((D1D << 4) | Q1D)
       {
          case 0x23: return SmemPAMassApply3D<2,3>(NE, B, Bt, op, x, y);
          case 0x34: return SmemPAMassApply3D<3,4>(NE, B, Bt, op, x, y);

fem/fe.cpp

@@ -9635,6 +9635,7 @@ void L2_TetrahedronElement::ProjectDelta(int vertex, Vector &dofs) const
             const IntegrationPoint &ip = Nodes.IntPoint(i);
             dofs[i] = pow(ip.y, Order);
          }
+         break;
       case 3:
          for (int i = 0; i < Dof; i++)
          {
@@ -11965,6 +11966,10 @@ Linear3DFiniteElement TetrahedronFE;
 // Object declared in mesh/wedge.hpp.
 // Defined here to ensure it is constructed after 'poly1d' and before
 // 'Geometries'.
+// TODO: define as thread_local to prevent race conditions in GLVis, because
+// there is no "LinearWedgeFiniteElement" and WedgeFE is in turn used from two
+// different threads for different things in GLVis. We also don't want to turn
+// MFEM_THREAD_SAFE on globally. (See PR #731)
 H1_WedgeElement WedgeFE(1);
 
 // Object declared in geom.hpp.

fem/fem.hpp

@@ -31,6 +31,7 @@
 #include "estimators.hpp"
 #include "staticcond.hpp"
 #include "tmop.hpp"
+#include "tmop_tools.hpp"
 
 #ifdef MFEM_USE_MPI
 #include "pfespace.hpp"

fem/fespace.cpp

@@ -567,6 +567,40 @@ bool FiniteElementSpace::DofFinalizable(int dof, const Array<bool>& finalized,
    return true;
 }
 
+void FiniteElementSpace::GetDegenerateFaceDofs(int index,
+                                               Array<int> &dofs) const
+{
+   // In NC meshes with prisms, a special constraint occurs where a prism edge
+   // is slave to a quadrilateral face. Rather than introduce a new edge-face
+   // constraint type, we handle such cases as degenerate face-face constraints,
+   // where the point-matrix rectangle has zero height. This method returns
+   // DOFs for the first edge of the rectangle, duplicated in the orthogonal
+   // direction, to resemble DOFs for a quadrilateral face. The extra DOFs are
+   // ignored by FiniteElementSpace::AddDependencies.
+
+   Array<int> edof;
+   GetEdgeDofs(-1 - index, edof);
+
+   int nv = fec->DofForGeometry(Geometry::POINT);
+   int ne = fec->DofForGeometry(Geometry::SEGMENT);
+   int nn = 2*nv + ne;
+
+   dofs.SetSize(nn*nn);
+   dofs = edof[0];
+
+   // copy first two vertex DOFs
+   for (int i = 0; i < nv; i++)
+   {
+      dofs[i] = edof[i];
+      dofs[nv+i] = edof[nv+i];
+   }
+   // copy first edge DOFs
+   for (int i = 0; i < ne; i++)
+   {
+      dofs[4*nv + i] = edof[2*nv + i];
+   }
+}
+
 void
 FiniteElementSpace::GetEntityDofs(int entity, int index, Array<int> &dofs) const
 {
@@ -574,7 +608,8 @@ FiniteElementSpace::GetEntityDofs(int entity, int index, Array<int> &dofs) const
    {
       case 0: GetVertexDofs(index, dofs); break;
       case 1: GetEdgeDofs(index, dofs); break;
-      case 2: GetFaceDofs(index, dofs); break;
+      case 2: (index >= 0) ? GetFaceDofs(index, dofs)
+         /*             */ : GetDegenerateFaceDofs(index, dofs);
    }
 }
 
@@ -597,28 +632,33 @@ void FiniteElementSpace::BuildConformingInterpolation() const
    // collect local edge/face dependencies
    for (int entity = 1; entity <= 2; entity++)
    {
-      const NCMesh::NCList &list = (entity > 1) ? mesh->ncmesh->GetFaceList()
-                                   /*        */ : mesh->ncmesh->GetEdgeList();
+      const NCMesh::NCList &list = mesh->ncmesh->GetNCList(entity);
       if (!list.masters.size()) { continue; }
 
-      IsoparametricTransformation T;
-      if (entity > 1) { T.SetFE(&QuadrilateralFE); }
-      else { T.SetFE(&SegmentFE); }
-
-      Geometry::Type geom = (entity > 1) ? Geometry::SQUARE : Geometry::SEGMENT;
-      const FiniteElement* fe = fec->FiniteElementForGeometry(geom);
-      if (!fe) { continue; }
-
       Array<int> master_dofs, slave_dofs;
-      DenseMatrix I(fe->GetDof());
+
+      IsoparametricTransformation T;
+      DenseMatrix I;
 
       // loop through all master edges/faces, constrain their slave edges/faces
       for (unsigned mi = 0; mi < list.masters.size(); mi++)
       {
          const NCMesh::Master &master = list.masters[mi];
+
          GetEntityDofs(entity, master.index, master_dofs);
          if (!master_dofs.Size()) { continue; }
 
+         const FiniteElement* fe = fec->FiniteElementForGeometry(master.Geom());
+         if (!fe) { continue; }
+
+         switch (master.geom)
+         {
+            case Geometry::SQUARE:   T.SetFE(&QuadrilateralFE); break;
+            case Geometry::TRIANGLE: T.SetFE(&TriangleFE); break;
+            case Geometry::SEGMENT:  T.SetFE(&SegmentFE); break;
+            default: MFEM_ABORT("unsupported geometry");
+         }
+
          for (int si = master.slaves_begin; si < master.slaves_end; si++)
          {
             const NCMesh::Slave &slave = list.slaves[si];
@@ -791,8 +831,14 @@ const Operator *FiniteElementSpace::GetElementRestriction(
 {
    // Check if we have a discontinuous space using the FE collection:
    const L2_FECollection *dg_space = dynamic_cast<const L2_FECollection*>(fec);
-   if (dg_space) { return NULL; }
-   // TODO: support other DG collections.
+   if (dg_space)
+   {
+      if (L2E_nat.Ptr() == NULL)
+      {
+         L2E_nat.Reset(new L2ElementRestriction(*this));
+      }
+      return L2E_nat.Ptr();
+   }
    if (e_ordering == ElementDofOrdering::LEXICOGRAPHIC)
    {
       if (L2E_lex.Ptr() == NULL)
@@ -905,7 +951,7 @@ void FiniteElementSpace::GetLocalRefinementMatrices(
    const FiniteElement *fe = fec->FiniteElementForGeometry(geom);
 
    const CoarseFineTransformations &rtrans = mesh->GetRefinementTransforms();
-   const DenseTensor &pmats = rtrans.GetPointMatrices(geom);
+   const DenseTensor &pmats = rtrans.point_matrices[geom];
 
    int nmat = pmats.SizeK();
    int ldof = fe->GetDof(); // assuming the same FE everywhere
@@ -944,7 +990,9 @@ FiniteElementSpace::RefinementOperator::RefinementOperator
    : fespace(fespace)
    , old_elem_dof(old_elem_dof)
 {
-   MFEM_VERIFY(fespace->GetNDofs() >= old_ndofs,
+   const Mesh* mesh = fespace->GetMesh();
+   MFEM_VERIFY(mesh->ReduceInt(fespace->GetNDofs()) >=
+               mesh->ReduceInt(old_ndofs),
                "Previous space is not coarser.");
 
    width = old_ndofs * fespace->GetVDim();
@@ -1054,7 +1102,7 @@ FiniteElementSpace::DerefinementOperator::DerefinementOperator(
          f_fes->fec->FiniteElementForGeometry(geom);
       const FiniteElement *coarse_fe =
          c_fes->fec->FiniteElementForGeometry(geom);
-      const DenseTensor &pmats = rtrans.GetPointMatrices(geom);
+      const DenseTensor &pmats = rtrans.point_matrices[geom];
 
       lP.SetSize(fine_fe->GetDof(), coarse_fe->GetDof(), pmats.SizeK());
       lM.SetSize(fine_fe->GetDof(),   fine_fe->GetDof(), pmats.SizeK());
@@ -1170,7 +1218,7 @@ void FiniteElementSpace::GetLocalDerefinementMatrices(Geometry::Type geom,
 
    const CoarseFineTransformations &dtrans =
       mesh->ncmesh->GetDerefinementTransforms();
-   const DenseTensor &pmats = dtrans.GetPointMatrices(geom);
+   const DenseTensor &pmats = dtrans.point_matrices[geom];
 
    const int nmat = pmats.SizeK();
    const int ldof = fe->GetDof();
@@ -1275,7 +1323,7 @@ void FiniteElementSpace::GetLocalRefinementMatrices(
       coarse_fes.fec->FiniteElementForGeometry(geom);
 
    const CoarseFineTransformations &rtrans = mesh->GetRefinementTransforms();
-   const DenseTensor &pmats = rtrans.GetPointMatrices(geom);
+   const DenseTensor &pmats = rtrans.point_matrices[geom];
 
    int nmat = pmats.SizeK();
 
@@ -1368,31 +1416,26 @@ void FiniteElementSpace::UpdateNURBS()
 void FiniteElementSpace::Construct()
 {
    // This method should be used only for non-NURBS spaces.
-   MFEM_ASSERT(!NURBSext, "internal error");
+   MFEM_VERIFY(!NURBSext, "internal error");
 
    elem_dof = NULL;
    bdrElem_dof = NULL;
 
-   nvdofs = mesh->GetNV() * fec->DofForGeometry(Geometry::POINT);
-
-   if ( mesh->Dimension() > 1 )
-   {
-      nedofs = mesh->GetNEdges() * fec->DofForGeometry(Geometry::SEGMENT);
-   }
-   else
-   {
-      nedofs = 0;
-   }
-
    ndofs = 0;
-   nfdofs = 0;
-   nbdofs = 0;
+   nedofs = nfdofs = nbdofs = 0;
    bdofs = NULL;
    fdofs = NULL;
    cP = NULL;
    cR = NULL;
    cP_is_set = false;
-   // Th is initialized/destroyed before this method is called.
+   // 'Th' is initialized/destroyed before this method is called.
+
+   nvdofs = mesh->GetNV() * fec->DofForGeometry(Geometry::POINT);
+
+   if (mesh->Dimension() > 1)
+   {
+      nedofs = mesh->GetNEdges() * fec->DofForGeometry(Geometry::SEGMENT);
+   }
 
    if (mesh->GetNFaces() > 0)
    {
@@ -1424,8 +1467,7 @@ void FiniteElementSpace::Construct()
       bdofs[0] = 0;
       for (int i = 0; i < mesh->GetNE(); i++)
       {
-         Geometry::Type geom = mesh->GetElementBaseGeometry(i);
-         nbdofs += fec->DofForGeometry(geom);
+         nbdofs += fec->DofForGeometry(mesh->GetElementBaseGeometry(i));
          bdofs[i+1] = nbdofs;
       }
    }
@@ -1436,7 +1478,7 @@ void FiniteElementSpace::Construct()
    // later.
 }
 
-void FiniteElementSpace::GetElementDofs (int i, Array<int> &dofs) const
+void FiniteElementSpace::GetElementDofs(int i, Array<int> &dofs) const
 {
    if (elem_dof)
    {
@@ -2513,7 +2555,7 @@ L2ProjectionGridTransfer::L2Projection::L2Projection(
    Vector shape_lor(ndof_lor);
 
    const Geometry::Type geom = fe_ho->GetGeomType();
-   const DenseTensor &pmats = cf_tr.GetPointMatrices(geom);
+   const DenseTensor &pmats = cf_tr.point_matrices[geom];
    emb_tr.SetIdentityTransformation(geom);
 
    for (int iho=0; iho<nel_ho; ++iho)
@@ -2536,7 +2578,7 @@ L2ProjectionGridTransfer::L2Projection::L2Projection(
 
          // Create the transformation that embeds the fine low-order element
          // within the coarse high-order element in reference space
-         emb_tr.GetPointMat() = pmats(iref);
+         emb_tr.GetPointMat() = pmats(cf_tr.embeddings[ilor].matrix);
          emb_tr.FinalizeTransformation();
 
          int order = fe_lor->GetOrder() + fe_ho->GetOrder() + el_tr->OrderW();
@@ -2637,6 +2679,70 @@ const Operator &L2ProjectionGridTransfer::BackwardOperator()
    return *B;
 }
 
+L2ElementRestriction::L2ElementRestriction(const FiniteElementSpace &fes)
+   : ne(fes.GetNE()),
+     vdim(fes.GetVDim()),
+     byvdim(fes.GetOrdering() == Ordering::byVDIM),
+     ndof(ne > 0 ? fes.GetFE(0)->GetDof() : 0)
+{
+   height = vdim*ne*ndof;
+   width = vdim*ne*ndof;
+}
+
+void L2ElementRestriction::Mult(const Vector &x, Vector &y) const
+{
+   for (int iel=0; iel<ne; ++iel)
+   {
+      for (int vd=0; vd<vdim; ++vd)
+      {
+         for (int idof=0; idof<ndof; ++idof)
+         {
+            // E-vector dimensions (dofs, vdim, elements)
+            // L-vector dimensions: byVDIM:  (vdim, dofs, element)
+            //                      byNODES: (dofs, elements, vdim)
+            int yidx = iel*vdim*ndof + vd*ndof + idof;
+            int xidx;
+            if (byvdim)
+            {
+               xidx = iel*ndof*vdim + idof*vdim + vd;
+            }
+            else
+            {
+               xidx = vd*ne*ndof + iel*ndof + idof;
+            }
+            y[yidx] = x[xidx];
+         }
+      }
+   }
+}
+
+void L2ElementRestriction::MultTranspose(const Vector &x, Vector &y) const
+{
+   // Since this restriction is a permutation, the transpose is the inverse
+   for (int iel=0; iel<ne; ++iel)
+   {
+      for (int vd=0; vd<vdim; ++vd)
+      {
+         for (int idof=0; idof<ndof; ++idof)
+         {
+            // E-vector dimensions (dofs, vdim, elements)
+            // L-vector dimensions: byVDIM:  (vdim, dofs, element)
+            //                      byNODES: (dofs, elements, vdim)
+            int xidx = iel*vdim*ndof + vd*ndof + idof;
+            int yidx;
+            if (byvdim)
+            {
+               yidx = iel*ndof*vdim + idof*vdim + vd;
+            }
+            else
+            {
+               yidx = vd*ne*ndof + iel*ndof + idof;
+            }
+            y[yidx] = x[xidx];
+         }
+      }
+   }
+}
 
 ElementRestriction::ElementRestriction(const FiniteElementSpace &f,
                                        ElementDofOrdering e_ordering)

fem/fespace.hpp

@@ -146,6 +146,8 @@ protected:
 
    /// Helper to get vertex, edge or face DOFs (entity=0,1,2 resp.).
    void GetEntityDofs(int entity, int index, Array<int> &dofs) const;
+   // Get degenerate face DOFs: see explanation in method implementation.
+   void GetDegenerateFaceDofs(int index, Array<int> &dofs) const;
 
    /// Calculate the cP and cR matrices for a nonconforming mesh.
    void BuildConformingInterpolation() const;
@@ -156,6 +158,7 @@ protected:
    static bool DofFinalizable(int dof, const Array<bool>& finalized,
                               const SparseMatrix& deps);
 
+   /// Replicate 'mat' in the vector dimension, according to vdim ordering mode.
    void MakeVDimMatrix(SparseMatrix &mat) const;
 
    /// GridFunction interpolation operator applicable after mesh refinement.
@@ -304,8 +307,9 @@ public:
        The parameter @a e_ordering describes how the local DOFs in each element
        should be ordered, see ElementDofOrdering.
 
-       For discontinuous spaces, where the element-restriction is the identity,
-       this method will return NULL.
+       For discontinuous spaces, the element restriction corresponds to a
+       permutation of the degrees of freedom, implemented by the
+       L2ElementRestriction class.
 
        The returned Operator is owned by the FiniteElementSpace. */
    const Operator *GetElementRestriction(ElementDofOrdering e_ordering) const;
@@ -896,6 +900,22 @@ public:
    void MultTranspose(const Vector &x, Vector &y) const;
 };
 
+/// Operator that converts L2 FiniteElementSpace L-vectors to E-vectors.
+/** Objects of this type are typically created and owned by FiniteElementSpace
+    objects, see FiniteElementSpace::GetElementRestriction(). L-vectors
+    corresponding to grid functions in L2 finite element spaces differ from
+    E-vectors only in the ordering of the degrees of freedom. */
+class L2ElementRestriction : public Operator
+{
+   const int ne;
+   const int vdim;
+   const bool byvdim;
+   const int ndof;
+public:
+   L2ElementRestriction(const FiniteElementSpace&);
+   void Mult(const Vector &x, Vector &y) const;
+   void MultTranspose(const Vector &x, Vector &y) const;
+};
 
 /** @brief A class that performs interpolation from an E-vector to quadrature
     point values and/or derivatives (Q-vectors). */

fem/gridfunc.cpp

@@ -1716,6 +1716,7 @@ void GridFunction::ProjectDiscCoefficient(VectorCoefficient &coeff,
    Array<int> vdofs;
    Vector vals;
 
+   HostWrite();
    // maximal element attribute for each dof
    dof_attr.SetSize(fes->GetVSize());
    dof_attr = -1;

fem/gridfunc.hpp

@@ -434,6 +434,8 @@ public:
    /** The GridFunction is resized using the SetSize() method. */
    virtual void SetSpace(FiniteElementSpace *f);
 
+   using Vector::MakeRef;
+
    /** @brief Make the GridFunction reference external data on a new
        FiniteElementSpace. */
    /** This method changes the FiniteElementSpace associated with the

fem/linearform.cpp

@@ -138,7 +138,11 @@ void LinearForm::Assemble()
          eltrans = fes -> GetBdrElementTransformation (i);
          for (int k=0; k < blfi.Size(); k++)
          {
+            if (blfi_marker[k] &&
+                (*blfi_marker[k])[bdr_attr-1] == 0) { continue; }
+
             blfi[k]->AssembleRHSElementVect(*fes->GetBE(i), *eltrans, elemvect);
+
             AddElementVector (vdofs, elemvect);
          }
       }

fem/lininteg.cpp

@@ -350,7 +350,6 @@ void VectorFEDomainLFIntegrator::AssembleDeltaElementVect(
    vshape.Mult(vec, elvect);
 }
 
-
 void VectorBoundaryFluxLFIntegrator::AssembleRHSElementVect(
    const FiniteElement &el, ElementTransformation &Tr, Vector &elvect)
 {
@@ -397,19 +396,26 @@ void VectorFEBoundaryFluxLFIntegrator::AssembleRHSElementVect(
    if (ir == NULL)
    {
       int intorder = 2*el.GetOrder();  // <----------
+      if (F == NULL)
+      {
+         intorder -= el.GetOrder() + 1;
+      }
       ir = &IntRules.Get(el.GetGeomType(), intorder);
    }
 
    for (int i = 0; i < ir->GetNPoints(); i++)
    {
       const IntegrationPoint &ip = ir->IntPoint(i);
-
-      Tr.SetIntPoint (&ip);
-      double val = ip.weight*F.Eval(Tr, ip);
-
       el.CalcShape(ip, shape);
 
-      add(elvect, val, shape, elvect);
+      double val = ip.weight;
+      if (F)
+      {
+         Tr.SetIntPoint (&ip);
+         val *= F->Eval(Tr, ip);
+      }
+
+      elvect.Add(val, shape);
    }
 }
 

fem/lininteg.hpp

@@ -279,11 +279,12 @@ public:
 class VectorFEBoundaryFluxLFIntegrator : public LinearFormIntegrator
 {
 private:
-   Coefficient &F;
+   Coefficient *F;
    Vector shape;
 
 public:
-   VectorFEBoundaryFluxLFIntegrator(Coefficient &f) : F(f) { }
+   VectorFEBoundaryFluxLFIntegrator() : F(NULL) { }
+   VectorFEBoundaryFluxLFIntegrator(Coefficient &f) : F(&f) { }
 
    virtual void AssembleRHSElementVect(const FiniteElement &el,
                                        ElementTransformation &Tr,

fem/nonlinearform.cpp

@@ -65,6 +65,8 @@ double NonlinearForm::GetGridFunctionEnergy(const Vector &x) const
    Vector el_x;
    const FiniteElement *fe;
    ElementTransformation *T;
+   Mesh *mesh = fes->GetMesh();
+
    double energy = 0.0;
 
    if (dnfi.Size())
@@ -84,14 +86,81 @@ double NonlinearForm::GetGridFunctionEnergy(const Vector &x) const
 
    if (fnfi.Size())
    {
-      MFEM_ABORT("TODO: add energy contribution from interior face terms");
+      FaceElementTransformations *tr;
+      const FiniteElement *fe1, *fe2;
+      Array<int> vdofs2;
+
+      for (int i = 0; i < mesh->GetNumFaces(); i++)
+      {
+         tr = mesh->GetInteriorFaceTransformations(i);
+         if (tr != NULL)
+         {
+            fes->GetElementVDofs(tr->Elem1No, vdofs);
+            fes->GetElementVDofs(tr->Elem2No, vdofs2);
+            vdofs.Append (vdofs2);
+            x.GetSubVector(vdofs, el_x);
+            fe1 = fes->GetFE(tr->Elem1No);
+            fe2 = fes->GetFE(tr->Elem2No);
+            for (int k = 0; k < fnfi.Size(); k++)
+            {
+               energy += fnfi[k]->GetFaceEnergy(*fe1, *fe2, *tr, el_x);
+            }
+         }
+      }
    }
 
    if (bfnfi.Size())
    {
-      MFEM_ABORT("TODO: add energy contribution from boundary face terms");
-   }
+      FaceElementTransformations *tr;
+      const FiniteElement *fe1, *fe2;
 
+      // Which boundary attributes need to be processed?
+      Array<int> bdr_attr_marker(mesh->bdr_attributes.Size() ?
+                                 mesh->bdr_attributes.Max() : 0);
+      bdr_attr_marker = 0;
+      for (int k = 0; k < bfnfi.Size(); k++)
+      {
+         if (bfnfi_marker[k] == NULL)
+         {
+            bdr_attr_marker = 1;
+            break;
+         }
+         Array<int> &bdr_marker = *bfnfi_marker[k];
+         MFEM_ASSERT(bdr_marker.Size() == bdr_attr_marker.Size(),
+                     "invalid boundary marker for boundary face integrator #"
+                     << k << ", counting from zero");
+         for (int i = 0; i < bdr_attr_marker.Size(); i++)
+         {
+            bdr_attr_marker[i] |= bdr_marker[i];
+         }
+      }
+
+      for (int i = 0; i < fes -> GetNBE(); i++)
+      {
+         const int bdr_attr = mesh->GetBdrAttribute(i);
+         if (bdr_attr_marker[bdr_attr-1] == 0) { continue; }
+
+         tr = mesh->GetBdrFaceTransformations (i);
+         if (tr != NULL)
+         {
+            fes->GetElementVDofs(tr->Elem1No, vdofs);
+            x.GetSubVector(vdofs, el_x);
+
+            fe1 = fes->GetFE(tr->Elem1No);
+            // The fe2 object is really a dummy and not used on the boundaries,
+            // but we can't dereference a NULL pointer, and we don't want to
+            // actually make a fake element.
+            fe2 = fe1;
+            for (int k = 0; k < bfnfi.Size(); k++)
+            {
+               if (bfnfi_marker[k] &&
+                   (*bfnfi_marker[k])[bdr_attr-1] == 0) { continue; }
+
+               energy += bfnfi[k]->GetFaceEnergy(*fe1, *fe2, *tr, el_x);
+            }
+         }
+      }
+   }
    return energy;
 }
 

fem/nonlinearform.hpp

@@ -111,7 +111,7 @@ public:
        be fes->GetVSize(). */
    double GetGridFunctionEnergy(const Vector &x) const;
 
-   /// Compute the enery corresponding to the state @a x.
+   /// Compute the energy corresponding to the state @a x.
    /** In general, @a x may have non-homogeneous essential boundary values.
 
        The state @a x must be a true-dof vector. */

fem/nonlininteg.cpp

@@ -55,6 +55,14 @@ double NonlinearFormIntegrator::GetElementEnergy(
    return 0.0;
 }
 
+double NonlinearFormIntegrator::GetFaceEnergy(
+   const FiniteElement &el1, const FiniteElement &el2,
+   FaceElementTransformations &Tr, const Vector &elfun)
+{
+   mfem_error("NonlinearFormIntegrator::GetFaceEnergy"
+              " is not overloaded!");
+   return 0.0;
+}
 
 void BlockNonlinearFormIntegrator::AssembleElementVector(
    const Array<const FiniteElement *> &el,

fem/nonlininteg.hpp

@@ -63,11 +63,17 @@ public:
                                  FaceElementTransformations &Tr,
                                  const Vector &elfun, DenseMatrix &elmat);
 
-   /// Compute the local energy
+   /// Compute the local energy/functional
    virtual double GetElementEnergy(const FiniteElement &el,
                                    ElementTransformation &Tr,
                                    const Vector &elfun);
 
+   /// Compute the face(s) contribution to the energy/functional
+   virtual double GetFaceEnergy(const FiniteElement &el1,
+                                const FiniteElement &el2,
+                                FaceElementTransformations &Tr,
+                                const Vector &elfun);
+
    virtual ~NonlinearFormIntegrator() { }
 };
 

fem/occa.okl

@@ -38,19 +38,24 @@ typedef double* QLocal3D_t @dim(Q1D, Q1D, Q1D, NE);
 typedef double* Jacobian2D_t @dim(Q2D, 2, 2, NE);
 typedef double* Jacobian3D_t @dim(Q3D, 3, 3, NE);
 
+typedef double* Coeff2D_t @dim(Q2D, NE);
+typedef double* Coeff3D_t @dim(Q3D, NE);
+
 typedef double* SymmOperator2D_t @dim(Q2D, 3, NE);
 typedef double* SymmOperator3D_t @dim(Q3D, 6, NE);
 
 @kernel void DiffusionSetup2D(const int NE,
                               @restrict const double *W,
                               @restrict const Jacobian2D_t J,
-                              const double COEFF,
-                              @restrict SymmOperator2D_t op) {
+                              @restrict const Coeff2D_t C,
+                              @restrict SymmOperator2D_t op,
+                              const bool const_c) {
   for (int e = 0; e < NE; ++e; @outer) {
     for (int q = 0; q < Q2D; ++q; @inner) {
       const double J11 = J(q, 0, 0, e), J12 = J(q, 1, 0, e);
       const double J21 = J(q, 0, 1, e), J22 = J(q, 1, 1, e);
-      const double c_detJ = W[q] * COEFF / ((J11 * J22) - (J21 * J12));
+      const double coeff = const_c ? C(0,0) : C(q,e);
+      const double c_detJ = W[q] * coeff / ((J11 * J22) - (J21 * J12));
       op(q, 0, e) =  c_detJ * (J21*J21 + J22*J22); // (1,1)
       op(q, 1, e) = -c_detJ * (J21*J11 + J22*J12); // (1,2), (2,1)
       op(q, 2, e) =  c_detJ * (J11*J11 + J12*J12); // (2,2)
@@ -61,8 +66,9 @@ typedef double* SymmOperator3D_t @dim(Q3D, 6, NE);
 @kernel void DiffusionSetup3D(const int NE,
                               @restrict const double *W,
                               @restrict const Jacobian3D_t J,
-                              const double COEFF,
-                              @restrict SymmOperator3D_t op) {
+                              @restrict const Coeff3D_t C,
+                              @restrict SymmOperator3D_t op,
+                              const bool const_c) {
   for (int e = 0; e < NE; ++e; @outer) {
     for (int q = 0; q < Q3D; ++q; @inner) {
       const double J11 = J(q, 0, 0, e), J12 = J(q, 1, 0, e), J13 = J(q, 2, 0, e);
@@ -72,7 +78,8 @@ typedef double* SymmOperator3D_t @dim(Q3D, 6, NE);
       const double detJ = ((J11 * J22 * J33) + (J12 * J23 * J31) + (J13 * J21 * J32) -
                            (J13 * J22 * J31) - (J12 * J21 * J33) - (J11 * J23 * J32));
 
-      const double c_detJ = W[q] * COEFF / detJ;
+      const double coeff = const_c ? C(0,0) : C(q,e);
+      const double c_detJ = W[q] * coeff / detJ;
 
       // adj(J)
       const double A11 = (J22 * J33) - (J23 * J32);

fem/pfespace.cpp

@@ -14,6 +14,7 @@
 #ifdef MFEM_USE_MPI
 
 #include "pfespace.hpp"
+#include "../general/forall.hpp"
 #include "../general/sort_pairs.hpp"
 #include "../mesh/mesh_headers.hpp"
 #include "../general/binaryio.hpp"
@@ -97,6 +98,8 @@ void ParFiniteElementSpace::ParInit(ParMesh *pm)
 
    gcomm = NULL;
 
+   gfdofs = NULL;
+
    P = NULL;
    Pconf = NULL;
    R = NULL;
@@ -147,20 +150,37 @@ void ParFiniteElementSpace::Construct()
       // cut space.
       ConstructTrueDofs();
 
+      ngedofs = ngfdofs = 0;
+      gfdofs = NULL;
+
       // calculate number of ghost DOFs
       ngvdofs = pncmesh->GetNGhostVertices()
                 * fec->DofForGeometry(Geometry::POINT);
 
-      ngedofs = ngfdofs = 0;
       if (pmesh->Dimension() > 1)
       {
          ngedofs = pncmesh->GetNGhostEdges()
                    * fec->DofForGeometry(Geometry::SEGMENT);
       }
+
       if (pmesh->Dimension() > 2)
       {
-         ngfdofs = pncmesh->GetNGhostFaces()
-                   * fec->DofForGeometry(pncmesh->GetGhostFaceGeometry(0));
+         if (fdofs != NULL) // have mixed faces
+         {
+            gfdofs = new int[pncmesh->GetNGhostFaces()+1];
+            gfdofs[0] = 0;
+            for (int i = 0; i < pncmesh->GetNGhostFaces(); i++)
+            {
+               int ghost = pncmesh->GetNFaces() + i;
+               ngfdofs += fec->DofForGeometry(pncmesh->GetFaceGeometry(ghost));
+               gfdofs[i+1] = ngfdofs;
+            }
+         }
+         else
+         {
+            ngfdofs = pncmesh->GetNGhostFaces()
+                      * fec->DofForGeometry(pncmesh->GetFaceGeometry(0));
+         }
       }
 
       // total number of ghost DOFs. Ghost DOFs start at index 'ndofs', i.e.,
@@ -856,7 +876,20 @@ const Operator *ParFiniteElementSpace::GetProlongationMatrix() const
 {
    if (Conforming())
    {
-      if (!Pconf) { Pconf = new ConformingProlongationOperator(*this); }
+      if (!Pconf)
+      {
+         if (!Device::Allows(Backend::DEVICE_MASK))
+         {
+            Pconf = new ConformingProlongationOperator(*this);
+         }
+         else
+         {
+            if (NRanks > 1)
+            {
+               Pconf = new DeviceConformingProlongationOperator(*this);
+            }
+         }
+      }
       return Pconf;
    }
    else
@@ -1310,20 +1343,18 @@ void ParFiniteElementSpace::GetGhostEdgeDofs(const MeshId &edge_id,
 void ParFiniteElementSpace::GetGhostFaceDofs(const MeshId &face_id,
                                              Array<int> &dofs) const
 {
-   const int ghost_face_index = face_id.index - pncmesh->GetNFaces();
-   MFEM_ASSERT(pncmesh->GetGhostFaceGeometry(ghost_face_index)
-               == Geometry::SQUARE, "");
+   int nfv, V[4], E[4], Eo[4];
+   nfv = pmesh->pncmesh->GetFaceVerticesEdges(face_id, V, E, Eo);
 
    int nv = fec->DofForGeometry(Geometry::POINT);
    int ne = fec->DofForGeometry(Geometry::SEGMENT);
-   int nf = fec->DofForGeometry(Geometry::SQUARE);
-   dofs.SetSize(4*nv + 4*ne + nf);
+   int nf = fec->DofForGeometry((nfv == 3) ?
+                                Geometry::TRIANGLE : Geometry::SQUARE);
 
-   int V[4], E[4], Eo[4];
-   pmesh->pncmesh->GetFaceVerticesEdges(face_id, V, E, Eo);
+   dofs.SetSize(nfv*(nv + ne) + nf);
 
    int offset = 0;
-   for (int i = 0; i < 4; i++)
+   for (int i = 0; i < nfv; i++)
    {
       int ghost = pncmesh->GetNVertices();
       int first = (V[i] < ghost) ? V[i]*nv : (ndofs + (V[i] - ghost)*nv);
@@ -1333,7 +1364,7 @@ void ParFiniteElementSpace::GetGhostFaceDofs(const MeshId &face_id,
       }
    }
 
-   for (int i = 0; i < 4; i++)
+   for (int i = 0; i < nfv; i++)
    {
       int ghost = pncmesh->GetNEdges();
       int first = (E[i] < ghost) ? nvdofs + E[i]*ne
@@ -1346,8 +1377,10 @@ void ParFiniteElementSpace::GetGhostFaceDofs(const MeshId &face_id,
       }
    }
 
-   // Assuming all ghost faces have the same number of dofs:
-   int first = ndofs + ngvdofs + ngedofs + ghost_face_index*nf;
+   const int ghost_face_index = face_id.index - pncmesh->GetNFaces();
+   int first = ndofs + ngvdofs + ngedofs;
+   first += gfdofs ? gfdofs[ghost_face_index] : nf*ghost_face_index;
+
    for (int j = 0; j < nf; j++)
    {
       dofs[offset++] = first + j;
@@ -1389,12 +1422,19 @@ void ParFiniteElementSpace::GetBareDofs(int entity, int index,
          break;
 
       default:
-         MFEM_ASSERT(!pmesh->HasGeometry(Geometry::TRIANGLE), "");
-         ned = fec->DofForGeometry(Geometry::SQUARE);
+         ned = fec->DofForGeometry(pncmesh->GetFaceGeometry(index));
          ghost = pncmesh->GetNFaces();
-         first = (index < ghost)
-                 ? nvdofs + nedofs + index*ned // regular face
-                 : ndofs + ngvdofs + ngedofs + (index - ghost)*ned; // ghost
+
+         if (index < ghost) // regular face
+         {
+            first = nvdofs + nedofs + (fdofs ? fdofs[index] : index*ned);
+         }
+         else // ghost face
+         {
+            index -= ghost;
+            first = ndofs + ngvdofs + ngedofs +
+                    (gfdofs ? gfdofs[index] : index*ned);
+         }
          break;
    }
 
@@ -1430,16 +1470,30 @@ int ParFiniteElementSpace::PackDof(int entity, int index, int edof) const
                 : ndofs + ngvdofs + (index - ghost)*ned + edof; // ghost edge
 
       default:
-         MFEM_ASSERT(!pmesh->HasGeometry(Geometry::TRIANGLE), "");
          ghost = pncmesh->GetNFaces();
-         ned = fec->DofForGeometry(Geometry::SQUARE);
+         ned = fec->DofForGeometry(pncmesh->GetFaceGeometry(index));
 
-         return (index < ghost)
-                ? nvdofs + nedofs + index*ned + edof // regular face
-                : ndofs + ngvdofs + ngedofs + (index - ghost)*ned + edof; //ghost
+         if (index < ghost) // regular face
+         {
+            return nvdofs + nedofs + (fdofs ? fdofs[index] : index*ned) + edof;
+         }
+         else // ghost face
+         {
+            index -= ghost;
+            return ndofs + ngvdofs + ngedofs +
+                   (gfdofs ? gfdofs[index] : index*ned) + edof;
+         }
    }
 }
 
+static int bisect(int* array, int size, int value)
+{
+   int* end = array + size;
+   int* pos = std::upper_bound(array, end, value);
+   MFEM_VERIFY(pos != end, "value not found");
+   return pos - array;
+}
+
 /** Dissect a DOF number to obtain the entity type (0=vertex, 1=edge, 2=face),
  *  entity index and the DOF number within the entity.
  */
@@ -1465,9 +1519,17 @@ void ParFiniteElementSpace::UnpackDof(int dof,
       dof -= nedofs;
       if (dof < nfdofs) // regular face
       {
-         MFEM_ASSERT(!pmesh->HasGeometry(Geometry::TRIANGLE), "");
-         int nf = fec->DofForGeometry(Geometry::SQUARE);
-         entity = 2, index = dof / nf, edof = dof % nf;
+         if (fdofs) // have mixed faces
+         {
+            index = bisect(fdofs+1, mesh->GetNFaces(), dof);
+            edof = dof - fdofs[index];
+         }
+         else // uniform faces
+         {
+            int nf = fec->DofForGeometry(pncmesh->GetFaceGeometry(0));
+            index = dof / nf, edof = dof % nf;
+         }
+         entity = 2;
          return;
       }
       MFEM_ABORT("Cannot unpack internal DOF");
@@ -1491,8 +1553,17 @@ void ParFiniteElementSpace::UnpackDof(int dof,
       dof -= ngedofs;
       if (dof < ngfdofs) // ghost face
       {
-         int nf = fec->DofForGeometry(pncmesh->GetGhostFaceGeometry(0));
-         entity = 2, index = pncmesh->GetNFaces() + dof / nf, edof = dof % nf;
+         if (gfdofs) // have mixed faces
+         {
+            index = bisect(gfdofs+1, pncmesh->GetNGhostFaces(), dof);
+            edof = dof - gfdofs[index];
+         }
+         else // uniform faces
+         {
+            int nf = fec->DofForGeometry(pncmesh->GetFaceGeometry(0));
+            index = pncmesh->GetNFaces() + dof / nf, edof = dof % nf;
+         }
+         entity = 2;
          return;
       }
       MFEM_ABORT("Out of range DOF.");
@@ -1675,7 +1746,7 @@ void NeighborRowMessage::Encode(int rank)
          mfem::out << "Rank " << pncmesh->MyRank << " sending to " << rank
                    << ": ent " << ri.entity << ", index " << ri.index
                    << ", edof " << ri.edof << " (id " << id.element << "/"
-                   << id.local << ")" << std::endl;
+                   << int(id.local) << ")" << std::endl;
 #endif
 
          // handle orientation and sign change
@@ -1718,8 +1789,6 @@ void NeighborRowMessage::Decode(int rank)
    rows.clear();
    rows.reserve(nrows);
 
-   Geometry::Type fgeom = pncmesh->GetFaceGeometry();
-
    // read rows
    for (int ent = 0, gi = 0; ent < 3; ent++)
    {
@@ -1738,8 +1807,9 @@ void NeighborRowMessage::Decode(int rank)
          }
          else if (ent == 2)
          {
+            Geometry::Type geom = pncmesh->GetFaceGeometry(id.index);
             int fo = pncmesh->GetFaceOrientation(id.index);
-            ind = fec->DofOrderForOrientation(fgeom, fo);
+            ind = fec->DofOrderForOrientation(geom, fo);
          }
 
          double s = 1.0;
@@ -1828,7 +1898,7 @@ void ParFiniteElementSpace
    for (int i = 0; i < dof_group.Size(); i++)
    {
       os << i << ": ";
-      if (i < (nvdofs + nedofs + nfdofs) || i > ndofs)
+      if (i < (nvdofs + nedofs + nfdofs) || i >= ndofs)
       {
          int ent, idx, edof;
          UnpackDof(i, ent, idx, edof);
@@ -1910,15 +1980,7 @@ int ParFiniteElementSpace
          if (!list.masters.size()) { continue; }
 
          IsoparametricTransformation T;
-         if (entity > 1) { T.SetFE(&QuadrilateralFE); }
-         else { T.SetFE(&SegmentFE); }
-
-         Geometry::Type geom = (entity > 1) ?
-                               Geometry::SQUARE : Geometry::SEGMENT;
-         const FiniteElement* fe = fec->FiniteElementForGeometry(geom);
-         if (!fe) { continue; }
-
-         DenseMatrix I(fe->GetDof());
+         DenseMatrix I;
 
          // process masters that we own or that affect our edges/faces
          for (unsigned mi = 0; mi < list.masters.size(); mi++)
@@ -1932,6 +1994,17 @@ int ParFiniteElementSpace
 
             if (!master_dofs.Size()) { continue; }
 
+            const FiniteElement* fe = fec->FiniteElementForGeometry(mf.Geom());
+            if (!fe) { continue; }
+
+            switch (mf.Geom())
+            {
+               case Geometry::SQUARE:   T.SetFE(&QuadrilateralFE); break;
+               case Geometry::TRIANGLE: T.SetFE(&TriangleFE); break;
+               case Geometry::SEGMENT:  T.SetFE(&SegmentFE); break;
+               default: MFEM_ABORT("unsupported geometry");
+            }
+
             // constrain slaves that exist in our mesh
             for (int si = mf.slaves_begin; si < mf.slaves_end; si++)
             {
@@ -1982,6 +2055,8 @@ int ParFiniteElementSpace
                   (l == 1) ? (const MeshId&) list.masters[i]
                   /*    */ : (const MeshId&) list.slaves[i];
 
+               if (id.index < 0) { continue; }
+
                GroupId owner = pncmesh->GetEntityOwnerId(entity, id.index);
                GroupId group = pncmesh->GetEntityGroupId(entity, id.index);
 
@@ -2478,6 +2553,9 @@ ParFiniteElementSpace::ParallelDerefinementMatrix(int old_ndofs,
    int nrk = HYPRE_AssumedPartitionCheck() ? 2 : NRanks;
 
    MFEM_VERIFY(Nonconforming(), "Not implemented for conforming meshes.");
+   MFEM_VERIFY(pmesh->GetNumGeometries(pmesh->Dimension()) == 1,
+               "Not implemented for mixed meshes.");
+
    MFEM_VERIFY(old_dof_offsets[nrk], "Missing previous (finer) space.");
    MFEM_VERIFY(dof_offsets[nrk] <= old_dof_offsets[nrk],
                "Previous space is not finer.");
@@ -2491,7 +2569,7 @@ ParFiniteElementSpace::ParallelDerefinementMatrix(int old_ndofs,
    Vector row;
 
    ParNCMesh* pncmesh = pmesh->pncmesh;
-   Geometry::Type geom = pncmesh->GetElementGeometry();
+   Geometry::Type geom = pncmesh->GetElementGeometry(0); // TODO mixed meshes
    int ldof = fec->FiniteElementForGeometry(geom)->GetDof();
 
    const CoarseFineTransformations &dtrans = pncmesh->GetDerefinementTransforms();
@@ -2715,6 +2793,8 @@ void ParFiniteElementSpace::Destroy()
    delete Pconf; Pconf = NULL;
    delete R; R = NULL;
 
+   delete [] gfdofs; gfdofs = NULL;
+
    delete gcomm; gcomm = NULL;
 
    num_face_nbr_dofs = -1;
@@ -2932,6 +3012,242 @@ void ConformingProlongationOperator::MultTranspose(
    gc.ReduceEnd<double>(ydata, out_layout, GroupCommunicator::Sum);
 }
 
+DeviceConformingProlongationOperator::DeviceConformingProlongationOperator(
+   const ParFiniteElementSpace &pfes) :
+   ConformingProlongationOperator(pfes),
+   mpi_gpu_aware(Device::GetGPUAwareMPI())
+{
+   MFEM_ASSERT(pfes.Conforming(), "internal error");
+   const SparseMatrix *R = pfes.GetRestrictionMatrix();
+   MFEM_ASSERT(R->Finalized(), "");
+   const int tdofs = R->Height();
+   MFEM_ASSERT(tdofs == pfes.GetTrueVSize(), "");
+   MFEM_ASSERT(tdofs == R->GetI()[tdofs], "");
+   ltdof_ldof = Array<int>(const_cast<int*>(R->GetJ()), tdofs);
+   ltdof_ldof.UseDevice();
+   {
+      Table nbr_ltdof;
+      gc.GetNeighborLTDofTable(nbr_ltdof);
+      const int nb_connections = nbr_ltdof.Size_of_connections();
+      shr_ltdof.SetSize(nb_connections);
+      shr_ltdof.CopyFrom(nbr_ltdof.GetJ());
+      shr_buf.SetSize(nb_connections);
+      shr_buf.UseDevice(true);
+      shr_buf_offsets = nbr_ltdof.GetI();
+      {
+         Array<int> shr_ltdof(nbr_ltdof.GetJ(), nb_connections);
+         Array<int> unique_ltdof(shr_ltdof);
+         unique_ltdof.Sort();
+         unique_ltdof.Unique();
+         // Note: the next loop modifies the J array of nbr_ltdof
+         for (int i = 0; i < shr_ltdof.Size(); i++)
+         {
+            shr_ltdof[i] = unique_ltdof.FindSorted(shr_ltdof[i]);
+            MFEM_ASSERT(shr_ltdof[i] != -1, "internal error");
+         }
+         Table unique_shr;
+         Transpose(shr_ltdof, unique_shr, unique_ltdof.Size());
+         unq_ltdof = Array<int>(unique_ltdof, unique_ltdof.Size());
+         unq_shr_i = Array<int>(unique_shr.GetI(), unique_shr.Size()+1);
+         unq_shr_j = Array<int>(unique_shr.GetJ(), unique_shr.Size_of_connections());
+      }
+      delete [] nbr_ltdof.GetJ();
+      nbr_ltdof.LoseData();
+   }
+   {
+      Table nbr_ldof;
+      gc.GetNeighborLDofTable(nbr_ldof);
+      const int nb_connections = nbr_ldof.Size_of_connections();
+      ext_ldof.SetSize(nb_connections);
+      ext_ldof.CopyFrom(nbr_ldof.GetJ());
+      ext_buf.SetSize(nb_connections);
+      ext_buf.UseDevice(true);
+      ext_buf_offsets = nbr_ldof.GetI();
+      delete [] nbr_ldof.GetJ();
+      nbr_ldof.LoseData();
+   }
+   const GroupTopology &gtopo = gc.GetGroupTopology();
+   int req_counter = 0;
+   for (int nbr = 1; nbr < gtopo.GetNumNeighbors(); nbr++)
+   {
+      const int send_offset = shr_buf_offsets[nbr];
+      const int send_size = shr_buf_offsets[nbr+1] - send_offset;
+      if (send_size > 0) { req_counter++; }
+
+      const int recv_offset = ext_buf_offsets[nbr];
+      const int recv_size = ext_buf_offsets[nbr+1] - recv_offset;
+      if (recv_size > 0) { req_counter++; }
+   }
+   requests = new MPI_Request[req_counter];
+}
+
+static void ExtractSubVector(const int N,
+                             const Array<int> &indices,
+                             const Vector &in, Vector &out)
+{
+   auto y = out.Write();
+   const auto x = in.Read();
+   const auto I = indices.Read();
+   MFEM_FORALL(i, N, y[i] = x[I[i]];); // indices can be repeated
+}
+
+void DeviceConformingProlongationOperator::BcastBeginCopy(
+   const Vector &x) const
+{
+   // shr_buf[i] = src[shr_ltdof[i]]
+   if (shr_ltdof.Size() == 0) { return; }
+   ExtractSubVector(shr_ltdof.Size(), shr_ltdof, x, shr_buf);
+   // If the above kernel is executed asynchronously, we should wait for it to
+   // complete
+   if (mpi_gpu_aware) { Device::Synchronize(); }
+}
+
+static void SetSubVector(const int N,
+                         const Array<int> &indices,
+                         const Vector &in, Vector &out)
+{
+   auto y = out.Write();
+   const auto x = in.Read();
+   const auto I = indices.Read();
+   MFEM_FORALL(i, N, y[I[i]] = x[i];);
+}
+
+void DeviceConformingProlongationOperator::BcastLocalCopy(
+   const Vector &x, Vector &y) const
+{
+   // dst[ltdof_ldof[i]] = src[i]
+   if (ltdof_ldof.Size() == 0) { return; }
+   SetSubVector(ltdof_ldof.Size(), ltdof_ldof, x, y);
+}
+
+void DeviceConformingProlongationOperator::BcastEndCopy(
+   Vector &y) const
+{
+   // dst[ext_ldof[i]] = ext_buf[i]
+   if (ext_ldof.Size() == 0) { return; }
+   SetSubVector(ext_ldof.Size(), ext_ldof, ext_buf, y);
+}
+
+void DeviceConformingProlongationOperator::Mult(const Vector &x,
+                                                Vector &y) const
+{
+   const GroupTopology &gtopo = gc.GetGroupTopology();
+   BcastBeginCopy(x); // copy to 'shr_buf'
+   int req_counter = 0;
+   for (int nbr = 1; nbr < gtopo.GetNumNeighbors(); nbr++)
+   {
+      const int send_offset = shr_buf_offsets[nbr];
+      const int send_size = shr_buf_offsets[nbr+1] - send_offset;
+      if (send_size > 0)
+      {
+         auto send_buf = mpi_gpu_aware ? shr_buf.Read() : shr_buf.HostRead();
+         MPI_Isend(send_buf + send_offset, send_size, MPI_DOUBLE,
+                   gtopo.GetNeighborRank(nbr), 41822,
+                   gtopo.GetComm(), &requests[req_counter++]);
+      }
+      const int recv_offset = ext_buf_offsets[nbr];
+      const int recv_size = ext_buf_offsets[nbr+1] - recv_offset;
+      if (recv_size > 0)
+      {
+         auto recv_buf = mpi_gpu_aware ? ext_buf.Write() : ext_buf.HostWrite();
+         MPI_Irecv(recv_buf + recv_offset, recv_size, MPI_DOUBLE,
+                   gtopo.GetNeighborRank(nbr), 41822,
+                   gtopo.GetComm(), &requests[req_counter++]);
+      }
+   }
+   BcastLocalCopy(x, y);
+   MPI_Waitall(req_counter, requests, MPI_STATUSES_IGNORE);
+   BcastEndCopy(y); // copy from 'ext_buf'
+}
+
+DeviceConformingProlongationOperator::~DeviceConformingProlongationOperator()
+{
+   delete [] requests;
+   delete [] ext_buf_offsets;
+   delete [] shr_buf_offsets;
+}
+
+void DeviceConformingProlongationOperator::ReduceBeginCopy(
+   const Vector &x) const
+{
+   // ext_buf[i] = src[ext_ldof[i]]
+   if (ext_ldof.Size() == 0) { return; }
+   ExtractSubVector(ext_ldof.Size(), ext_ldof, x, ext_buf);
+   // If the above kernel is executed asynchronously, we should wait for it to
+   // complete
+   if (mpi_gpu_aware) { Device::Synchronize(); }
+}
+
+void DeviceConformingProlongationOperator::ReduceLocalCopy(
+   const Vector &x, Vector &y) const
+{
+   // dst[i] = src[ltdof_ldof[i]]
+   if (ltdof_ldof.Size() == 0) { return; }
+   ExtractSubVector(ltdof_ldof.Size(), ltdof_ldof, x, y);
+}
+
+static void AddSubVector(const int num_unique_dst_indices,
+                         const Array<int> &unique_dst_indices,
+                         const Array<int> &unique_to_src_offsets,
+                         const Array<int> &unique_to_src_indices,
+                         const Vector &src,
+                         Vector &dst)
+{
+   auto y = dst.Write();
+   const auto x = src.Read();
+   const auto DST_I = unique_dst_indices.Read();
+   const auto SRC_O = unique_to_src_offsets.Read();
+   const auto SRC_I = unique_to_src_indices.Read();
+   MFEM_FORALL(i, num_unique_dst_indices,
+   {
+      const int dst_idx = DST_I[i];
+      double sum = y[dst_idx];
+      const int end = SRC_O[i+1];
+      for (int j = SRC_O[i]; j != end; ++j) { sum += x[SRC_I[j]]; }
+      y[dst_idx] = sum;
+   });
+}
+
+void DeviceConformingProlongationOperator::ReduceEndAssemble(Vector &y) const
+{
+   // dst[shr_ltdof[i]] += shr_buf[i]
+   const int unq_ltdof_size = unq_ltdof.Size();
+   if (unq_ltdof_size == 0) { return; }
+   AddSubVector(unq_ltdof_size, unq_ltdof, unq_shr_i, unq_shr_j, shr_buf, y);
+}
+
+void DeviceConformingProlongationOperator::MultTranspose(const Vector &x,
+                                                         Vector &y) const
+{
+   const GroupTopology &gtopo = gc.GetGroupTopology();
+   ReduceBeginCopy(x); // copy to 'ext_buf'
+   int req_counter = 0;
+   for (int nbr = 1; nbr < gtopo.GetNumNeighbors(); nbr++)
+   {
+      const int send_offset = ext_buf_offsets[nbr];
+      const int send_size = ext_buf_offsets[nbr+1] - send_offset;
+      if (send_size > 0)
+      {
+         auto send_buf = mpi_gpu_aware ? ext_buf.Read() : ext_buf.HostRead();
+         MPI_Isend(send_buf + send_offset, send_size, MPI_DOUBLE,
+                   gtopo.GetNeighborRank(nbr), 41823,
+                   gtopo.GetComm(), &requests[req_counter++]);
+      }
+      const int recv_offset = shr_buf_offsets[nbr];
+      const int recv_size = shr_buf_offsets[nbr+1] - recv_offset;
+      if (recv_size > 0)
+      {
+         auto recv_buf = mpi_gpu_aware ? shr_buf.Write() : shr_buf.HostWrite();
+         MPI_Irecv(recv_buf + recv_offset, recv_size, MPI_DOUBLE,
+                   gtopo.GetNeighborRank(nbr), 41823,
+                   gtopo.GetComm(), &requests[req_counter++]);
+      }
+   }
+   ReduceLocalCopy(x, y);
+   MPI_Waitall(req_counter, requests, MPI_STATUSES_IGNORE);
+   ReduceEndAssemble(y); // assemble from 'shr_buf'
+}
+
 } // namespace mfem
 
 #endif

fem/pfespace.hpp

@@ -46,6 +46,7 @@ private:
 
    /// Number of vertex/edge/face/total ghost DOFs (nonconforming case).
    int ngvdofs, ngedofs, ngfdofs, ngdofs;
+   int* gfdofs;
 
    /// The group of each local dof.
    Array<int> ldof_group;
@@ -113,7 +114,7 @@ private:
    void GetGhostFaceDofs(const MeshId &face_id, Array<int> &dofs) const;
 
    void GetGhostDofs(int entity, const MeshId &id, Array<int> &dofs) const;
-   // Return the dofs associated with the interior of the given mesh entity.
+   /// Return the dofs associated with the interior of the given mesh entity.
    void GetBareDofs(int entity, int index, Array<int> &dofs) const;
 
    int  PackDof(int entity, int index, int edof) const;
@@ -387,6 +388,52 @@ public:
    virtual void MultTranspose(const Vector &x, Vector &y) const;
 };
 
+/// Auxiliary device class used by ParFiniteElementSpace.
+class DeviceConformingProlongationOperator: public
+   ConformingProlongationOperator
+{
+protected:
+   bool mpi_gpu_aware;
+   Array<int> shr_ltdof, ext_ldof;
+   mutable Vector shr_buf, ext_buf;
+   int *shr_buf_offsets, *ext_buf_offsets;
+   Array<int> ltdof_ldof, unq_ltdof;
+   Array<int> unq_shr_i, unq_shr_j;
+   MPI_Request *requests;
+   // Kernel: copy ltdofs from 'src' to 'shr_buf' - prepare for send.
+   //         shr_buf[i] = src[shr_ltdof[i]]
+   void BcastBeginCopy(const Vector &src) const;
+
+   // Kernel: copy ltdofs from 'src' to ldofs in 'dst'.
+   //         dst[ltdof_ldof[i]] = src[i]
+   void BcastLocalCopy(const Vector &src, Vector &dst) const;
+
+   // Kernel: copy ext. dofs from 'ext_buf' to 'dst' - after recv.
+   //         dst[ext_ldof[i]] = ext_buf[i]
+   void BcastEndCopy(Vector &dst) const;
+
+   // Kernel: copy ext. dofs from 'src' to 'ext_buf' - prepare for send.
+   //         ext_buf[i] = src[ext_ldof[i]]
+   void ReduceBeginCopy(const Vector &src) const;
+
+   // Kernel: copy owned ldofs from 'src' to ltdofs in 'dst'.
+   //         dst[i] = src[ltdof_ldof[i]]
+   void ReduceLocalCopy(const Vector &src, Vector &dst) const;
+
+   // Kernel: assemble dofs from 'shr_buf' into to 'dst' - after recv.
+   //         dst[shr_ltdof[i]] += shr_buf[i]
+   void ReduceEndAssemble(Vector &dst) const;
+
+public:
+   DeviceConformingProlongationOperator(const ParFiniteElementSpace &pfes);
+
+   virtual ~DeviceConformingProlongationOperator();
+
+   virtual void Mult(const Vector &x, Vector &y) const;
+
+   virtual void MultTranspose(const Vector &x, Vector &y) const;
+};
+
 }
 
 #endif // MFEM_USE_MPI

fem/pgridfunc.cpp

@@ -225,11 +225,13 @@ void ParGridFunction::ExchangeFaceNbrData()
    MPI_Request *recv_requests = requests + num_face_nbrs;
    MPI_Status  *statuses = new MPI_Status[num_face_nbrs];
 
+   const double *h_data = this->HostRead();
    for (int i = 0; i < send_data.Size(); i++)
    {
-      send_data[i] = data[send_ldof[i]];
+      send_data[i] = h_data[send_ldof[i]];
    }
 
+   double *h_face_nbr_data = face_nbr_data.HostWrite();
    for (int fn = 0; fn < num_face_nbrs; fn++)
    {
       int nbr_rank = pmesh->GetFaceNbrRank(fn);
@@ -239,7 +241,7 @@ void ParGridFunction::ExchangeFaceNbrData()
                 send_offset[fn+1] - send_offset[fn],
                 MPI_DOUBLE, nbr_rank, tag, MyComm, &send_requests[fn]);
 
-      MPI_Irecv(&face_nbr_data(recv_offset[fn]),
+      MPI_Irecv(&h_face_nbr_data[recv_offset[fn]],
                 recv_offset[fn+1] - recv_offset[fn],
                 MPI_DOUBLE, nbr_rank, tag, MyComm, &recv_requests[fn]);
    }

fem/pgridfunc.hpp

@@ -112,6 +112,8 @@ public:
    /// Associate a new parallel space with the ParGridFunction.
    void SetSpace(ParFiniteElementSpace *f);
 
+   using GridFunction::MakeRef;
+
    /** @brief Make the ParGridFunction reference external data on a new
        FiniteElementSpace. */
    /** This method changes the FiniteElementSpace associated with the

fem/pnonlinearform.cpp

@@ -46,7 +46,7 @@ double ParNonlinearForm::GetParGridFunctionEnergy(const Vector &x) const
 void ParNonlinearForm::Mult(const Vector &x, Vector &y) const
 {
    NonlinearForm::Mult(x, y); // x --(P)--> aux1 --(A_local)--> aux2
-   Y.SetData(aux2.GetData()); // aux2 contains A_local.P.x
+   Y.MakeRef(aux2, 0); // aux2 contains A_local.P.x
 
    if (fnfi.Size())
    {
@@ -58,7 +58,7 @@ void ParNonlinearForm::Mult(const Vector &x, Vector &y) const
       Array<int> vdofs1, vdofs2;
       Vector el_x, el_y;
 
-      X.SetData(aux1.GetData()); // aux1 contains P.x
+      X.MakeRef(aux1, 0); // aux1 contains P.x
       X.ExchangeFaceNbrData();
       const int n_shared_faces = pmesh->GetNSharedFaces();
       for (int i = 0; i < n_shared_faces; i++)

fem/sidredatacollection.cpp

@@ -16,9 +16,7 @@
 
 #include "fem.hpp"
 
-#ifdef MFEM_USE_MPI
-#include <sidre/IOManager.hpp>
-#endif
+#include <axom/sidre.hpp>
 
 #include <string>
 #include <iomanip>      // for setw, setfill
@@ -204,10 +202,10 @@ SidreDataCollection::get_file_path(const std::string &filename) const
 
 axom::sidre::View *
 SidreDataCollection::AllocNamedBuffer(const std::string& buffer_name,
-                                      axom::sidre::SidreLength sz,
+                                      axom::sidre::IndexType sz,
                                       axom::sidre::TypeID type)
 {
-   sz = std::max(sz, sidre::SidreLength(0));
+   sz = std::max(sz, sidre::IndexType(0));
    sidre::Group *f = named_buffers_grp();
    sidre::View  *v = NULL;
 
@@ -825,7 +823,7 @@ void SidreDataCollection::Save(const std::string& filename,
 void SidreDataCollection::
 addScalarBasedGridFunction(const std::string &field_name, GridFunction *gf,
                            const std::string &buffer_name,
-                           axom::sidre::SidreLength offset)
+                           axom::sidre::IndexType offset)
 {
    sidre::Group* grp = m_bp_grp->getGroup("fields/" + field_name);
    MFEM_ASSERT(grp != NULL, "field " << field_name << " does not exist");
@@ -888,7 +886,7 @@ addScalarBasedGridFunction(const std::string &field_name, GridFunction *gf,
 void SidreDataCollection::
 addVectorBasedGridFunction(const std::string& field_name, GridFunction *gf,
                            const std::string &buffer_name,
-                           axom::sidre::SidreLength offset)
+                           axom::sidre::IndexType offset)
 {
    sidre::Group* grp = m_bp_grp->getGroup("fields/" + field_name);
    MFEM_ASSERT(grp != NULL, "field " << field_name << " does not exist");
@@ -1013,7 +1011,7 @@ DeregisterFieldInBPIndex(const std::string& field_name)
 void SidreDataCollection::RegisterField(const std::string &field_name,
                                         GridFunction *gf,
                                         const std::string &buffer_name,
-                                        axom::sidre::SidreLength offset)
+                                        axom::sidre::IndexType offset)
 {
    if ( field_name.empty() || buffer_name.empty() ||
         gf == NULL || gf->FESpace() == NULL )

fem/sidredatacollection.hpp

@@ -25,7 +25,7 @@
 #  pragma GCC diagnostic ignored "-Wpedantic"
 # endif
 #endif
-#include <sidre/sidre.hpp>
+#include <axom/sidre.hpp>
 #ifdef MFEM_HAVE_GCC_PRAGMA_DIAGNOSTIC
 # pragma GCC diagnostic pop
 #endif
@@ -246,7 +246,7 @@ public:
     */
    void RegisterField(const std::string &field_name, GridFunction *gf,
                       const std::string &buffer_name,
-                      axom::sidre::SidreLength offset);
+                      axom::sidre::IndexType offset);
 
    /// Registers an attribute field in the Sidre DataStore
    /** The registration process is similar to that of RegisterField()
@@ -385,7 +385,7 @@ public:
     */
    axom::sidre::View *
    AllocNamedBuffer(const std::string& buffer_name,
-                    axom::sidre::SidreLength sz,
+                    axom::sidre::IndexType sz,
                     axom::sidre::TypeID type =
                        axom::sidre::DOUBLE_ID);
 
@@ -469,7 +469,7 @@ private:
    void addScalarBasedGridFunction(const std::string& field_name,
                                    GridFunction* gf,
                                    const std::string &buffer_name,
-                                   axom::sidre::SidreLength offset);
+                                   axom::sidre::IndexType offset);
 
    /**
     * \brief A private helper function to set up the views associated with the
@@ -483,7 +483,7 @@ private:
    void addVectorBasedGridFunction(const std::string& field_name,
                                    GridFunction* gf,
                                    const std::string &buffer_name,
-                                   axom::sidre::SidreLength offset);
+                                   axom::sidre::IndexType offset);
 
    /** @brief A private helper function to set up the Views associated with
        attribute field named @a field_name */

fem/tmop.cpp

@@ -12,6 +12,7 @@
 #include "tmop.hpp"
 #include "linearform.hpp"
 #include "pgridfunc.hpp"
+#include "tmop_tools.hpp"
 
 namespace mfem
 {
@@ -768,7 +769,7 @@ void TMOP_Metric_352::AssembleH(const DenseMatrix &Jpt,
 void TargetConstructor::ComputeAvgVolume() const
 {
    MFEM_VERIFY(nodes, "Nodes are not given!");
-   MFEM_ASSERT(avg_volume == 0.0, "the average volume is already computed!");
+   MFEM_ASSERT(avg_volume == 0.0, "The average volume is already computed!");
 
    Mesh *mesh = nodes->FESpace()->GetMesh();
    const int NE = mesh->GetNE();
@@ -787,9 +788,13 @@ void TargetConstructor::ComputeAvgVolume() const
          volume += ip.weight * Tr.Weight();
       }
    }
-   if (!Parallel())
+
+   NCMesh *ncmesh = mesh->ncmesh;
+   if (Parallel() == false)
    {
-      avg_volume = volume / NE;
+      avg_volume = (ncmesh == NULL) ?
+                   volume / NE : volume / ncmesh->GetNumRootElements();
+
    }
 #ifdef MFEM_USE_MPI
    else
@@ -797,7 +802,8 @@ void TargetConstructor::ComputeAvgVolume() const
       double area_NE[4];
       area_NE[0] = volume; area_NE[1] = NE;
       MPI_Allreduce(area_NE, area_NE + 2, 2, MPI_DOUBLE, MPI_SUM, comm);
-      avg_volume = area_NE[2] / area_NE[3];
+      avg_volume = (ncmesh == NULL) ?
+                   area_NE[2] / area_NE[3] : area_NE[2] / ncmesh->GetNumRootElements();
    }
 #endif
 }
@@ -805,6 +811,7 @@ void TargetConstructor::ComputeAvgVolume() const
 // virtual method
 void TargetConstructor::ComputeElementTargets(int e_id, const FiniteElement &fe,
                                               const IntegrationRule &ir,
+                                              const Vector &elfun,
                                               DenseTensor &Jtr) const
 {
    MFEM_ASSERT(target_type == IDEAL_SHAPE_UNIT_SIZE || nodes != NULL, "");
@@ -827,7 +834,15 @@ void TargetConstructor::ComputeElementTargets(int e_id, const FiniteElement &fe,
       {
          if (avg_volume == 0.0) { ComputeAvgVolume(); }
          DenseMatrix W(Wideal.Height());
-         W.Set(std::pow(volume_scale * avg_volume / Wideal.Det(),
+
+         NCMesh *ncmesh = nodes->FESpace()->GetMesh()->ncmesh;
+         double el_volume = avg_volume;
+         if (ncmesh)
+         {
+            el_volume = avg_volume / ncmesh->GetElementSizeReduction(e_id);
+         }
+
+         W.Set(std::pow(volume_scale * el_volume / Wideal.Det(),
                         1./W.Height()), Wideal);
          for (int i = 0; i < ir.GetNPoints(); i++) { Jtr(i) = W; }
          break;
@@ -853,7 +868,7 @@ void TargetConstructor::ComputeElementTargets(int e_id, const FiniteElement &fe,
             if (target_type == IDEAL_SHAPE_GIVEN_SIZE)
             {
                const double det = Jtr(i).Det();
-               MFEM_VERIFY(det > 0.0, "Initial mesh is inverted!");
+               MFEM_VERIFY(det > 0.0, "The given mesh is inverted!");
                Jtr(i).Set(std::pow(det / detW, 1./dim), Wideal);
             }
          }
@@ -864,6 +879,162 @@ void TargetConstructor::ComputeElementTargets(int e_id, const FiniteElement &fe,
    }
 }
 
+void AnalyticAdaptTC::SetAnalyticTargetSpec(Coefficient *sspec,
+                                            VectorCoefficient *vspec,
+                                            MatrixCoefficient *mspec)
+{
+   scalar_tspec = sspec;
+   vector_tspec = vspec;
+   matrix_tspec = mspec;
+}
+
+void AnalyticAdaptTC::ComputeElementTargets(int e_id, const FiniteElement &fe,
+                                            const IntegrationRule &ir,
+                                            const Vector &elfun,
+                                            DenseTensor &Jtr) const
+{
+   DenseMatrix point_mat;
+   point_mat.UseExternalData(elfun.GetData(), fe.GetDof(), fe.GetDim());
+
+   switch (target_type)
+   {
+      case GIVEN_FULL:
+      {
+         MFEM_VERIFY(matrix_tspec != NULL,
+                     "Target type GIVEN_FULL requires a MatrixCoefficient.");
+
+         IsoparametricTransformation Tpr;
+         Tpr.SetFE(&fe);
+         Tpr.ElementNo = e_id;
+         Tpr.GetPointMat().Transpose(point_mat);
+
+         for (int i = 0; i < ir.GetNPoints(); i++)
+         {
+            const IntegrationPoint &ip = ir.IntPoint(i);
+            Tpr.SetIntPoint(&ip);
+            matrix_tspec->Eval(Jtr(i), Tpr, ip);
+         }
+         break;
+      }
+      default:
+         MFEM_ABORT("Incompatible target type for analytic adaptation!");
+   }
+}
+
+#ifdef MFEM_USE_MPI
+void DiscreteAdaptTC::SetParDiscreteTargetSpec(ParGridFunction &tspec)
+{
+   target_spec.SetSize(tspec.Size());
+   target_spec = tspec;
+   tspec_fes   = tspec.FESpace();
+
+   // Default evaluator is based on CG advection.
+   if (adapt_eval == NULL) { adapt_eval = new AdvectorCG; }
+
+   adapt_eval->SetParMetaInfo(*tspec.ParFESpace()->GetParMesh(),
+                              *tspec.FESpace()->FEColl(),
+                              tspec.FESpace()->GetVDim());
+
+   adapt_eval->SetInitialField
+   (*tspec.FESpace()->GetMesh()->GetNodes(), target_spec);
+}
+#endif
+
+void DiscreteAdaptTC::SetSerialDiscreteTargetSpec(GridFunction &tspec)
+{
+   target_spec.SetSize(tspec.Size());
+   target_spec = tspec;
+   tspec_fes   = tspec.FESpace();
+
+   // Default evaluator is based on CG advection.
+   if (adapt_eval == NULL) { adapt_eval = new AdvectorCG; }
+
+   adapt_eval->SetSerialMetaInfo(*tspec.FESpace()->GetMesh(),
+                                 *tspec.FESpace()->FEColl(),
+                                 tspec.FESpace()->GetVDim());
+
+   adapt_eval->SetInitialField
+   (*tspec.FESpace()->GetMesh()->GetNodes(), target_spec);
+}
+
+void DiscreteAdaptTC::UpdateTargetSpecification(const Vector &new_x)
+{
+   MFEM_VERIFY(target_spec.Size() > 0, "Target specification is not set!");
+
+   adapt_eval->ComputeAtNewPosition(new_x, target_spec);
+}
+
+void DiscreteAdaptTC::ComputeElementTargets(int e_id, const FiniteElement &fe,
+                                            const IntegrationRule &ir,
+                                            const Vector &elfun,
+                                            DenseTensor &Jtr) const
+{
+   MFEM_VERIFY(tspec_fes, "A call to SetDiscreteTargerSpec() is needed.");
+
+   switch (target_type)
+   {
+      case IDEAL_SHAPE_GIVEN_SIZE:
+      {
+         const DenseMatrix &Wideal =
+            Geometries.GetGeomToPerfGeomJac(fe.GetGeomType());
+         const int dim = Wideal.Height(),
+                   ntspec_dofs = tspec_fes->GetFE(0)->GetDof();
+
+         Vector shape(ntspec_dofs), tspec_vals(ntspec_dofs);
+         Array<int> dofs;
+         tspec_fes->GetElementDofs(e_id, dofs);
+         target_spec.GetSubVector(dofs, tspec_vals);
+
+         const double min_size = tspec_vals.Min();
+         MFEM_ASSERT(min_size > 0.0,
+                     "Non-positive size propagated in the target definition.");
+
+         for (int i = 0; i < ir.GetNPoints(); i++)
+         {
+            const IntegrationPoint &ip = ir.IntPoint(i);
+            tspec_fes->GetFE(e_id)->CalcShape(ip, shape);
+            const double size = std::max(shape * tspec_vals, min_size);
+            Jtr(i).Set(std::pow(size / Wideal.Det(), 1.0/dim), Wideal);
+         }
+         break;
+      }
+      default:
+         MFEM_ABORT("Incompatible target type for analytic adaptation!");
+   }
+}
+
+void AdaptivityEvaluator::SetSerialMetaInfo(const Mesh &m,
+                                            const FiniteElementCollection &fec,
+                                            int num_comp)
+{
+   delete fes;
+   delete mesh;
+   mesh = new Mesh(m, true);
+   fes = new FiniteElementSpace(mesh, &fec, num_comp);
+}
+
+#ifdef MFEM_USE_MPI
+void AdaptivityEvaluator::SetParMetaInfo(const ParMesh &m,
+                                         const FiniteElementCollection &fec,
+                                         int num_comp)
+{
+   delete pfes;
+   delete pmesh;
+   pmesh = new ParMesh(m, true);
+   pfes  = new ParFiniteElementSpace(pmesh, &fec, num_comp);
+}
+#endif
+
+AdaptivityEvaluator::~AdaptivityEvaluator()
+{
+   delete fes;
+   delete mesh;
+#ifdef MFEM_USE_MPI
+   delete pfes;
+   delete pmesh;
+#endif
+}
+
 void TMOP_Integrator::EnableLimiting(const GridFunction &n0,
                                      const GridFunction &dist, Coefficient &w0,
                                      TMOP_LimiterFunction *lfunc)
@@ -921,7 +1092,7 @@ double TMOP_Integrator::GetElementEnergy(const FiniteElement &el,
 
    energy = 0.0;
    DenseTensor Jtr(dim, dim, ir->GetNPoints());
-   targetC->ComputeElementTargets(T.ElementNo, el, *ir, Jtr);
+   targetC->ComputeElementTargets(T.ElementNo, el, *ir, elfun, Jtr);
 
    // Limited case.
    Vector shape, p, p0, d_vals;
@@ -990,6 +1161,7 @@ double TMOP_Integrator::GetElementEnergy(const FiniteElement &el,
       energy += weight * val;
    }
    delete Tpr;
+
    return energy;
 }
 
@@ -1016,7 +1188,7 @@ void TMOP_Integrator::AssembleElementVector(const FiniteElement &el,
 
    elvect = 0.0;
    DenseTensor Jtr(dim, dim, ir->GetNPoints());
-   targetC->ComputeElementTargets(T.ElementNo, el, *ir, Jtr);
+   targetC->ComputeElementTargets(T.ElementNo, el, *ir, elfun, Jtr);
 
    // Limited case.
    DenseMatrix pos0;
@@ -1072,6 +1244,8 @@ void TMOP_Integrator::AssembleElementVector(const FiniteElement &el,
       P *= weight_m;
       AddMultABt(DS, P, PMatO);
 
+      // TODO: derivatives of adaptivity-based targets.
+
       if (coeff0)
       {
          el.CalcShape(ip, shape);
@@ -1107,7 +1281,7 @@ void TMOP_Integrator::AssembleElementGrad(const FiniteElement &el,
 
    elmat = 0.0;
    DenseTensor Jtr(dim, dim, ir->GetNPoints());
-   targetC->ComputeElementTargets(T.ElementNo, el, *ir, Jtr);
+   targetC->ComputeElementTargets(T.ElementNo, el, *ir, elfun, Jtr);
 
    // Limited case.
    DenseMatrix pos0, grad_grad;
@@ -1160,6 +1334,8 @@ void TMOP_Integrator::AssembleElementGrad(const FiniteElement &el,
 
       metric->AssembleH(Jpt, DS, weight_m, elmat);
 
+      // TODO: derivatives of adaptivity-based targets.
+
       if (coeff0)
       {
          el.CalcShape(ip, shape);
@@ -1234,11 +1410,12 @@ void TMOP_Integrator::ComputeNormalizationEnergies(const GridFunction &x,
    for (int i = 0; i < fes->GetNE(); i++)
    {
       fe = fes->GetFE(i);
-      targetC->ComputeElementTargets(i, *fe, *ir, Jtr);
       fes->GetElementVDofs(i, vdofs);
       x.GetSubVector(vdofs, x_vals);
       PMatI.UseExternalData(x_vals.GetData(), dof, dim);
 
+      targetC->ComputeElementTargets(i, *fe, *ir, x_vals, Jtr);
+
       for (int i = 0; i < ir->GetNPoints(); i++)
       {
          const IntegrationPoint &ip = ir->IntPoint(i);
@@ -1274,9 +1451,6 @@ void InterpolateTMOP_QualityMetric(TMOP_QualityMetric &metric,
       const IntegrationRule &ir = metric_gf.FESpace()->GetFE(i)->GetNodes();
       const int nsp = ir.GetNPoints(), dof = fe_pos.GetDof();
 
-      W.SetSize(dim, dim, nsp);
-      tc.ComputeElementTargets(i, fe_pos, ir, W);
-
       dshape.SetSize(dof, dim);
       pos.SetSize(dof, dim);
       posV.SetDataAndSize(pos.Data(), dof * dim);
@@ -1285,6 +1459,9 @@ void InterpolateTMOP_QualityMetric(TMOP_QualityMetric &metric,
       nodes.FESpace()->GetElementVDofs(i, pos_dofs);
       nodes.GetSubVector(pos_dofs, posV);
 
+      W.SetSize(dim, dim, nsp);
+      tc.ComputeElementTargets(i, fe_pos, ir, posV, W);
+
       for (int j = 0; j < nsp; j++)
       {
          const DenseMatrix &Wj = W(j);

fem/tmop.hpp

@@ -12,7 +12,6 @@
 #ifndef MFEM_TMOP_HPP
 #define MFEM_TMOP_HPP
 
-#include "../config/config.hpp"
 #include "../linalg/invariants.hpp"
 #include "nonlininteg.hpp"
 
@@ -514,6 +513,51 @@ public:
    virtual ~TMOP_QuadraticLimiter() { }
 };
 
+class FiniteElementCollection;
+class FiniteElementSpace;
+class ParFiniteElementSpace;
+
+class AdaptivityEvaluator
+{
+protected:
+   // Owned.
+   Mesh *mesh;
+   FiniteElementSpace *fes;
+
+#ifdef MFEM_USE_MPI
+   // Owned.
+   ParMesh *pmesh;
+   ParFiniteElementSpace *pfes;
+#endif
+
+public:
+   AdaptivityEvaluator() : mesh(NULL), fes(NULL)
+   {
+#ifdef MFEM_USE_MPI
+      pmesh = NULL;
+      pfes = NULL;
+#endif
+   }
+   virtual ~AdaptivityEvaluator();
+
+   /** Specifies the Mesh and FiniteElementCollection of the solution that will
+       be evaluated. The given mesh will be copied into the internal object. */
+   void SetSerialMetaInfo(const Mesh &m,
+                          const FiniteElementCollection &fec, int num_comp);
+
+#ifdef MFEM_USE_MPI
+   /// Parallel version of SetSerialMetaInfo.
+   void SetParMetaInfo(const ParMesh &m,
+                       const FiniteElementCollection &fec, int num_comp);
+#endif
+
+   // TODO use GridFunctions to make clear it's on the ldofs?
+   virtual void SetInitialField(const Vector &init_nodes,
+                                const Vector &init_field) = 0;
+
+   virtual void ComputeAtNewPosition(const Vector &new_nodes,
+                                     Vector &new_field) = 0;
+};
 
 /** @brief Base class representing target-matrix construction algorithms for
     mesh optimization via the target-matrix optimization paradigm (TMOP). */
@@ -538,9 +582,11 @@ public:
       IDEAL_SHAPE_GIVEN_SIZE, /**<
          Ideal shape, given size/volume; the given nodes define the target
          volume at all quadrature points. */
-      GIVEN_SHAPE_AND_SIZE /**<
+      GIVEN_SHAPE_AND_SIZE, /**<
          Given shape, given size/volume; the given nodes define the exact target
          Jacobian matrix at all quadrature points. */
+      GIVEN_FULL /**<
+         Full target tensor is specified at every quadrature point. */
    };
 
 protected:
@@ -589,14 +635,89 @@ public:
    void SetVolumeScale(double vol_scale) { volume_scale = vol_scale; }
 
    /** @brief Given an element and quadrature rule, computes ref->target
-       transformation Jacobians for each quadrature point in the element. */
+       transformation Jacobians for each quadrature point in the element.
+       The physical positions of the element's nodes are given by @a elfun. */
    virtual void ComputeElementTargets(int e_id, const FiniteElement &fe,
                                       const IntegrationRule &ir,
+                                      const Vector &elfun,
+                                      DenseTensor &Jtr) const;
+};
+
+class AnalyticAdaptTC : public TargetConstructor
+{
+protected:
+   // Analytic target specification.
+   Coefficient *scalar_tspec;
+   VectorCoefficient *vector_tspec;
+   MatrixCoefficient *matrix_tspec;
+
+public:
+   AnalyticAdaptTC(TargetType ttype)
+      : TargetConstructor(ttype),
+        scalar_tspec(NULL), vector_tspec(NULL), matrix_tspec(NULL) { }
+
+   virtual void SetAnalyticTargetSpec(Coefficient *sspec,
+                                      VectorCoefficient *vspec,
+                                      MatrixCoefficient *mspec);
+
+   /** @brief Given an element and quadrature rule, computes ref->target
+       transformation Jacobians for each quadrature point in the element.
+       The physical positions of the element's nodes are given by @a elfun. */
+   virtual void ComputeElementTargets(int e_id, const FiniteElement &fe,
+                                      const IntegrationRule &ir,
+                                      const Vector &elfun,
                                       DenseTensor &Jtr) const;
 };
 
 class ParGridFunction;
 
+class DiscreteAdaptTC : public TargetConstructor
+{
+protected:
+   // Discrete target specification.
+   // Data is owned, updated by UpdateTargetSpecification.
+   Vector target_spec;
+   // Note: do not use the Nodes of this space as they may not be on the
+   // positions corresponding to the values of tspec.
+   const FiniteElementSpace *tspec_fes;
+
+   // Evaluation of the discrete target specification on different meshes.
+   // Owned.
+   AdaptivityEvaluator *adapt_eval;
+
+public:
+   DiscreteAdaptTC(TargetType ttype)
+      : TargetConstructor(ttype),
+        target_spec(), tspec_fes(NULL), adapt_eval(NULL) { }
+
+   virtual ~DiscreteAdaptTC() { delete adapt_eval; }
+
+   virtual void SetSerialDiscreteTargetSpec(GridFunction &tspec);
+#ifdef MFEM_USE_MPI
+   virtual void SetParDiscreteTargetSpec(ParGridFunction &tspec);
+#endif
+
+   /** Used to update the target specification after the mesh has changed. The
+       new mesh positions are given by new_x. */
+   void UpdateTargetSpecification(const Vector &new_x);
+
+   void SetAdaptivityEvaluator(AdaptivityEvaluator *ae)
+   {
+      if (adapt_eval) { delete adapt_eval; }
+      adapt_eval = ae;
+   }
+
+   /** @brief Given an element and quadrature rule, computes ref->target
+       transformation Jacobians for each quadrature point in the element.
+       The physical positions of the element's nodes are given by @a elfun.
+       Note that this function assumes that UpdateTargetSpecification() has
+       been called with the position vector corresponding to @a elfun. */
+   virtual void ComputeElementTargets(int e_id, const FiniteElement &fe,
+                                      const IntegrationRule &ir,
+                                      const Vector &elfun,
+                                      DenseTensor &Jtr) const;
+};
+
 /** @brief A TMOP integrator class based on any given TMOP_QualityMetric and
     TargetConstructor.
 

fem/tmop_tools.cpp

@@ -0,0 +1,518 @@
+// Copyright (c) 2010, Lawrence Livermore National Security, LLC. Produced at
+// the Lawrence Livermore National Laboratory. LLNL-CODE-443211. All Rights
+// reserved. See file COPYRIGHT for details.
+//
+// This file is part of the MFEM library. For more information and source code
+// availability see http://mfem.org.
+//
+// MFEM is free software; you can redistribute it and/or modify it under the
+// terms of the GNU Lesser General Public License (as published by the Free
+// Software Foundation) version 2.1 dated February 1999.
+
+#include "tmop_tools.hpp"
+#include "nonlinearform.hpp"
+#include "pnonlinearform.hpp"
+#include "../general/osockstream.hpp"
+
+namespace mfem
+{
+
+using namespace mfem;
+
+void AdvectorCG::SetInitialField(const Vector &init_nodes,
+                                 const Vector &init_field)
+{
+   nodes0 = init_nodes;
+   field0 = init_field;
+}
+
+void AdvectorCG::ComputeAtNewPosition(const Vector &new_nodes,
+                                      Vector &new_field)
+{
+   int myid = 0;
+   Mesh *m = mesh;
+
+#ifdef MFEM_USE_MPI
+   if (pfes) { MPI_Comm_rank(pfes->GetComm(), &myid); }
+   if (pmesh) { m = pmesh; }
+#endif
+
+   MFEM_VERIFY(m != NULL, "No mesh has been given to the AdaptivityEvaluator.");
+
+   // This will be used to move the positions.
+   GridFunction *mesh_nodes = m->GetNodes();
+   *mesh_nodes = nodes0;
+   new_field = field0;
+
+   // Velocity of the positions.
+   GridFunction u(mesh_nodes->FESpace());
+   subtract(new_nodes, nodes0, u);
+
+   TimeDependentOperator *oper = NULL;
+   // This must be the fes of the ind, associated with the object's mesh.
+   if (fes)  { oper = new SerialAdvectorCGOper(nodes0, u, *fes); }
+#ifdef MFEM_USE_MPI
+   else if (pfes) { oper = new ParAdvectorCGOper(nodes0, u, *pfes); }
+#endif
+   MFEM_VERIFY(oper != NULL,
+               "No FE space has been given to the AdaptivityEvaluator.");
+   ode_solver.Init(*oper);
+
+   // Compute some time step [mesh_size / speed].
+   double min_h = std::numeric_limits<double>::infinity();
+   for (int i = 0; i < m->GetNE(); i++)
+   {
+      min_h = std::min(min_h, m->GetElementSize(i));
+   }
+   double v_max = 0.0;
+   const int s = u.FESpace()->GetVSize() / 2;
+   for (int i = 0; i < s; i++)
+   {
+      const double vel = u(i) * u(i) + u(i+s) * u(i+s);
+      v_max = std::max(v_max, vel);
+   }
+   if (v_max == 0.0)
+   {
+      // No need to change the field.
+      return;
+   }
+   v_max = std::sqrt(v_max);
+   double dt = 0.5 * min_h / v_max;
+   double glob_dt = dt;
+#ifdef MFEM_USE_MPI
+   if (pfes)
+   {
+      MPI_Allreduce(&dt, &glob_dt, 1, MPI_DOUBLE, MPI_MIN, pfes->GetComm());
+   }
+#endif
+
+   double t = 0.0;
+   bool last_step = false;
+   for (int ti = 1; !last_step; ti++)
+   {
+      if (t + glob_dt >= 1.0)
+      {
+#ifdef MFEM_DEBUG
+         if (myid == 0)
+         {
+            mfem::out << "Remap took " << ti << " steps." << std::endl;
+         }
+#endif
+         glob_dt = 1.0 - t;
+         last_step = true;
+      }
+      ode_solver.Step(new_field, t, glob_dt);
+   }
+
+   // Trim the overshoots and undershoots.
+   const double minv = field0.Min(), maxv = field0.Max();
+   for (int i = 0; i < new_field.Size(); i++)
+   {
+      if (new_field(i) < minv) { new_field(i) = minv; }
+      if (new_field(i) > maxv) { new_field(i) = maxv; }
+   }
+
+   nodes0 = new_nodes;
+   field0 = new_field;
+
+   delete oper;
+}
+
+SerialAdvectorCGOper::SerialAdvectorCGOper(const Vector &x_start,
+                                           GridFunction &vel,
+                                           FiniteElementSpace &fes)
+   : TimeDependentOperator(fes.GetVSize()),
+     x0(x_start), x_now(*fes.GetMesh()->GetNodes()),
+     u(vel), u_coeff(&u), M(&fes), K(&fes)
+{
+   ConvectionIntegrator *Kinteg = new ConvectionIntegrator(u_coeff);
+   K.AddDomainIntegrator(Kinteg);
+   K.Assemble(0);
+   K.Finalize(0);
+
+   MassIntegrator *Minteg = new MassIntegrator;
+   M.AddDomainIntegrator(Minteg);
+   M.Assemble();
+   M.Finalize();
+}
+
+void SerialAdvectorCGOper::Mult(const Vector &ind, Vector &di_dt) const
+{
+   // Move the mesh.
+   const double t = GetTime();
+   add(x0, t, u, x_now);
+
+   // Assemble on the new mesh.
+   K.BilinearForm::operator=(0.0);
+   K.Assemble();
+   Vector rhs(K.Size());
+   K.Mult(ind, rhs);
+   M.BilinearForm::operator=(0.0);
+   M.Assemble();
+
+   di_dt = 0.0;
+   CGSolver lin_solver;
+   DSmoother prec;
+   lin_solver.SetPreconditioner(prec);
+   lin_solver.SetOperator(M.SpMat());
+   lin_solver.SetRelTol(1e-12); lin_solver.SetAbsTol(0.0);
+   lin_solver.SetMaxIter(100);
+   lin_solver.SetPrintLevel(0);
+   lin_solver.Mult(rhs, di_dt);
+}
+
+#ifdef MFEM_USE_MPI
+ParAdvectorCGOper::ParAdvectorCGOper(const Vector &x_start,
+                                     GridFunction &vel,
+                                     ParFiniteElementSpace &pfes)
+   : TimeDependentOperator(pfes.GetVSize()),
+     x0(x_start), x_now(*pfes.GetMesh()->GetNodes()),
+     u(vel), u_coeff(&u), M(&pfes), K(&pfes)
+{
+   ConvectionIntegrator *Kinteg = new ConvectionIntegrator(u_coeff);
+   K.AddDomainIntegrator(Kinteg);
+   K.Assemble(0);
+   K.Finalize(0);
+
+   MassIntegrator *Minteg = new MassIntegrator;
+   M.AddDomainIntegrator(Minteg);
+   M.Assemble();
+   M.Finalize();
+}
+
+void ParAdvectorCGOper::Mult(const Vector &ind, Vector &di_dt) const
+{
+   // Move the mesh.
+   const double t = GetTime();
+   add(x0, t, u, x_now);
+
+   // Assemble on the new mesh.
+   K.BilinearForm::operator=(0.0);
+   K.Assemble();
+   ParGridFunction rhs(K.ParFESpace());
+   K.Mult(ind, rhs);
+   M.BilinearForm::operator=(0.0);
+   M.Assemble();
+
+   HypreParVector *RHS = rhs.ParallelAssemble();
+   HypreParVector X(K.ParFESpace());
+   X = 0.0;
+   HypreParMatrix *Mh  = M.ParallelAssemble();
+
+   CGSolver lin_solver(M.ParFESpace()->GetParMesh()->GetComm());
+   HypreSmoother prec;
+   prec.SetType(HypreSmoother::Jacobi, 1);
+   lin_solver.SetPreconditioner(prec);
+   lin_solver.SetOperator(*Mh);
+   lin_solver.SetRelTol(1e-8);
+   lin_solver.SetAbsTol(0.0);
+   lin_solver.SetMaxIter(100);
+   lin_solver.SetPrintLevel(0);
+   lin_solver.Mult(*RHS, X);
+   K.ParFESpace()->GetProlongationMatrix()->Mult(X, di_dt);
+
+   delete Mh;
+   delete RHS;
+}
+#endif
+
+double TMOPNewtonSolver::ComputeScalingFactor(const Vector &x,
+                                              const Vector &b) const
+{
+   const FiniteElementSpace *fes = NULL;
+   double energy_in = 0.0;
+#ifdef MFEM_USE_MPI
+   const ParNonlinearForm *p_nlf = dynamic_cast<const ParNonlinearForm *>(oper);
+   MFEM_VERIFY(!(parallel && p_nlf == NULL), "Invalid Operator subclass.");
+   if (parallel)
+   {
+      fes = p_nlf->FESpace();
+      energy_in = p_nlf->GetEnergy(x);
+   }
+#endif
+   const bool serial = !parallel;
+   const NonlinearForm *nlf = dynamic_cast<const NonlinearForm *>(oper);
+   MFEM_VERIFY(!(serial && nlf == NULL), "Invalid Operator subclass.");
+   if (serial)
+   {
+      fes = nlf->FESpace();
+      energy_in = nlf->GetEnergy(x);
+   }
+
+   const bool have_b = (b.Size() == Height());
+
+   const int NE = fes->GetMesh()->GetNE(), dim = fes->GetFE(0)->GetDim(),
+             dof = fes->GetFE(0)->GetDof(), nsp = ir.GetNPoints();
+   Array<int> xdofs(dof * dim);
+   DenseMatrix Jpr(dim), dshape(dof, dim), pos(dof, dim);
+   Vector posV(pos.Data(), dof * dim);
+
+   Vector x_out(x.Size()), x_out_loc(fes->GetVSize());
+   bool x_out_ok = false;
+   double scale = 1.0, energy_out;
+   double norm0 = Norm(r);
+
+   // Decreases the scaling of the update until the new mesh is valid.
+   for (int i = 0; i < 12; i++)
+   {
+      add(x, -scale, c, x_out);
+
+      if (serial)
+      {
+         const SparseMatrix *cP = fes->GetConformingProlongation();
+         if (!cP) {x_out_loc.SetData(x_out.GetData());}
+         else {cP->Mult(x_out,x_out_loc);}
+         energy_out = nlf->GetGridFunctionEnergy(x_out_loc);
+      }
+#ifdef MFEM_USE_MPI
+      else
+      {
+         fes->GetProlongationMatrix()->Mult(x_out, x_out_loc);
+         energy_out = p_nlf->GetParGridFunctionEnergy(x_out_loc);
+      }
+#endif
+
+      if (energy_out > 1.2*energy_in || std::isnan(energy_out) != 0)
+      {
+         if (print_level >= 0)
+         { mfem::out << "Scale = " << scale << " Increasing energy.\n"; }
+         scale *= 0.5; continue;
+      }
+
+      int jac_ok = 1;
+      for (int i = 0; i < NE; i++)
+      {
+         fes->GetElementVDofs(i, xdofs);
+         x_out_loc.GetSubVector(xdofs, posV);
+         for (int j = 0; j < nsp; j++)
+         {
+            fes->GetFE(i)->CalcDShape(ir.IntPoint(j), dshape);
+            MultAtB(pos, dshape, Jpr);
+            if (Jpr.Det() <= 0.0) { jac_ok = 0; goto break2; }
+         }
+      }
+   break2:
+      int jac_ok_all = jac_ok;
+#ifdef MFEM_USE_MPI
+      if (parallel)
+      {
+         MPI_Allreduce(&jac_ok, &jac_ok_all, 1, MPI_INT, MPI_LAND,
+                       p_nlf->ParFESpace()->GetComm());
+      }
+#endif
+
+      if (jac_ok_all == 0)
+      {
+         if (print_level >= 0)
+         { mfem::out << "Scale = " << scale << " Neg det(J) found.\n"; }
+         scale *= 0.5; continue;
+      }
+
+      oper->Mult(x_out, r);
+      if (have_b) { r -= b; }
+      double norm = Norm(r);
+
+      if (norm > 1.2*norm0)
+      {
+         if (print_level >= 0)
+         { mfem::out << "Scale = " << scale << " Norm increased.\n"; }
+         scale *= 0.5; continue;
+      }
+      else { x_out_ok = true; break; }
+   }
+
+   if (print_level >= 0)
+   {
+      mfem::out << "Energy decrease: "
+                << (energy_in - energy_out) / energy_in * 100.0
+                << "% with " << scale << " scaling.\n";
+   }
+
+   if (x_out_ok == false) { scale = 0.0; }
+   return scale;
+}
+
+void TMOPNewtonSolver::ProcessNewState(const Vector &x) const
+{
+   if (discr_tc)
+   {
+      if (parallel)
+      {
+#ifdef MFEM_USE_MPI
+         const ParNonlinearForm *nlf =
+            dynamic_cast<const ParNonlinearForm *>(oper);
+         Vector x_loc(nlf->ParFESpace()->GetVSize());
+         nlf->ParFESpace()->GetProlongationMatrix()->Mult(x, x_loc);
+         discr_tc->UpdateTargetSpecification(x_loc);
+#endif
+      }
+      else { discr_tc->UpdateTargetSpecification(x); }
+   }
+}
+
+double TMOPDescentNewtonSolver::ComputeScalingFactor(const Vector &x,
+                                                     const Vector &b) const
+{
+   const FiniteElementSpace *fes = NULL;
+   double energy_in = 0.0;
+#ifdef MFEM_USE_MPI
+   const ParNonlinearForm *p_nlf = dynamic_cast<const ParNonlinearForm *>(oper);
+   MFEM_VERIFY(!(parallel && p_nlf == NULL), "Invalid Operator subclass.");
+   if (parallel)
+   {
+      fes = p_nlf->FESpace();
+      energy_in = p_nlf->GetEnergy(x);
+   }
+#endif
+   const bool serial = !parallel;
+   const NonlinearForm *nlf = dynamic_cast<const NonlinearForm *>(oper);
+   MFEM_VERIFY(!(serial && nlf == NULL), "Invalid Operator subclass.");
+   if (serial)
+   {
+      fes = nlf->FESpace();
+      energy_in = nlf->GetEnergy(x);
+   }
+
+   const int NE = fes->GetMesh()->GetNE(), dim = fes->GetFE(0)->GetDim(),
+             dof = fes->GetFE(0)->GetDof(), nsp = ir.GetNPoints();
+   Array<int> xdofs(dof * dim);
+   DenseMatrix Jpr(dim), dshape(dof, dim), pos(dof, dim);
+   Vector posV(pos.Data(), dof * dim);
+   Vector x_loc(fes->GetVSize());
+
+   double min_detJ = infinity();
+   for (int i = 0; i < NE; i++)
+   {
+      fes->GetElementVDofs(i, xdofs);
+      x_loc.GetSubVector(xdofs, posV);
+
+      for (int j = 0; j < nsp; j++)
+      {
+         fes->GetFE(i)->CalcDShape(ir.IntPoint(j), dshape);
+         MultAtB(pos, dshape, Jpr);
+         min_detJ = std::min(min_detJ, Jpr.Det());
+      }
+   }
+   double min_detJ_all = min_detJ;
+#ifdef MFEM_USE_MPI
+   if (parallel)
+   {
+      MPI_Allreduce(&min_detJ, &min_detJ_all, 1, MPI_DOUBLE, MPI_MIN,
+                    p_nlf->ParFESpace()->GetComm());
+   }
+#endif
+   if (print_level >= 0)
+   {
+      mfem::out << "Minimum det(J) = " << min_detJ_all << '\n';
+   }
+
+   Vector x_out(x.Size());
+   bool x_out_ok = false;
+   double scale = 1.0, energy_out;
+
+   for (int i = 0; i < 7; i++)
+   {
+      add(x, -scale, c, x_out);
+      if (serial)
+      {
+         const SparseMatrix *cP = fes->GetConformingProlongation();
+         if (!cP) {x_loc.SetData(x_out.GetData());}
+         else {cP->Mult(x_out,x_loc);}
+         energy_out = nlf->GetGridFunctionEnergy(x_loc);
+      }
+#ifdef MFEM_USE_MPI
+      else
+      {
+         fes->GetProlongationMatrix()->Mult(x_out, x_loc);
+         energy_out = p_nlf->GetParGridFunctionEnergy(x_loc);
+      }
+#endif
+
+      if (energy_out > energy_in || std::isnan(energy_out) != 0)
+      {
+         scale *= 0.5;
+      }
+      else { x_out_ok = true; break; }
+   }
+
+   if (print_level >= 0)
+   {
+      mfem::out << "Energy decrease: "
+                << (energy_in - energy_out) / energy_in * 100.0
+                << "% with " << scale << " scaling.\n";
+   }
+
+   if (x_out_ok == false) { return 0.0; }
+
+   return scale;
+}
+
+void TMOPDescentNewtonSolver::ProcessNewState(const Vector &x) const
+{
+   if (discr_tc)
+   {
+      if (parallel)
+      {
+#ifdef MFEM_USE_MPI
+         const ParNonlinearForm *nlf =
+            dynamic_cast<const ParNonlinearForm *>(oper);
+         Vector x_loc(nlf->ParFESpace()->GetVSize());
+         nlf->ParFESpace()->GetProlongationMatrix()->Mult(x, x_loc);
+         discr_tc->UpdateTargetSpecification(x_loc);
+#endif
+      }
+      else { discr_tc->UpdateTargetSpecification(x); }
+   }
+}
+
+#ifdef MFEM_USE_MPI
+// Metric values are visualized by creating an L2 finite element functions and
+// computing the metric values at the nodes.
+void vis_tmop_metric_p(int order, TMOP_QualityMetric &qm,
+                       const TargetConstructor &tc, ParMesh &pmesh,
+                       char *title, int position)
+{
+   L2_FECollection fec(order, pmesh.Dimension(), BasisType::GaussLobatto);
+   ParFiniteElementSpace fes(&pmesh, &fec, 1);
+   ParGridFunction metric(&fes);
+   InterpolateTMOP_QualityMetric(qm, tc, pmesh, metric);
+   socketstream sock;
+   if (pmesh.GetMyRank() == 0)
+   {
+      sock.open("localhost", 19916);
+      sock << "solution\n";
+   }
+   pmesh.PrintAsOne(sock);
+   metric.SaveAsOne(sock);
+   if (pmesh.GetMyRank() == 0)
+   {
+      sock << "window_title '"<< title << "'\n"
+           << "window_geometry "
+           << position << " " << 0 << " " << 600 << " " << 600 << "\n"
+           << "keys jRmclA\n";
+   }
+}
+#endif
+
+// Metric values are visualized by creating an L2 finite element functions and
+// computing the metric values at the nodes.
+void vis_tmop_metric_s(int order, TMOP_QualityMetric &qm,
+                       const TargetConstructor &tc, Mesh &mesh,
+                       char *title, int position)
+{
+   L2_FECollection fec(order, mesh.Dimension(), BasisType::GaussLobatto);
+   FiniteElementSpace fes(&mesh, &fec, 1);
+   GridFunction metric(&fes);
+   InterpolateTMOP_QualityMetric(qm, tc, mesh, metric);
+   osockstream sock(19916, "localhost");
+   sock << "solution\n";
+   mesh.Print(sock);
+   metric.Save(sock);
+   sock.send();
+   sock << "window_title '"<< title << "'\n"
+        << "window_geometry "
+        << position << " " << 0 << " " << 600 << " " << 600 << "\n"
+        << "keys jRmclA\n";
+}
+
+}

fem/tmop_tools.hpp

@@ -0,0 +1,140 @@
+// Copyright (c) 2010, Lawrence Livermore National Security, LLC. Produced at
+// the Lawrence Livermore National Laboratory. LLNL-CODE-443211. All Rights
+// reserved. See file COPYRIGHT for details.
+//
+// This file is part of the MFEM library. For more information and source code
+// availability see http://mfem.org.
+//
+// MFEM is free software; you can redistribute it and/or modify it under the
+// terms of the GNU Lesser General Public License (as published by the Free
+// Software Foundation) version 2.1 dated February 1999.
+
+#ifndef MFEM_TMOP_TOOLS_HPP
+#define MFEM_TMOP_TOOLS_HPP
+
+#include "bilinearform.hpp"
+#include "pbilinearform.hpp"
+#include "tmop.hpp"
+
+namespace mfem
+{
+
+// Performs the full remap advection loop.
+class AdvectorCG : public AdaptivityEvaluator
+{
+private:
+   RK4Solver ode_solver;
+   Vector nodes0;
+   Vector field0;
+
+public:
+   AdvectorCG() : AdaptivityEvaluator(), ode_solver(), nodes0(), field0() { }
+
+   virtual void SetInitialField(const Vector &init_nodes,
+                                const Vector &init_field);
+
+   virtual void ComputeAtNewPosition(const Vector &new_nodes,
+                                     Vector &new_field);
+};
+
+/// Performs a single remap advection step in serial.
+class SerialAdvectorCGOper : public TimeDependentOperator
+{
+protected:
+   const Vector &x0;
+   Vector &x_now;
+   GridFunction &u;
+   VectorGridFunctionCoefficient u_coeff;
+   mutable BilinearForm M, K;
+
+public:
+   /** Here @a fes is the FESpace of the function that will be moved. Note
+       that Mult() moves the nodes of the mesh corresponding to @a fes. */
+   SerialAdvectorCGOper(const Vector &x_start, GridFunction &vel,
+                        FiniteElementSpace &fes);
+
+   virtual void Mult(const Vector &ind, Vector &di_dt) const;
+};
+
+#ifdef MFEM_USE_MPI
+/// Performs a single remap advection step in parallel.
+class ParAdvectorCGOper : public TimeDependentOperator
+{
+protected:
+   const Vector &x0;
+   Vector &x_now;
+   GridFunction &u;
+   VectorGridFunctionCoefficient u_coeff;
+   mutable ParBilinearForm M, K;
+
+public:
+   /** Here @a pfes is the ParFESpace of the function that will be moved. Note
+       that Mult() moves the nodes of the mesh corresponding to @a pfes. */
+   ParAdvectorCGOper(const Vector &x_start, GridFunction &vel,
+                     ParFiniteElementSpace &pfes);
+
+   virtual void Mult(const Vector &ind, Vector &di_dt) const;
+};
+#endif
+
+class TMOPNewtonSolver : public NewtonSolver
+{
+private:
+   bool parallel;
+
+   // Quadrature points that are checked for negative Jacobians etc.
+   const IntegrationRule &ir;
+
+   mutable DiscreteAdaptTC *discr_tc;
+
+public:
+#ifdef MFEM_USE_MPI
+   TMOPNewtonSolver(MPI_Comm comm, const IntegrationRule &irule)
+      : NewtonSolver(comm), parallel(true), ir(irule), discr_tc(NULL) { }
+#endif
+   TMOPNewtonSolver(const IntegrationRule &irule)
+      : NewtonSolver(), parallel(false), ir(irule), discr_tc(NULL) { }
+
+   void SetDiscreteAdaptTC(DiscreteAdaptTC *tc) { discr_tc = tc; }
+
+   virtual double ComputeScalingFactor(const Vector &x, const Vector &b) const;
+
+   virtual void ProcessNewState(const Vector &x) const;
+};
+
+/// Allows negative Jacobians. Used for untangling.
+class TMOPDescentNewtonSolver : public NewtonSolver
+{
+private:
+   bool parallel;
+
+   // Quadrature points that are checked for negative Jacobians etc.
+   const IntegrationRule &ir;
+
+   mutable DiscreteAdaptTC *discr_tc;
+
+public:
+#ifdef MFEM_USE_MPI
+   TMOPDescentNewtonSolver(MPI_Comm comm, const IntegrationRule &irule)
+      : NewtonSolver(comm), parallel(true), ir(irule), discr_tc(NULL) { }
+#endif
+   TMOPDescentNewtonSolver(const IntegrationRule &irule)
+      : NewtonSolver(), parallel(false), ir(irule), discr_tc(NULL) { }
+
+   virtual double ComputeScalingFactor(const Vector &x, const Vector &b) const;
+
+   virtual void ProcessNewState(const Vector &x) const;
+};
+
+void vis_tmop_metric_s(int order, TMOP_QualityMetric &qm,
+                       const TargetConstructor &tc, Mesh &pmesh,
+                       char *title, int position);
+#ifdef MFEM_USE_MPI
+void vis_tmop_metric_p(int order, TMOP_QualityMetric &qm,
+                       const TargetConstructor &tc, ParMesh &pmesh,
+                       char *title, int position);
+#endif
+
+}
+
+#endif

general/array.hpp

@@ -254,6 +254,10 @@ public:
    template <typename U>
    inline void CopyTo(U *dest) { std::copy(begin(), end(), dest); }
 
+   template <typename U>
+   inline void CopyFrom(const U *src)
+   { std::memcpy(begin(), src, MemoryUsage()); }
+
    // STL-like begin/end
    inline T* begin() { return data; }
    inline T* end() { return data + size; }
@@ -429,7 +433,7 @@ class BlockArray
 public:
    BlockArray(int block_size = 16*1024);
    BlockArray(const BlockArray<T> &other); // deep copy
-   ~BlockArray();
+   ~BlockArray() { Destroy(); }
 
    /// Allocate and construct a new item in the array, return its index.
    int Append();
@@ -459,6 +463,9 @@ public:
    /// Return the current capacity of the BlockArray.
    int Capacity() const { return blocks.Size()*(mask+1); }
 
+   /// Destroy all items, set size to zero.
+   void DeleteAll() { Destroy(); blocks.DeleteAll(); size = 0; }
+
    void Swap(BlockArray<T> &other);
 
    long MemoryUsage() const;
@@ -563,6 +570,8 @@ protected:
       MFEM_ASSERT(index >= 0 && index < size,
                   "Out of bounds access: " << index << ", size = " << size);
    }
+
+   void Destroy();
 };
 
 
@@ -994,7 +1003,7 @@ long BlockArray<T>::MemoryUsage() const
 }
 
 template<typename T>
-BlockArray<T>::~BlockArray()
+void BlockArray<T>::Destroy()
 {
    int bsize = size & mask;
    for (int i = blocks.Size(); i != 0; )

general/communication.cpp

@@ -513,6 +513,78 @@ void GroupCommunicator::SetLTDofTable(const Array<int> &ldof_ltdof)
    group_ltdof.ShiftUpI();
 }
 
+void GroupCommunicator::GetNeighborLTDofTable(Table &nbr_ltdof) const
+{
+   nbr_ltdof.MakeI(nbr_send_groups.Size());
+   for (int nbr = 1; nbr < nbr_send_groups.Size(); nbr++)
+   {
+      const int num_send_groups = nbr_send_groups.RowSize(nbr);
+      if (num_send_groups > 0)
+      {
+         const int *grp_list = nbr_send_groups.GetRow(nbr);
+         for (int i = 0; i < num_send_groups; i++)
+         {
+            const int group = grp_list[i];
+            const int nltdofs = group_ltdof.RowSize(group);
+            nbr_ltdof.AddColumnsInRow(nbr, nltdofs);
+         }
+      }
+   }
+   nbr_ltdof.MakeJ();
+   for (int nbr = 1; nbr < nbr_send_groups.Size(); nbr++)
+   {
+      const int num_send_groups = nbr_send_groups.RowSize(nbr);
+      if (num_send_groups > 0)
+      {
+         const int *grp_list = nbr_send_groups.GetRow(nbr);
+         for (int i = 0; i < num_send_groups; i++)
+         {
+            const int group = grp_list[i];
+            const int nltdofs = group_ltdof.RowSize(group);
+            const int *ltdofs = group_ltdof.GetRow(group);
+            nbr_ltdof.AddConnections(nbr, ltdofs, nltdofs);
+         }
+      }
+   }
+   nbr_ltdof.ShiftUpI();
+}
+
+void GroupCommunicator::GetNeighborLDofTable(Table &nbr_ldof) const
+{
+   nbr_ldof.MakeI(nbr_recv_groups.Size());
+   for (int nbr = 1; nbr < nbr_recv_groups.Size(); nbr++)
+   {
+      const int num_recv_groups = nbr_recv_groups.RowSize(nbr);
+      if (num_recv_groups > 0)
+      {
+         const int *grp_list = nbr_recv_groups.GetRow(nbr);
+         for (int i = 0; i < num_recv_groups; i++)
+         {
+            const int group = grp_list[i];
+            const int nldofs = group_ldof.RowSize(group);
+            nbr_ldof.AddColumnsInRow(nbr, nldofs);
+         }
+      }
+   }
+   nbr_ldof.MakeJ();
+   for (int nbr = 1; nbr < nbr_recv_groups.Size(); nbr++)
+   {
+      const int num_recv_groups = nbr_recv_groups.RowSize(nbr);
+      if (num_recv_groups > 0)
+      {
+         const int *grp_list = nbr_recv_groups.GetRow(nbr);
+         for (int i = 0; i < num_recv_groups; i++)
+         {
+            const int group = grp_list[i];
+            const int nldofs = group_ldof.RowSize(group);
+            const int *ldofs = group_ldof.GetRow(group);
+            nbr_ldof.AddConnections(nbr, ldofs, nldofs);
+         }
+      }
+   }
+   nbr_ldof.ShiftUpI();
+}
+
 template <class T>
 T *GroupCommunicator::CopyGroupToBuffer(const T *ldata, T *buf, int group,
                                         int layout) const

general/communication.hpp

@@ -179,6 +179,12 @@ public:
    /// Get a const reference to the associated GroupTopology object
    const GroupTopology &GetGroupTopology() const { return gtopo; }
 
+   /// Dofs to be sent to communication neighbors
+   void GetNeighborLTDofTable(Table &nbr_ltdof) const;
+
+   /// Dofs to be received from communication neighbors
+   void GetNeighborLDofTable(Table &nbr_ldof) const;
+
    /** @brief Data structure on which we define reduce operations.
 
      The data is associated with (and the operation is performed on) one group
@@ -316,7 +322,9 @@ struct VarMessage
    std::string data;
    MPI_Request send_request;
 
-   /// Non-blocking send to processor 'rank'.
+   /** Non-blocking send to processor 'rank'. Returns immediately. Completion
+       (as tested by MPI_Wait/Test) does not mean the message was received --
+       it may be on its way or just buffered locally. */
    void Isend(int rank, MPI_Comm comm)
    {
       Encode(rank);
@@ -324,12 +332,20 @@ struct VarMessage
                 &send_request);
    }
 
+   /** Non-blocking synchronous send to processor 'rank'. Returns immediately.
+       Completion (MPI_Wait/Test) means that the message was received. */
+   void Issend(int rank, MPI_Comm comm)
+   {
+      Encode(rank);
+      MPI_Issend((void*) data.data(), data.length(), MPI_BYTE, rank, Tag, comm,
+                 &send_request);
+   }
+
    /// Helper to send all messages in a rank-to-message map container.
    template<typename MapT>
    static void IsendAll(MapT& rank_msg, MPI_Comm comm)
    {
-      typename MapT::iterator it;
-      for (it = rank_msg.begin(); it != rank_msg.end(); ++it)
+      for (auto it = rank_msg.begin(); it != rank_msg.end(); ++it)
       {
          it->second.Isend(it->first, comm);
       }
@@ -339,14 +355,32 @@ struct VarMessage
    template<typename MapT>
    static void WaitAllSent(MapT& rank_msg)
    {
-      typename MapT::iterator it;
-      for (it = rank_msg.begin(); it != rank_msg.end(); ++it)
+      for (auto it = rank_msg.begin(); it != rank_msg.end(); ++it)
       {
          MPI_Wait(&it->second.send_request, MPI_STATUS_IGNORE);
          it->second.Clear();
       }
    }
 
+   /** Return true if all messages in the map container were sent, otherwise
+       return false, without waiting. */
+   template<typename MapT>
+   static bool TestAllSent(MapT& rank_msg)
+   {
+      for (auto it = rank_msg.begin(); it != rank_msg.end(); ++it)
+      {
+         VarMessage &msg = it->second;
+         if (msg.send_request != MPI_REQUEST_NULL)
+         {
+            int sent;
+            MPI_Test(&msg.send_request, &sent, MPI_STATUS_IGNORE);
+            if (!sent) { return false; }
+            msg.Clear();
+         }
+      }
+      return true;
+   }
+
    /** Blocking probe for incoming message of this type from any rank.
        Returns the rank and message size. */
    static void Probe(int &rank, int &size, MPI_Comm comm)

general/cuda.cpp

@@ -130,4 +130,13 @@ void* CuMemcpyDtoHAsync(void *dst, const void *src, size_t bytes)
    return dst;
 }
 
+int CuGetDeviceCount()
+{
+   int num_gpus = -1;
+#ifdef MFEM_USE_CUDA
+   MFEM_GPU_CHECK(cudaGetDeviceCount(&num_gpus));
+#endif
+   return num_gpus;
+}
+
 } // namespace mfem

general/cuda.hpp

@@ -39,9 +39,11 @@
       } \
    } \
    while (0)
+#define MFEM_DEVICE_SYNC MFEM_GPU_CHECK(cudaDeviceSynchronize())
 #else
 #define MFEM_DEVICE
 #define MFEM_HOST_DEVICE
+#define MFEM_DEVICE_SYNC
 #endif // MFEM_USE_CUDA
 
 // Define the MFEM inner threading macros
@@ -92,6 +94,9 @@ void* CuMemcpyDtoH(void *h_dst, const void *d_src, size_t bytes);
 /// Copies memory from Device to Host
 void* CuMemcpyDtoHAsync(void *h_dst, const void *d_src, size_t bytes);
 
+/// Get the number of CUDA devices
+int CuGetDeviceCount();
+
 } // namespace mfem
 
 #endif // MFEM_CUDA_HPP

general/device.cpp

@@ -138,7 +138,7 @@ void Device::Enable()
 #ifdef MFEM_USE_CUDA
 static void DeviceSetup(const int dev, int &ngpu)
 {
-   MFEM_GPU_CHECK(cudaGetDeviceCount(&ngpu));
+   ngpu = CuGetDeviceCount();
    MFEM_VERIFY(ngpu > 0, "No CUDA device found!");
    MFEM_GPU_CHECK(cudaSetDevice(dev));
 }

general/device.hpp

@@ -12,6 +12,7 @@
 #ifndef MFEM_DEVICE_HPP
 #define MFEM_DEVICE_HPP
 
+#include "cuda.hpp"
 #include "globals.hpp"
 #include "mem_manager.hpp"
 
@@ -110,6 +111,7 @@ private:
    unsigned long backends; ///< Bitwise-OR of all configured backends.
    /// Set to true during configuration, except in 'device_singleton'.
    bool destroy_mm;
+   bool mpi_gpu_aware;
 
    MemoryType mem_type;    ///< Current Device MemoryType
    MemoryClass mem_class;  ///< Current Device MemoryClass
@@ -147,6 +149,7 @@ public:
       : mode(Device::SEQUENTIAL),
         backends(Backend::CPU),
         destroy_mm(false),
+        mpi_gpu_aware(false),
         mem_type(MemoryType::HOST),
         mem_class(MemoryClass::HOST)
    { }
@@ -161,6 +164,7 @@ public:
       : mode(Device::SEQUENTIAL),
         backends(Backend::CPU),
         destroy_mm(false),
+        mpi_gpu_aware(false),
         mem_type(MemoryType::HOST),
         mem_class(MemoryClass::HOST)
    { Configure(device, dev); }
@@ -216,6 +220,13 @@ public:
    /** @brief Get the current Device MemoryClass. This is the MemoryClass used
        by most MFEM device kernels to access Memory objects. */
    static inline MemoryClass GetMemoryClass() { return Get().mem_class; }
+
+   static void SetGPUAwareMPI(const bool force = true)
+   { Get().mpi_gpu_aware = force; }
+
+   static bool GetGPUAwareMPI() { return Get().mpi_gpu_aware; }
+
+   static void Synchronize() { MFEM_DEVICE_SYNC; }
 };
 
 
@@ -265,7 +276,7 @@ inline T *Write(Memory<T> &mem, int size, bool on_dev = true)
 
 /** @brief Shortcut to Write(const Memory<T> &mem, int size, false) */
 template <typename T>
-inline const T *HostWrite(const Memory<T> &mem, int size)
+inline T *HostWrite(Memory<T> &mem, int size)
 {
    return mfem::Write(mem, size, false);
 }
@@ -287,9 +298,9 @@ inline T *ReadWrite(Memory<T> &mem, int size, bool on_dev = true)
    }
 }
 
-/** @brief Shortcut to ReadWrite(const Memory<T> &mem, int size, false) */
+/** @brief Shortcut to ReadWrite(Memory<T> &mem, int size, false) */
 template <typename T>
-inline const T *HostReadWrite(const Memory<T> &mem, int size)
+inline T *HostReadWrite(Memory<T> &mem, int size)
 {
    return mfem::ReadWrite(mem, size, false);
 }

general/error.hpp

@@ -138,4 +138,10 @@ void mfem_warning(const char *msg = NULL);
 // Generate a warning message - always generated, regardless of MFEM_DEBUG.
 #define MFEM_WARNING(msg) _MFEM_MESSAGE("MFEM Warning: " << msg, 1)
 
+// Macro that checks (in MFEM_DEBUG mode) that i is in the range [imin,imax).
+#define MFEM_ASSERT_INDEX_IN_RANGE(i,imin,imax) \
+   MFEM_ASSERT((imin) <= (i) && (i) < (imax), \
+   "invalid index " #i << " = " << (i) << \
+   ", valid range is [" << (imin) << ',' << (imax) << ')')
+
 #endif

general/forall.hpp

@@ -84,28 +84,76 @@ void OmpWrap(const int N, HBODY &&h_body)
 
 
 /// RAJA Cuda backend
-template <int BLOCKS, typename DBODY>
-void RajaCudaWrap(const int N, DBODY &&d_body)
-{
 #if defined(MFEM_USE_RAJA) && defined(RAJA_ENABLE_CUDA)
-   RAJA::forall<RAJA::cuda_exec<BLOCKS>>(RAJA::RangeSegment(0,N),d_body);
-#else
-   MFEM_ABORT("RAJA::Cuda requested but RAJA::Cuda is not enabled!");
-#endif
+
+using RAJA::statement::Segs;
+
+template <const int BLOCKS = MFEM_CUDA_BLOCKS, typename DBODY>
+void RajaCudaWrap1D(const int N, DBODY &&d_body)
+{
+   //true denotes asynchronous kernel
+   RAJA::forall<RAJA::cuda_exec<BLOCKS,true>>(RAJA::RangeSegment(0,N),d_body);
 }
 
+template <typename DBODY>
+void RajaCudaWrap2D(const int N, DBODY &&d_body,
+                    const int X, const int Y, const int BZ)
+{
+   MFEM_VERIFY(N>0, "");
+   MFEM_VERIFY(BZ>0, "");
+   const int G = (N+BZ-1)/BZ;
+   RAJA::kernel<RAJA::KernelPolicy<
+   RAJA::statement::CudaKernelAsync<
+   RAJA::statement::For<0, RAJA::cuda_block_x_direct,
+        RAJA::statement::For<1, RAJA::cuda_thread_x_direct,
+        RAJA::statement::For<2, RAJA::cuda_thread_y_direct,
+        RAJA::statement::For<3, RAJA::cuda_thread_z_direct,
+        RAJA::statement::Lambda<0, Segs<0>>>>>>>>>
+        (RAJA::make_tuple(RAJA::RangeSegment(0,G), RAJA::RangeSegment(0,X),
+                          RAJA::RangeSegment(0,Y), RAJA::RangeSegment(0,BZ)),
+         [=] RAJA_DEVICE (const int n)
+   {
+      const int k = n*BZ + threadIdx.z;
+      if (k >= N) { return; }
+      d_body(k);
+   });
+   MFEM_GPU_CHECK(cudaGetLastError());
+}
+
+template <typename DBODY>
+void RajaCudaWrap3D(const int N, DBODY &&d_body,
+                    const int X, const int Y, const int Z)
+{
+   MFEM_VERIFY(N>0, "");
+   RAJA::kernel<RAJA::KernelPolicy<
+   RAJA::statement::CudaKernelAsync<
+   RAJA::statement::For<0, RAJA::cuda_block_x_direct,
+        RAJA::statement::For<1, RAJA::cuda_thread_x_direct,
+        RAJA::statement::For<2, RAJA::cuda_thread_y_direct,
+        RAJA::statement::For<3, RAJA::cuda_thread_z_direct,
+        RAJA::statement::Lambda<0, Segs<0>>>>>>>>>
+        (RAJA::make_tuple(RAJA::RangeSegment(0,N), RAJA::RangeSegment(0,X),
+                          RAJA::RangeSegment(0,Y), RAJA::RangeSegment(0,Z)),
+   [=] RAJA_DEVICE (const int k) { d_body(k); });
+   MFEM_GPU_CHECK(cudaGetLastError());
+}
+
+#endif
+
 
 /// RAJA OpenMP backend
+#if defined(MFEM_USE_RAJA) && defined(RAJA_ENABLE_OPENMP)
+
+using RAJA::statement::Segs;
+
 template <typename HBODY>
 void RajaOmpWrap(const int N, HBODY &&h_body)
 {
-#if defined(MFEM_USE_RAJA) && defined(RAJA_ENABLE_OPENMP)
    RAJA::forall<RAJA::omp_parallel_for_exec>(RAJA::RangeSegment(0,N), h_body);
-#else
-   MFEM_ABORT("RAJA::OpenMP requested but RAJA::OpenMP is not enabled!");
-#endif
 }
 
+#endif
+
 
 /// RAJA sequential loop backend
 template <typename HBODY>
@@ -160,6 +208,7 @@ void CuWrap2D(const int N, DBODY &&d_body,
               const int X, const int Y, const int BZ)
 {
    if (N==0) { return; }
+   MFEM_VERIFY(BZ>0, "");
    const int GRID = (N+BZ-1)/BZ;
    const dim3 BLCK(X,Y,BZ);
    CuKernel2D<<<GRID,BLCK>>>(N,d_body,BZ);
@@ -251,8 +300,14 @@ inline void ForallWrap(const bool use_dev, const int N,
 
 #if defined(MFEM_USE_RAJA) && defined(RAJA_ENABLE_CUDA)
    // Handle all allowed CUDA backends except Backend::CUDA
-   if (Device::Allows(Backend::CUDA_MASK & ~Backend::CUDA))
-   { return RajaCudaWrap<MFEM_CUDA_BLOCKS>(N, d_body); }
+   if (DIM == 1 && Device::Allows(Backend::CUDA_MASK & ~Backend::CUDA))
+   { return RajaCudaWrap1D(N, d_body); }
+
+   if (DIM == 2 && Device::Allows(Backend::CUDA_MASK & ~Backend::CUDA))
+   { return RajaCudaWrap2D(N, d_body, X, Y, Z); }
+
+   if (DIM == 3 && Device::Allows(Backend::CUDA_MASK & ~Backend::CUDA))
+   { return RajaCudaWrap3D(N, d_body, X, Y, Z); }
 #endif
 
 #ifdef MFEM_USE_CUDA

general/globals.hpp

@@ -107,4 +107,9 @@ void SetGlobalMPI_Comm(MPI_Comm comm);
 
 } // namespace mfem
 
+
+// Request a global object to be instantiated for each thread in its TLS.
+#define MFEM_THREAD_LOCAL thread_local
+
+
 #endif

general/hash.hpp

@@ -79,22 +79,22 @@ public:
 
    /// Get item whose parents are p1, p2... Create it if it doesn't exist.
    T* Get(int p1, int p2);
-   T* Get(int p1, int p2, int p3, int p4);
+   T* Get(int p1, int p2, int p3, int p4 = -1 /* p4 optional */);
 
    /// Get id of item whose parents are p1, p2... Create it if it doesn't exist.
    int GetId(int p1, int p2);
-   int GetId(int p1, int p2, int p3, int p4);
+   int GetId(int p1, int p2, int p3, int p4 = -1);
 
    /// Find item whose parents are p1, p2... Return NULL if it doesn't exist.
    T* Find(int p1, int p2);
-   T* Find(int p1, int p2, int p3, int p4);
+   T* Find(int p1, int p2, int p3, int p4 = -1);
 
    const T* Find(int p1, int p2) const;
-   const T* Find(int p1, int p2, int p3, int p4) const;
+   const T* Find(int p1, int p2, int p3, int p4 = -1) const;
 
    /// Find id of item whose parents are p1, p2... Return -1 if it doesn't exist.
    int FindId(int p1, int p2) const;
-   int FindId(int p1, int p2, int p3, int p4) const;
+   int FindId(int p1, int p2, int p3, int p4 = -1) const;
 
    /// Return the number of elements currently stored in the HashTable.
    int Size() const { return Base::Size() - unused.Size(); }
@@ -113,9 +113,12 @@ public:
    /** Its id will be reused by newly added items. */
    void Delete(int id);
 
+   /// Remove all items.
+   void DeleteAll();
+
    /// Make an item hashed under different parent IDs.
    void Reparent(int id, int new_p1, int new_p2);
-   void Reparent(int id, int new_p1, int new_p2, int new_p3, int new_p4);
+   void Reparent(int id, int new_p1, int new_p2, int new_p3, int new_p4 = -1);
 
    /// Return total size of allocated memory (tables plus items), in bytes.
    long MemoryUsage() const;
@@ -246,6 +249,18 @@ inline void sort4(int &a, int &b, int &c, int &d)
    sort3(b, c, d);
 }
 
+inline void sort4_ext(int &a, int &b, int &c, int &d)
+{
+   if (d < 0) // support optional last index
+   {
+      sort3(a, b, c);
+   }
+   else
+   {
+      sort4(a, b, c, d);
+   }
+}
+
 } // internal
 
 template<typename T>
@@ -295,7 +310,7 @@ template<typename T>
 int HashTable<T>::GetId(int p1, int p2, int p3, int p4)
 {
    // search for the item in the hashtable
-   internal::sort4(p1, p2, p3, p4);
+   internal::sort4_ext(p1, p2, p3, p4);
    int idx = Hash(p1, p2, p3);
    int id = SearchList(table[idx], p1, p2, p3);
    if (id >= 0) { return id; }
@@ -361,7 +376,7 @@ int HashTable<T>::FindId(int p1, int p2) const
 template<typename T>
 int HashTable<T>::FindId(int p1, int p2, int p3, int p4) const
 {
-   internal::sort4(p1, p2, p3, p4);
+   internal::sort4_ext(p1, p2, p3, p4);
    return SearchList(table[Hash(p1, p2, p3)], p1, p2, p3);
 }
 
@@ -459,6 +474,14 @@ void HashTable<T>::Delete(int id)
    unused.Append(id); // add its id to the unused ids
 }
 
+template<typename T>
+void HashTable<T>::DeleteAll()
+{
+   Base::DeleteAll();
+   for (int i = 0; i <= mask; i++) { table[i] = -1; }
+   unused.DeleteAll();
+}
+
 template<typename T>
 void HashTable<T>::Reparent(int id, int new_p1, int new_p2)
 {
@@ -481,7 +504,7 @@ void HashTable<T>::Reparent(int id,
    T& item = Base::At(id);
    Unlink(Hash(item), id);
 
-   internal::sort4(new_p1, new_p2, new_p3, new_p4);
+   internal::sort4_ext(new_p1, new_p2, new_p3, new_p4);
    item.p1 = new_p1;
    item.p2 = new_p2;
    item.p3 = new_p3;

general/mem_manager.cpp

@@ -11,7 +11,7 @@
 
 #include "../general/forall.hpp"
 
-#include <cstring> // std::memcpy
+#include <cstring> // std::memcpy, std::memcmp
 
 #include <list>
 #include <unordered_map>
@@ -57,6 +57,15 @@ MemoryClass operator*(MemoryClass mc1, MemoryClass mc2)
 }
 
 
+// Instantiate Memory<T>::PrintFlags for T = int and T = double.
+template void Memory<int>::PrintFlags() const;
+template void Memory<double>::PrintFlags() const;
+
+// Instantiate Memory<T>::CompareHostAndDevice for T = int and T = double.
+template int Memory<int>::CompareHostAndDevice(int size) const;
+template int Memory<double>::CompareHostAndDevice(int size) const;
+
+
 namespace internal
 {
 
@@ -695,6 +704,19 @@ void MemoryManager::CopyFromHost_(void *dest_h_ptr, const void *src_h_ptr,
                 ~(dest_on_host ? Mem::VALID_DEVICE : Mem::VALID_HOST);
 }
 
+int MemoryManager::CompareHostAndDevice_(void *h_ptr, size_t size,
+                                         unsigned flags)
+{
+   void *d_ptr = (flags & Mem::ALIAS) ?
+                 mm.GetAliasDevicePtr(h_ptr, size, false) :
+                 mm.GetDevicePtr(h_ptr, size, false);
+   char *h_buf = new char[size];
+   CuMemcpyDtoH(h_buf, d_ptr, size);
+   int res = std::memcmp(h_ptr, h_buf, size);
+   delete [] h_buf;
+   return res;
+}
+
 
 void MemoryPrintFlags(unsigned flags)
 {

general/mem_manager.hpp

@@ -377,6 +377,16 @@ public:
    /// Copy @a size entries from @a *this to the host pointer @a dest.
    /** The given @a size should not exceed the Capacity() of @a *this. */
    inline void CopyToHost(T *dest, int size) const;
+
+   /// Print the internal flags.
+   /** This method can be useful for debugging. It is explicitly instantiated
+       for Memory<T> with T = int and T = double. */
+   inline void PrintFlags() const;
+
+   /// If both the host and the device data are valid, compare their contents.
+   /** This method can be useful for debugging. It is explicitly instantiated
+       for Memory<T> with T = int and T = double. */
+   inline int CompareHostAndDevice(int size) const;
 };
 
 
@@ -447,6 +457,11 @@ private:
    static void CopyFromHost_(void *dest_h_ptr, const void *src_h_ptr,
                              std::size_t size, unsigned &dest_flags);
 
+   // Compare the contents of the host and the device memory - useful for
+   // debugging.
+   static int CompareHostAndDevice_(void *h_ptr, size_t size, unsigned flags);
+
+
    /// Adds an address in the map
    void *Insert(void *ptr, const std::size_t bytes);
 
@@ -727,10 +742,24 @@ inline void Memory<T>::CopyToHost(T *dest, int size) const
 
 
 /** @brief Print the state of a Memory object based on its internal flags.
-    Useful in a debugger. */
+    Useful in a debugger. See also Memory<T>::PrintFlags(). */
 extern void MemoryPrintFlags(unsigned flags);
 
 
+template <typename T>
+inline void Memory<T>::PrintFlags() const
+{
+   MemoryPrintFlags(flags);
+}
+
+template <typename T>
+inline int Memory<T>::CompareHostAndDevice(int size) const
+{
+   if (!(flags & VALID_HOST) || !(flags & VALID_DEVICE)) { return 0; }
+   return MemoryManager::CompareHostAndDevice_(h_ptr, size*sizeof(T), flags);
+}
+
+
 /// The (single) global memory manager object
 extern MemoryManager mm;
 

linalg/densemat.cpp

@@ -4456,7 +4456,8 @@ const
    int n = SizeI(), ne = SizeK();
    const int *I = elem_dof.GetI(), *J = elem_dof.GetJ(), *dofs;
    const double *d_col = tdata;
-   double *yp = y, x_col;
+   double *yp = y.HostReadWrite();
+   double x_col;
    const double *xp = x;
    // the '4' here can be tuned for given platform and compiler
    if (n <= 4)

linalg/densemat.hpp

@@ -721,11 +721,26 @@ public:
    { return const_cast<DenseTensor&>(*this)(k); }
 
    double &operator()(int i, int j, int k)
-   { return tdata[i+SizeI()*(j+SizeJ()*k)]; }
-   const double &operator()(int i, int j, int k) const
-   { return tdata[i+SizeI()*(j+SizeJ()*k)]; }
+   {
+      MFEM_ASSERT_INDEX_IN_RANGE(i, 0, SizeI());
+      MFEM_ASSERT_INDEX_IN_RANGE(j, 0, SizeJ());
+      MFEM_ASSERT_INDEX_IN_RANGE(k, 0, SizeK());
+      return tdata[i+SizeI()*(j+SizeJ()*k)];
+   }
 
-   double *GetData(int k) { return tdata+k*Mk.Height()*Mk.Width(); }
+   const double &operator()(int i, int j, int k) const
+   {
+      MFEM_ASSERT_INDEX_IN_RANGE(i, 0, SizeI());
+      MFEM_ASSERT_INDEX_IN_RANGE(j, 0, SizeJ());
+      MFEM_ASSERT_INDEX_IN_RANGE(k, 0, SizeK());
+      return tdata[i+SizeI()*(j+SizeJ()*k)];
+   }
+
+   double *GetData(int k)
+   {
+      MFEM_ASSERT_INDEX_IN_RANGE(k, 0, SizeK());
+      return tdata+k*Mk.Height()*Mk.Width();
+   }
 
    double *Data() { return tdata; }
 

linalg/hypre.cpp

@@ -185,7 +185,7 @@ Vector * HypreParVector::GlobalVector() const
 
 HypreParVector& HypreParVector::operator=(double d)
 {
-   hypre_ParVectorSetConstantValues(x,d);
+   Vector::operator=(d);
    return *this;
 }
 
@@ -198,10 +198,7 @@ HypreParVector& HypreParVector::operator=(const HypreParVector &y)
    }
 #endif
 
-   for (int i = 0; i < size; i++)
-   {
-      data[i] = y.data[i];
-   }
+   Vector::operator=(y);
    return *this;
 }
 
@@ -231,10 +228,6 @@ HypreParVector::~HypreParVector()
 
 #ifdef MFEM_USE_SUNDIALS
 
-#ifndef SUNFALSE
-#define SUNFALSE FALSE
-#endif
-
 void HypreParVector::ToNVector(N_Vector &nv)
 {
    MFEM_ASSERT(nv && N_VGetVectorID(nv) == SUNDIALS_NVEC_PARHYP,
@@ -1028,6 +1021,8 @@ HypreParMatrix * HypreParMatrix::Transpose() const
 HYPRE_Int HypreParMatrix::Mult(HypreParVector &x, HypreParVector &y,
                                double a, double b)
 {
+   x.HostRead();
+   (b == 0.0) ? y.HostWrite() : y.HostReadWrite();
    return hypre_ParCSRMatrixMatvec(a, A, x, b, y);
 }
 
@@ -1039,7 +1034,7 @@ void HypreParMatrix::Mult(double a, const Vector &x, double b, Vector &y) const
                << ", expected size = " << Height());
 
    auto x_data = x.HostRead();
-   auto y_data = y.HostWrite();
+   auto y_data = (b == 0.0) ? y.HostWrite() : y.HostReadWrite();
    if (X == NULL)
    {
       X = new HypreParVector(A->comm,
@@ -1071,7 +1066,7 @@ void HypreParMatrix::MultTranspose(double a, const Vector &x,
    // Note: x has the dimensions of Y (height), and
    //       y has the dimensions of X (width)
    auto x_data = x.HostRead();
-   auto y_data = y.HostWrite();
+   auto y_data = (b == 0.0) ? y.HostWrite() : y.HostReadWrite();
    if (X == NULL)
    {
       X = new HypreParVector(A->comm,
@@ -2013,10 +2008,12 @@ void HypreSmoother::Mult(const HypreParVector &b, HypreParVector &x) const
       return;
    }
 
+   b.HostRead();
    if (!iterative_mode)
    {
       if (type == 0 && relax_times == 1)
       {
+         x.HostWrite();
          HYPRE_ParCSRDiagScale(NULL, *A, b, x);
          if (relax_weight != 1.0)
          {
@@ -2026,6 +2023,7 @@ void HypreSmoother::Mult(const HypreParVector &b, HypreParVector &x) const
       }
       x = 0.0;
    }
+   x.HostReadWrite();
 
    if (V == NULL)
    {
@@ -2075,21 +2073,25 @@ void HypreSmoother::Mult(const Vector &b, Vector &x) const
       mfem_error("HypreSmoother::Mult (...) : HypreParMatrix A is missing");
       return;
    }
+
+   auto b_data = b.HostRead();
+   auto x_data = iterative_mode ? x.HostReadWrite() : x.HostWrite();
+
    if (B == NULL)
    {
       B = new HypreParVector(A->GetComm(),
                              A -> GetGlobalNumRows(),
-                             b.GetData(),
+                             const_cast<double*>(b_data),
                              A -> GetRowStarts());
       X = new HypreParVector(A->GetComm(),
                              A -> GetGlobalNumCols(),
-                             x.GetData(),
+                             x_data,
                              A -> GetColStarts());
    }
    else
    {
-      B -> SetData(b.GetData());
-      X -> SetData(x.GetData());
+      B -> SetData(const_cast<double*>(b_data));
+      X -> SetData(x_data);
    }
 
    Mult(*B, *X);
@@ -2206,7 +2208,14 @@ HypreSolver::~HypreSolver()
 }
 
 
-HyprePCG::HyprePCG(HypreParMatrix &_A) : HypreSolver(&_A)
+HyprePCG::HyprePCG(MPI_Comm comm) : precond(NULL)
+{
+   iterative_mode = true;
+
+   HYPRE_ParCSRPCGCreate(comm, &pcg_solver);
+}
+
+HyprePCG::HyprePCG(HypreParMatrix &_A) : HypreSolver(&_A), precond(NULL)
 {
    MPI_Comm comm;
 
@@ -2217,6 +2226,26 @@ HyprePCG::HyprePCG(HypreParMatrix &_A) : HypreSolver(&_A)
    HYPRE_ParCSRPCGCreate(comm, &pcg_solver);
 }
 
+void HyprePCG::SetOperator(const Operator &op)
+{
+   const HypreParMatrix *new_A = dynamic_cast<const HypreParMatrix *>(&op);
+   MFEM_VERIFY(new_A, "new Operator must be a HypreParMatrix!");
+
+   // update base classes: Operator, Solver, HypreSolver
+   height = new_A->Height();
+   width  = new_A->Width();
+   A = const_cast<HypreParMatrix *>(new_A);
+   if (precond)
+   {
+      precond->SetOperator(*A);
+      this->SetPreconditioner(*precond);
+   }
+   setup_called = 0;
+   delete X;
+   delete B;
+   B = X = NULL;
+}
+
 void HyprePCG::SetTol(double tol)
 {
    HYPRE_PCGSetTol(pcg_solver, tol);
@@ -2237,12 +2266,14 @@ void HyprePCG::SetPrintLevel(int print_lvl)
    HYPRE_ParCSRPCGSetPrintLevel(pcg_solver, print_lvl);
 }
 
-void HyprePCG::SetPreconditioner(HypreSolver &precond)
+void HyprePCG::SetPreconditioner(HypreSolver &_precond)
 {
+   precond = &_precond;
+
    HYPRE_ParCSRPCGSetPrecond(pcg_solver,
-                             precond.SolveFcn(),
-                             precond.SetupFcn(),
-                             precond);
+                             _precond.SolveFcn(),
+                             _precond.SetupFcn(),
+                             _precond);
 }
 
 void HyprePCG::SetResidualConvergenceOptions(int res_frequency, double rtol)
@@ -2340,24 +2371,57 @@ HyprePCG::~HyprePCG()
 }
 
 
+HypreGMRES::HypreGMRES(MPI_Comm comm) : precond(NULL)
+{
+   iterative_mode = true;
+
+   HYPRE_ParCSRGMRESCreate(comm, &gmres_solver);
+   SetDefaultOptions();
+}
+
 HypreGMRES::HypreGMRES(HypreParMatrix &_A) : HypreSolver(&_A)
 {
    MPI_Comm comm;
 
-   int k_dim    = 50;
-   int max_iter = 100;
-   double tol   = 1e-6;
-
    iterative_mode = true;
 
    HYPRE_ParCSRMatrixGetComm(*A, &comm);
 
    HYPRE_ParCSRGMRESCreate(comm, &gmres_solver);
+   SetDefaultOptions();
+}
+
+void HypreGMRES::SetDefaultOptions()
+{
+   int k_dim    = 50;
+   int max_iter = 100;
+   double tol   = 1e-6;
+
    HYPRE_ParCSRGMRESSetKDim(gmres_solver, k_dim);
    HYPRE_ParCSRGMRESSetMaxIter(gmres_solver, max_iter);
    HYPRE_ParCSRGMRESSetTol(gmres_solver, tol);
 }
 
+void HypreGMRES::SetOperator(const Operator &op)
+{
+   const HypreParMatrix *new_A = dynamic_cast<const HypreParMatrix *>(&op);
+   MFEM_VERIFY(new_A, "new Operator must be a HypreParMatrix!");
+
+   // update base classes: Operator, Solver, HypreSolver
+   height = new_A->Height();
+   width  = new_A->Width();
+   A = const_cast<HypreParMatrix *>(new_A);
+   if (precond)
+   {
+      precond->SetOperator(*A);
+      this->SetPreconditioner(*precond);
+   }
+   setup_called = 0;
+   delete X;
+   delete B;
+   B = X = NULL;
+}
+
 void HypreGMRES::SetTol(double tol)
 {
    HYPRE_GMRESSetTol(gmres_solver, tol);
@@ -2383,12 +2447,14 @@ void HypreGMRES::SetPrintLevel(int print_lvl)
    HYPRE_GMRESSetPrintLevel(gmres_solver, print_lvl);
 }
 
-void HypreGMRES::SetPreconditioner(HypreSolver &precond)
+void HypreGMRES::SetPreconditioner(HypreSolver &_precond)
 {
+   precond = &_precond;
+
    HYPRE_ParCSRGMRESSetPrecond(gmres_solver,
-                               precond.SolveFcn(),
-                               precond.SetupFcn(),
-                               precond);
+                               _precond.SolveFcn(),
+                               _precond.SetupFcn(),
+                               _precond);
 }
 
 void HypreGMRES::Mult(const HypreParVector &b, HypreParVector &x) const
@@ -2465,10 +2531,40 @@ HypreGMRES::~HypreGMRES()
 }
 
 
+void HypreDiagScale::SetOperator(const Operator &op)
+{
+   const HypreParMatrix *new_A = dynamic_cast<const HypreParMatrix *>(&op);
+   MFEM_VERIFY(new_A, "new Operator must be a HypreParMatrix!");
+
+   // update base classes: Operator, Solver, HypreSolver
+   height = new_A->Height();
+   width  = new_A->Width();
+   A = const_cast<HypreParMatrix *>(new_A);
+   setup_called = 0;
+   delete X;
+   delete B;
+   B = X = NULL;
+}
+
+
+HypreParaSails::HypreParaSails(MPI_Comm comm)
+{
+   HYPRE_ParaSailsCreate(comm, &sai_precond);
+   SetDefaultOptions();
+}
+
 HypreParaSails::HypreParaSails(HypreParMatrix &A) : HypreSolver(&A)
 {
    MPI_Comm comm;
 
+   HYPRE_ParCSRMatrixGetComm(A, &comm);
+
+   HYPRE_ParaSailsCreate(comm, &sai_precond);
+   SetDefaultOptions();
+}
+
+void HypreParaSails::SetDefaultOptions()
+{
    int    sai_max_levels = 1;
    double sai_threshold  = 0.1;
    double sai_filter     = 0.1;
@@ -2477,9 +2573,6 @@ HypreParaSails::HypreParaSails(HypreParMatrix &A) : HypreSolver(&A)
    int    sai_reuse      = 0;
    int    sai_logging    = 1;
 
-   HYPRE_ParCSRMatrixGetComm(A, &comm);
-
-   HYPRE_ParaSailsCreate(comm, &sai_precond);
    HYPRE_ParaSailsSetParams(sai_precond, sai_threshold, sai_max_levels);
    HYPRE_ParaSailsSetFilter(sai_precond, sai_filter);
    HYPRE_ParaSailsSetSym(sai_precond, sai_sym);
@@ -2488,6 +2581,58 @@ HypreParaSails::HypreParaSails(HypreParMatrix &A) : HypreSolver(&A)
    HYPRE_ParaSailsSetLogging(sai_precond, sai_logging);
 }
 
+void HypreParaSails::ResetSAIPrecond(MPI_Comm comm)
+{
+   HYPRE_Int  sai_max_levels;
+   HYPRE_Real sai_threshold;
+   HYPRE_Real sai_filter;
+   HYPRE_Int  sai_sym;
+   HYPRE_Real sai_loadbal;
+   HYPRE_Int  sai_reuse;
+   HYPRE_Int  sai_logging;
+
+   // hypre_ParAMGData *amg_data = (hypre_ParAMGData *)sai_precond;
+   HYPRE_ParaSailsGetNlevels(sai_precond, &sai_max_levels);
+   HYPRE_ParaSailsGetThresh(sai_precond, &sai_threshold);
+   HYPRE_ParaSailsGetFilter(sai_precond, &sai_filter);
+   HYPRE_ParaSailsGetSym(sai_precond, &sai_sym);
+   HYPRE_ParaSailsGetLoadbal(sai_precond, &sai_loadbal);
+   HYPRE_ParaSailsGetReuse(sai_precond, &sai_reuse);
+   HYPRE_ParaSailsGetLogging(sai_precond, &sai_logging);
+
+   HYPRE_ParaSailsDestroy(sai_precond);
+   HYPRE_ParaSailsCreate(comm, &sai_precond);
+
+   HYPRE_ParaSailsSetParams(sai_precond, sai_threshold, sai_max_levels);
+   HYPRE_ParaSailsSetFilter(sai_precond, sai_filter);
+   HYPRE_ParaSailsSetSym(sai_precond, sai_sym);
+   HYPRE_ParaSailsSetLoadbal(sai_precond, sai_loadbal);
+   HYPRE_ParaSailsSetReuse(sai_precond, sai_reuse);
+   HYPRE_ParaSailsSetLogging(sai_precond, sai_logging);
+}
+
+void HypreParaSails::SetOperator(const Operator &op)
+{
+   const HypreParMatrix *new_A = dynamic_cast<const HypreParMatrix *>(&op);
+   MFEM_VERIFY(new_A, "new Operator must be a HypreParMatrix!");
+
+   if (A)
+   {
+      MPI_Comm comm;
+      HYPRE_ParCSRMatrixGetComm(*A, &comm);
+      ResetSAIPrecond(comm);
+   }
+
+   // update base classes: Operator, Solver, HypreSolver
+   height = new_A->Height();
+   width  = new_A->Width();
+   A = const_cast<HypreParMatrix *>(new_A);
+   setup_called = 0;
+   delete X;
+   delete B;
+   B = X = NULL;
+}
+
 void HypreParaSails::SetSymmetry(int sym)
 {
    HYPRE_ParaSailsSetSym(sai_precond, sym);
@@ -2499,19 +2644,30 @@ HypreParaSails::~HypreParaSails()
 }
 
 
+HypreEuclid::HypreEuclid(MPI_Comm comm)
+{
+   HYPRE_EuclidCreate(comm, &euc_precond);
+   SetDefaultOptions();
+}
+
 HypreEuclid::HypreEuclid(HypreParMatrix &A) : HypreSolver(&A)
 {
    MPI_Comm comm;
 
+   HYPRE_ParCSRMatrixGetComm(A, &comm);
+
+   HYPRE_EuclidCreate(comm, &euc_precond);
+   SetDefaultOptions();
+}
+
+void HypreEuclid::SetDefaultOptions()
+{
    int    euc_level = 1; // We use ILU(1)
    int    euc_stats = 0; // No logging
    int    euc_mem   = 0; // No memory logging
    int    euc_bj    = 0; // 1: Use Block Jacobi
    int    euc_ro_sc = 0; // 1: Use Row scaling
 
-   HYPRE_ParCSRMatrixGetComm(A, &comm);
-
-   HYPRE_EuclidCreate(comm, &euc_precond);
    HYPRE_EuclidSetLevel(euc_precond, euc_level);
    HYPRE_EuclidSetStats(euc_precond, euc_stats);
    HYPRE_EuclidSetMem(euc_precond, euc_mem);
@@ -2519,6 +2675,38 @@ HypreEuclid::HypreEuclid(HypreParMatrix &A) : HypreSolver(&A)
    HYPRE_EuclidSetRowScale(euc_precond, euc_ro_sc);
 }
 
+void HypreEuclid::ResetEuclidPrecond(MPI_Comm comm)
+{
+   // Euclid does not seem to offer access to its current configuration, so we
+   // simply reset it to its default options.
+   HYPRE_EuclidDestroy(euc_precond);
+   HYPRE_EuclidCreate(comm, &euc_precond);
+
+   SetDefaultOptions();
+}
+
+void HypreEuclid::SetOperator(const Operator &op)
+{
+   const HypreParMatrix *new_A = dynamic_cast<const HypreParMatrix *>(&op);
+   MFEM_VERIFY(new_A, "new Operator must be a HypreParMatrix!");
+
+   if (A)
+   {
+      MPI_Comm comm;
+      HYPRE_ParCSRMatrixGetComm(*new_A, &comm);
+      ResetEuclidPrecond(comm);
+   }
+
+   // update base classes: Operator, Solver, HypreSolver
+   height = new_A->Height();
+   width  = new_A->Width();
+   A = const_cast<HypreParMatrix *>(new_A);
+   setup_called = 0;
+   delete X;
+   delete B;
+   B = X = NULL;
+}
+
 HypreEuclid::~HypreEuclid()
 {
    HYPRE_EuclidDestroy(euc_precond);
@@ -2794,9 +2982,18 @@ HypreBoomerAMG::~HypreBoomerAMG()
    HYPRE_BoomerAMGDestroy(amg_precond);
 }
 
+HypreAMS::HypreAMS(ParFiniteElementSpace *edge_fespace)
+{
+   Init(edge_fespace);
+}
 
 HypreAMS::HypreAMS(HypreParMatrix &A, ParFiniteElementSpace *edge_fespace)
    : HypreSolver(&A)
+{
+   Init(edge_fespace);
+}
+
+void HypreAMS::Init(ParFiniteElementSpace *edge_fespace)
 {
    int cycle_type       = 13;
    int rlx_type         = 2;
@@ -2980,6 +3177,22 @@ HypreAMS::HypreAMS(HypreParMatrix &A, ParFiniteElementSpace *edge_fespace)
    error_mode = IGNORE_HYPRE_ERRORS;
 }
 
+void HypreAMS::SetOperator(const Operator &op)
+{
+   const HypreParMatrix *new_A = dynamic_cast<const HypreParMatrix *>(&op);
+   MFEM_VERIFY(new_A, "new Operator must be a HypreParMatrix!");
+
+   // update base classes: Operator, Solver, HypreSolver
+   height = new_A->Height();
+   width  = new_A->Width();
+   A = const_cast<HypreParMatrix *>(new_A);
+
+   setup_called = 0;
+   delete X;
+   delete B;
+   B = X = NULL;
+}
+
 HypreAMS::~HypreAMS()
 {
    HYPRE_AMSDestroy(ams);
@@ -3000,8 +3213,18 @@ void HypreAMS::SetPrintLevel(int print_lvl)
    HYPRE_AMSSetPrintLevel(ams, print_lvl);
 }
 
+HypreADS::HypreADS(ParFiniteElementSpace *face_fespace)
+{
+   Init(face_fespace);
+}
+
 HypreADS::HypreADS(HypreParMatrix &A, ParFiniteElementSpace *face_fespace)
    : HypreSolver(&A)
+{
+   Init(face_fespace);
+}
+
+void HypreADS::Init(ParFiniteElementSpace *face_fespace)
 {
    int cycle_type       = 11;
    int rlx_type         = 2;
@@ -3224,6 +3447,22 @@ HypreADS::HypreADS(HypreParMatrix &A, ParFiniteElementSpace *face_fespace)
    error_mode = IGNORE_HYPRE_ERRORS;
 }
 
+void HypreADS::SetOperator(const Operator &op)
+{
+   const HypreParMatrix *new_A = dynamic_cast<const HypreParMatrix *>(&op);
+   MFEM_VERIFY(new_A, "new Operator must be a HypreParMatrix!");
+
+   // update base classes: Operator, Solver, HypreSolver
+   height = new_A->Height();
+   width  = new_A->Width();
+   A = const_cast<HypreParMatrix *>(new_A);
+
+   setup_called = 0;
+   delete X;
+   delete B;
+   B = X = NULL;
+}
+
 HypreADS::~HypreADS()
 {
    HYPRE_ADSDestroy(ads);

linalg/hypre.hpp

@@ -735,9 +735,15 @@ class HyprePCG : public HypreSolver
 private:
    HYPRE_Solver pcg_solver;
 
+   HypreSolver * precond;
+
 public:
+   HyprePCG(MPI_Comm comm);
+
    HyprePCG(HypreParMatrix &_A);
 
+   virtual void SetOperator(const Operator &op);
+
    void SetTol(double tol);
    void SetMaxIter(int max_iter);
    void SetLogging(int logging);
@@ -784,9 +790,18 @@ class HypreGMRES : public HypreSolver
 private:
    HYPRE_Solver gmres_solver;
 
+   HypreSolver * precond;
+
+   /// Default, generally robust, GMRES options
+   void SetDefaultOptions();
+
 public:
+   HypreGMRES(MPI_Comm comm);
+
    HypreGMRES(HypreParMatrix &_A);
 
+   virtual void SetOperator(const Operator &op);
+
    void SetTol(double tol);
    void SetMaxIter(int max_iter);
    void SetKDim(int dim);
@@ -838,6 +853,8 @@ public:
    explicit HypreDiagScale(HypreParMatrix &A) : HypreSolver(&A) { }
    virtual operator HYPRE_Solver() const { return NULL; }
 
+   virtual void SetOperator(const Operator &op);
+
    virtual HYPRE_PtrToParSolverFcn SetupFcn() const
    { return (HYPRE_PtrToParSolverFcn) HYPRE_ParCSRDiagScaleSetup; }
    virtual HYPRE_PtrToParSolverFcn SolveFcn() const
@@ -853,9 +870,21 @@ class HypreParaSails : public HypreSolver
 private:
    HYPRE_Solver sai_precond;
 
+   /// Default, generally robust, ParaSails options
+   void SetDefaultOptions();
+
+   // If sai_precond is NULL, this method allocates it and sets default options.
+   // Otherwise the method saves the options from sai_precond, destroys it,
+   // allocates a new object, and sets its options to the saved values.
+   void ResetSAIPrecond(MPI_Comm comm);
+
 public:
+   HypreParaSails(MPI_Comm comm);
+
    HypreParaSails(HypreParMatrix &A);
 
+   virtual void SetOperator(const Operator &op);
+
    void SetSymmetry(int sym);
 
    /// The typecast to HYPRE_Solver returns the internal sai_precond
@@ -882,9 +911,21 @@ class HypreEuclid : public HypreSolver
 private:
    HYPRE_Solver euc_precond;
 
+   /// Default, generally robust, Euclid options
+   void SetDefaultOptions();
+
+   // If euc_precond is NULL, this method allocates it and sets default options.
+   // Otherwise the method saves the options from euc_precond, destroys it,
+   // allocates a new object, and sets its options to the saved values.
+   void ResetEuclidPrecond(MPI_Comm comm);
+
 public:
+   HypreEuclid(MPI_Comm comm);
+
    HypreEuclid(HypreParMatrix &A);
 
+   virtual void SetOperator(const Operator &op);
+
    /// The typecast to HYPRE_Solver returns the internal euc_precond
    virtual operator HYPRE_Solver() const { return euc_precond; }
 
@@ -963,6 +1004,9 @@ HypreParMatrix* DiscreteCurl(ParFiniteElementSpace *face_fespace,
 class HypreAMS : public HypreSolver
 {
 private:
+   /// Constuct AMS solver from finite element space
+   void Init(ParFiniteElementSpace *edge_space);
+
    HYPRE_Solver ams;
 
    /// Vertex coordinates
@@ -973,8 +1017,12 @@ private:
    HypreParMatrix *Pi, *Pix, *Piy, *Piz;
 
 public:
+   HypreAMS(ParFiniteElementSpace *edge_fespace);
+
    HypreAMS(HypreParMatrix &A, ParFiniteElementSpace *edge_fespace);
 
+   virtual void SetOperator(const Operator &op);
+
    void SetPrintLevel(int print_lvl);
 
    /// Set this option when solving a curl-curl problem with zero mass term
@@ -995,6 +1043,9 @@ public:
 class HypreADS : public HypreSolver
 {
 private:
+   /// Constuct ADS solver from finite element space
+   void Init(ParFiniteElementSpace *face_fespace);
+
    HYPRE_Solver ads;
 
    /// Vertex coordinates
@@ -1009,8 +1060,12 @@ private:
    HypreParMatrix *RT_Pi, *RT_Pix, *RT_Piy, *RT_Piz;
 
 public:
+   HypreADS(ParFiniteElementSpace *face_fespace);
+
    HypreADS(HypreParMatrix &A, ParFiniteElementSpace *face_fespace);
 
+   virtual void SetOperator(const Operator &op);
+
    void SetPrintLevel(int print_lvl);
 
    /// The typecast to HYPRE_Solver returns the internal ads object

linalg/operator.cpp

@@ -135,6 +135,76 @@ void Operator::PrintMatlab(std::ostream & out, int n, int m) const
 }
 
 
+void TimeDependentOperator::ExplicitMult(const Vector &, Vector &) const
+{
+   mfem_error("TimeDependentOperator::ExplicitMult() is not overridden!");
+}
+
+void TimeDependentOperator::ImplicitMult(const Vector &, const Vector &,
+                                         Vector &) const
+{
+   mfem_error("TimeDependentOperator::ImplicitMult() is not overridden!");
+}
+
+void TimeDependentOperator::Mult(const Vector &, Vector &) const
+{
+   mfem_error("TimeDependentOperator::Mult() is not overridden!");
+}
+
+void TimeDependentOperator::ImplicitSolve(const double, const Vector &,
+                                          Vector &)
+{
+   mfem_error("TimeDependentOperator::ImplicitSolve() is not overridden!");
+}
+
+Operator &TimeDependentOperator::GetImplicitGradient(
+   const Vector &, const Vector &, double) const
+{
+   mfem_error("TimeDependentOperator::GetImplicitGradient() is "
+              "not overridden!");
+   return const_cast<Operator &>(dynamic_cast<const Operator &>(*this));
+}
+
+Operator &TimeDependentOperator::GetExplicitGradient(const Vector &) const
+{
+   mfem_error("TimeDependentOperator::GetExplicitGradient() is "
+              "not overridden!");
+   return const_cast<Operator &>(dynamic_cast<const Operator &>(*this));
+}
+
+int TimeDependentOperator::SUNImplicitSetup(const Vector &,
+                                            const Vector &,
+                                            int, int *, double)
+{
+   mfem_error("TimeDependentOperator::SUNImplicitSetup() is not overridden!");
+   return (-1);
+}
+
+int TimeDependentOperator::SUNImplicitSolve(const Vector &, Vector &, double)
+{
+   mfem_error("TimeDependentOperator::SUNImplicitSolve() is not overridden!");
+   return (-1);
+}
+
+int TimeDependentOperator::SUNMassSetup()
+{
+   mfem_error("TimeDependentOperator::SUNMassSetup() is not overridden!");
+   return (-1);
+}
+
+int TimeDependentOperator::SUNMassSolve(const Vector &, Vector &, double)
+{
+   mfem_error("TimeDependentOperator::SUNMassSolve() is not overridden!");
+   return (-1);
+}
+
+int TimeDependentOperator::SUNMassMult(const Vector &, Vector &)
+{
+   mfem_error("TimeDependentOperator::SUNMassMult() is not overridden!");
+   return (-1);
+}
+
+
 ProductOperator::ProductOperator(const Operator *A, const Operator *B,
                                  bool ownA, bool ownB)
    : Operator(A->Height(), B->Width()),

linalg/operator.hpp

@@ -188,7 +188,7 @@ public:
 /** Operator of the form: (x,t) -> f(x,t), where k = f(x,t) generally solves the
     algebraic equation F(x,k,t) = G(x,t). The functions F and G represent the
     _implicit_ and _explicit_ parts of the operator, respectively. For explicit
-    operators, F(x,k,t) = k, so f(x,t) = G(x,t).*/
+    operators, F(x,k,t) = k, so f(x,t) = G(x,t). */
 class TimeDependentOperator : public Operator
 {
 public:
@@ -199,21 +199,35 @@ public:
       HOMOGENEOUS ///< This type assumes that G(x,t) = 0.
    };
 
+   /// Evaluation mode. See SetEvalMode() for details.
+   enum EvalMode
+   {
+      /** Normal evaluation. */
+      NORMAL,
+      /** Assuming additive split, f(x,t) = f1(x,t) + f2(x,t), evaluate the
+          first term, f1. */
+      ADDITIVE_TERM_1,
+      /** Assuming additive split, f(x,t) = f1(x,t) + f2(x,t), evaluate the
+          second term, f2. */
+      ADDITIVE_TERM_2
+   };
+
 protected:
    double t;  ///< Current time.
    Type type; ///< Describes the form of the TimeDependentOperator.
+   EvalMode eval_mode; ///< Current evaluation mode.
 
 public:
    /** @brief Construct a "square" TimeDependentOperator y = f(x,t), where x and
        y have the same dimension @a n. */
    explicit TimeDependentOperator(int n = 0, double t_ = 0.0,
                                   Type type_ = EXPLICIT)
-      : Operator(n) { t = t_; type = type_; }
+      : Operator(n) { t = t_; type = type_; eval_mode = NORMAL; }
 
    /** @brief Construct a TimeDependentOperator y = f(x,t), where x and y have
        dimensions @a w and @a h, respectively. */
    TimeDependentOperator(int h, int w, double t_ = 0.0, Type type_ = EXPLICIT)
-      : Operator(h, w) { t = t_; type = type_; }
+      : Operator(h, w) { t = t_; type = type_; eval_mode = NORMAL; }
 
    /// Read the currently set time.
    virtual double GetTime() const { return t; }
@@ -228,33 +242,41 @@ public:
    /// True if #type is #HOMOGENEOUS.
    bool isHomogeneous() const { return (type == HOMOGENEOUS); }
 
+   /// Return the current evaluation mode. See SetEvalMode() for details.
+   EvalMode GetEvalMode() const { return eval_mode; }
+
+   /// Set the evaluation mode of the time-dependent operator.
+   /** The evaluation mode is a switch that allows time-stepping methods to
+       request evaluation of separate components/terms of the time-dependent
+       operator. For example, IMEX methods typically assume additive split of
+       the operator: f(x,t) = f1(x,t) + f2(x,t) and they rely on the ability to
+       evaluate the two terms separately.
+
+       Generally, setting the evaluation mode should affect the behavior of all
+       evaluation-related methods in the class, such as Mult(), ImplicitSolve(),
+       etc. However, the exact list of methods that need to support a specific
+       mode will depend on the used time-stepping method. */
+   virtual void SetEvalMode(const EvalMode new_eval_mode)
+   { eval_mode = new_eval_mode; }
+
    /** @brief Perform the action of the explicit part of the operator, G:
        @a y = G(@a x, t) where t is the current time.
 
        Presently, this method is used by some PETSc ODE solvers, for more
        details, see the PETSc Manual. */
-   virtual void ExplicitMult(const Vector &x, Vector &y) const
-   {
-      mfem_error("TimeDependentOperator::ExplicitMult() is not overridden!");
-   }
+   virtual void ExplicitMult(const Vector &x, Vector &y) const;
 
    /** @brief Perform the action of the implicit part of the operator, F:
        @a y = F(@a x, @a k, t) where t is the current time.
 
        Presently, this method is used by some PETSc ODE solvers, for more
        details, see the PETSc Manual.*/
-   virtual void ImplicitMult(const Vector &x, const Vector &k, Vector &y) const
-   {
-      mfem_error("TimeDependentOperator::ImplicitMult() is not overridden!");
-   }
+   virtual void ImplicitMult(const Vector &x, const Vector &k, Vector &y) const;
 
    /** @brief Perform the action of the operator: @a y = k = f(@a x, t), where
        k solves the algebraic equation F(@a x, k, t) = G(@a x, t) and t is the
        current time. */
-   virtual void Mult(const Vector &x, Vector &y) const
-   {
-      mfem_error("TimeDependentOperator::Mult() is not overridden!");
-   }
+   virtual void Mult(const Vector &x, Vector &y) const;
 
    /** @brief Solve the equation: @a k = f(@a x + @a dt @a k, t), for the
        unknown @a k at the current time t.
@@ -272,10 +294,7 @@ public:
        methods and the backward Euler method in particular.
 
        If not re-implemented, this method simply generates an error. */
-   virtual void ImplicitSolve(const double dt, const Vector &x, Vector &k)
-   {
-      mfem_error("TimeDependentOperator::ImplicitSolve() is not overridden!");
-   }
+   virtual void ImplicitSolve(const double dt, const Vector &x, Vector &k);
 
    /** @brief Return an Operator representing (dF/dk @a shift + dF/dx) at the
        given @a x, @a k, and the currently set time.
@@ -283,24 +302,75 @@ public:
        Presently, this method is used by some PETSc ODE solvers, for more
        details, see the PETSc Manual. */
    virtual Operator& GetImplicitGradient(const Vector &x, const Vector &k,
-                                         double shift) const
-   {
-      mfem_error("TimeDependentOperator::GetImplicitGradient() is "
-                 "not overridden!");
-      return const_cast<Operator &>(dynamic_cast<const Operator &>(*this));
-   }
+                                         double shift) const;
 
    /** @brief Return an Operator representing dG/dx at the given point @a x and
        the currently set time.
 
        Presently, this method is used by some PETSc ODE solvers, for more
        details, see the PETSc Manual. */
-   virtual Operator& GetExplicitGradient(const Vector &x) const
-   {
-      mfem_error("TimeDependentOperator::GetExplicitGradient() is "
-                 "not overridden!");
-      return const_cast<Operator &>(dynamic_cast<const Operator &>(*this));
-   }
+   virtual Operator& GetExplicitGradient(const Vector &x) const;
+
+   /** @brief Setup the ODE linear system \f$ A(x,t) = (I - gamma J) \f$ or
+       \f$ A = (M - gamma J) \f$, where \f$ J(x,t) = \frac{df}{dt(x,t)} \f$.
+
+       @param[in]  x     The state at which \f$A(x,t)\f$ should be evaluated.
+       @param[in]  fx    The current value of the ODE rhs function, \f$f(x,t)\f$.
+       @param[in]  jok   Flag indicating if the Jacobian should be updated.
+       @param[out] jcur  Flag to signal if the Jacobian was updated.
+       @param[in]  gamma The scaled time step value.
+
+       If not re-implemented, this method simply generates an error.
+
+       Presently, this method is used by SUNDIALS ODE solvers, for more
+       details, see the SUNDIALS User Guides. */
+   virtual int SUNImplicitSetup(const Vector &x, const Vector &fx,
+                                int jok, int *jcur, double gamma);
+
+   /** @brief Solve the ODE linear system \f$ A x = b \f$ as setup by
+       the method SUNImplicitSetup().
+
+       @param[in]      b   The linear system right-hand side.
+       @param[in,out]  x   On input, the initial guess. On output, the solution.
+       @param[in]      tol Linear solve tolerance.
+
+       If not re-implemented, this method simply generates an error.
+
+       Presently, this method is used by SUNDIALS ODE solvers, for more
+       details, see the SUNDIALS User Guides. */
+   virtual int SUNImplicitSolve(const Vector &b, Vector &x, double tol);
+
+   /** @brief Setup the mass matrix in the ODE system \f$ M y' = f(y,t) \f$ .
+
+       If not re-implemented, this method simply generates an error.
+
+       Presently, this method is used by SUNDIALS ARKStep integrator, for more
+       details, see the ARKode User Guide. */
+   virtual int SUNMassSetup();
+
+   /** @brief Solve the mass matrix linear system \f$ M x = b \f$
+       as setup by the method SUNMassSetup().
+
+       @param[in]      b   The linear system right-hand side.
+       @param[in,out]  x   On input, the initial guess. On output, the solution.
+       @param[in]      tol Linear solve tolerance.
+
+       If not re-implemented, this method simply generates an error.
+
+       Presently, this method is used by SUNDIALS ARKStep integrator, for more
+       details, see the ARKode User Guide. */
+   virtual int SUNMassSolve(const Vector &b, Vector &x, double tol);
+
+   /** @brief Compute the mass matrix-vector productv \f$ v = M x \f$ .
+
+       @param[in]   x The vector to multiply.
+       @param[out]  v The result of the matrix-vector product.
+
+       If not re-implemented, this method simply generates an error.
+
+       Presently, this method is used by SUNDIALS ARKStep integrator, for more
+       details, see the ARKode User Guide. */
+   virtual int SUNMassMult(const Vector &x, Vector &v);
 
    virtual ~TimeDependentOperator() { }
 };

linalg/petsc.cpp

@@ -422,6 +422,12 @@ PetscParVector& PetscParVector::operator*=(PetscScalar s)
    return *this;
 }
 
+PetscParVector& PetscParVector::operator+=(PetscScalar s)
+{
+   ierr = VecShift(x,s); PCHKERRQ(x,ierr);
+   return *this;
+}
+
 void PetscParVector::PlaceArray(PetscScalar *temp_data)
 {
    ierr = VecPlaceArray(x,temp_data); PCHKERRQ(x,ierr);
@@ -2303,17 +2309,6 @@ void PetscLinearSolver::SetOperator(const Operator &op)
                               (dynamic_cast<const PetscParMatrix *>(&op));
    const Operator       *oA = dynamic_cast<const Operator *>(&op);
 
-   // Preserve Pmat if already set
-   KSP ksp = (KSP)obj;
-   Mat P = NULL;
-   PetscBool pmat;
-   ierr = KSPGetOperatorsSet(ksp,NULL,&pmat); PCHKERRQ(ksp,ierr);
-   if (pmat)
-   {
-      ierr = KSPGetOperators(ksp,NULL,&P); PCHKERRQ(ksp,ierr);
-      ierr = PetscObjectReference((PetscObject)P); PCHKERRQ(ksp,ierr);
-   }
-
    // update base classes: Operator, Solver, PetscLinearSolver
    bool delete_pA = false;
    if (!pA)
@@ -2336,6 +2331,7 @@ void PetscLinearSolver::SetOperator(const Operator &op)
    MFEM_VERIFY(pA, "Unsupported operation!");
 
    // Set operators into PETSc KSP
+   KSP ksp = (KSP)obj;
    Mat A = pA->A;
    if (operatorset)
    {
@@ -2356,15 +2352,7 @@ void PetscLinearSolver::SetOperator(const Operator &op)
          wrap = false;
       }
    }
-   if (P)
-   {
-      ierr = KSPSetOperators(ksp,A,P); PCHKERRQ(ksp,ierr);
-      ierr = MatDestroy(&P); PCHKERRQ(ksp,ierr);
-   }
-   else
-   {
-      ierr = KSPSetOperators(ksp,A,A); PCHKERRQ(ksp,ierr);
-   }
+   ierr = KSPSetOperators(ksp,A,A); PCHKERRQ(ksp,ierr);
 
    // Update PetscSolver
    operatorset = true;
@@ -2500,7 +2488,7 @@ void PetscLinearSolver::SetPreconditioner(Solver &precond)
    }
 }
 
-void PetscLinearSolver::Mult(const Vector &b, Vector &x) const
+void PetscLinearSolver::MultKernel(const Vector &b, Vector &x, bool trans) const
 {
    KSP ksp = (KSP)obj;
 
@@ -2528,11 +2516,28 @@ void PetscLinearSolver::Mult(const Vector &b, Vector &x) const
    PCHKERRQ(ksp, ierr);
 
    // Solve the system.
-   ierr = KSPSolve(ksp, B->x, X->x); PCHKERRQ(ksp,ierr);
+   if (trans)
+   {
+      ierr = KSPSolveTranspose(ksp, B->x, X->x); PCHKERRQ(ksp,ierr);
+   }
+   else
+   {
+      ierr = KSPSolve(ksp, B->x, X->x); PCHKERRQ(ksp,ierr);
+   }
    B->ResetArray();
    X->ResetArray();
 }
 
+void PetscLinearSolver::Mult(const Vector &b, Vector &x) const
+{
+   (*this).MultKernel(b,x,false);
+}
+
+void PetscLinearSolver::MultTranspose(const Vector &b, Vector &x) const
+{
+   (*this).MultKernel(b,x,true);
+}
+
 PetscLinearSolver::~PetscLinearSolver()
 {
    MPI_Comm comm;
@@ -2654,7 +2659,8 @@ void PetscPreconditioner::SetOperator(const Operator &op)
    if (delete_pA) { delete pA; };
 }
 
-void PetscPreconditioner::Mult(const Vector &b, Vector &x) const
+void PetscPreconditioner::MultKernel(const Vector &b, Vector &x,
+                                     bool trans) const
 {
    PC pc = (PC)obj;
 
@@ -2679,11 +2685,28 @@ void PetscPreconditioner::Mult(const Vector &b, Vector &x) const
    Customize();
 
    // Apply the preconditioner.
-   ierr = PCApply(pc, B->x, X->x); PCHKERRQ(pc, ierr);
+   if (trans)
+   {
+      ierr = PCApplyTranspose(pc, B->x, X->x); PCHKERRQ(pc, ierr);
+   }
+   else
+   {
+      ierr = PCApply(pc, B->x, X->x); PCHKERRQ(pc, ierr);
+   }
    B->ResetArray();
    X->ResetArray();
 }
 
+void PetscPreconditioner::Mult(const Vector &b, Vector &x) const
+{
+   (*this).MultKernel(b,x,false);
+}
+
+void PetscPreconditioner::MultTranspose(const Vector &b, Vector &x) const
+{
+   (*this).MultKernel(b,x,true);
+}
+
 PetscPreconditioner::~PetscPreconditioner()
 {
    MPI_Comm comm;
@@ -3188,26 +3211,27 @@ PetscFieldSplitSolver::PetscFieldSplitSolver(MPI_Comm comm, Operator &op,
    : PetscPreconditioner(comm,op,prefix)
 {
    PC pc = (PC)obj;
+   ierr = PCSetType(pc,PCFIELDSPLIT); PCHKERRQ(pc,ierr);
 
    Mat pA;
    ierr = PCGetOperators(pc,&pA,NULL); PCHKERRQ(pc,ierr);
 
    // Check if pA is of type MATNEST
-   // (this requirement can be removed when we can pass fields).
    PetscBool isnest;
    ierr = PetscObjectTypeCompare((PetscObject)pA,MATNEST,&isnest);
-   PCHKERRQ(pA,ierr);
-   MFEM_VERIFY(isnest,
-               "PetscFieldSplitSolver needs the matrix in nested format.");
 
-   PetscInt nr;
-   IS  *isrow;
-   ierr = PCSetType(pc,PCFIELDSPLIT); PCHKERRQ(pc,ierr);
-   ierr = MatNestGetSize(pA,&nr,NULL); PCHKERRQ(pc,ierr);
-   ierr = PetscCalloc1(nr,&isrow); CCHKERRQ(PETSC_COMM_SELF,ierr);
-   ierr = MatNestGetISs(pA,isrow,NULL); PCHKERRQ(pc,ierr);
+   PetscInt nr = 0;
+   IS  *isrow = NULL;
+   if (isnest) // we now the fields
+   {
+      ierr = MatNestGetSize(pA,&nr,NULL); PCHKERRQ(pc,ierr);
+      ierr = PetscCalloc1(nr,&isrow); CCHKERRQ(PETSC_COMM_SELF,ierr);
+      ierr = MatNestGetISs(pA,isrow,NULL); PCHKERRQ(pc,ierr);
+   }
 
    // We need to customize here, before setting the index sets.
+   // This is because PCFieldSplitSetType customizes the function
+   // pointers. SubSolver options will be processed during PCApply
    Customize();
 
    for (PetscInt i=0; i<nr; i++)
@@ -3575,6 +3599,7 @@ void PetscODESolver::Run(Vector &x, double &t, double &dt, double t_final)
 }  // namespace mfem
 
 #include "petsc/private/petscimpl.h"
+#include "petsc/private/matimpl.h"
 
 // auxiliary functions
 static PetscErrorCode __mfem_ts_monitor(TS ts, PetscInt it, PetscReal t, Vec x,
@@ -3730,17 +3755,37 @@ static PetscErrorCode __mfem_ts_ijacobian(TS ts, PetscReal t, Vec x,
       pA->EliminateRowsCols(bchandler->GetTDofs(),dummy,dummy);
    }
 
+   // Get nonzerostate
+   PetscObjectState nonzerostate;
+   ierr = MatGetNonzeroState(P,&nonzerostate); CHKERRQ(ierr);
+
    // Avoid unneeded copy of the matrix by hacking
    Mat B;
    B = pA->ReleaseMat(false);
    ierr = MatHeaderReplace(P,&B); CHKERRQ(ierr);
    if (delete_pA) { delete pA; }
+
+   // Matrix-free case
    if (A && A != P)
    {
       ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);
       ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);
    }
 
+   // When using MATNEST and PCFIELDSPLIT, the second setup of the
+   // preconditioner fails because MatCreateSubMatrix_Nest does not
+   // actually return a matrix. Instead, for efficiency reasons,
+   // it returns a reference to the submatrix. The second time it
+   // is called, MAT_REUSE_MATRIX is used and MatCreateSubMatrix_Nest
+   // aborts since the two submatrices are actually different.
+   // We circumvent this issue by incrementing the nonzero state
+   // (i.e. PETSc thinks the operator sparsity pattern has changed)
+   // This does not impact performances in the case of MATNEST
+   PetscBool isnest;
+   ierr = PetscObjectTypeCompare((PetscObject)P,MATNEST,&isnest);
+   CHKERRQ(ierr);
+   if (isnest) { P->nonzerostate = nonzerostate + 1; }
+
    // Jacobian reusage
    ierr = PetscObjectStateGet((PetscObject)P,&ts_ctx->cached_ijacstate);
    CHKERRQ(ierr);
@@ -3897,6 +3942,7 @@ static PetscErrorCode __mfem_ts_computesplits(TS ts,PetscReal t,Vec x,Vec xp,
       ierr = MatAXPY(*pJxp,-1.0,*pJx,SAME_NONZERO_PATTERN); PCHKERRQ(ts,ierr);
    }
 
+   // Matrix-free cases
    if (Ax && Ax != Jx)
    {
       ierr = MatAssemblyBegin(Ax,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);
@@ -3983,12 +4029,31 @@ static PetscErrorCode __mfem_ts_rhsjacobian(TS ts, PetscReal t, Vec x,
       pA->EliminateRowsCols(bchandler->GetTDofs(),dummy,dummy);
    }
 
+   // Get nonzerostate
+   PetscObjectState nonzerostate;
+   ierr = MatGetNonzeroState(P,&nonzerostate); CHKERRQ(ierr);
+
    // Avoid unneeded copy of the matrix by hacking
    Mat B;
    B = pA->ReleaseMat(false);
    ierr = MatHeaderReplace(P,&B); CHKERRQ(ierr);
    if (delete_pA) { delete pA; }
 
+   // When using MATNEST and PCFIELDSPLIT, the second setup of the
+   // preconditioner fails because MatCreateSubMatrix_Nest does not
+   // actually return a matrix. Instead, for efficiency reasons,
+   // it returns a reference to the submatrix. The second time it
+   // is called, MAT_REUSE_MATRIX is used and MatCreateSubMatrix_Nest
+   // aborts since the two submatrices are actually different.
+   // We circumvent this issue by incrementing the nonzero state
+   // (i.e. PETSc thinks the operator sparsity pattern has changed)
+   // This does not impact performances in the case of MATNEST
+   PetscBool isnest;
+   ierr = PetscObjectTypeCompare((PetscObject)P,MATNEST,&isnest);
+   CHKERRQ(ierr);
+   if (isnest) { P->nonzerostate = nonzerostate + 1; }
+
+   // Matrix-free case
    if (A && A != P)
    {
       ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);
@@ -4092,10 +4157,30 @@ static PetscErrorCode __mfem_snes_jacobian(SNES snes, Vec x, Mat A, Mat P,
       pA->EliminateRowsCols(bchandler->GetTDofs(),dummy,dummy);
    }
 
+   // Get nonzerostate
+   PetscObjectState nonzerostate;
+   ierr = MatGetNonzeroState(P,&nonzerostate); CHKERRQ(ierr);
+
    // Avoid unneeded copy of the matrix by hacking
    Mat B = pA->ReleaseMat(false);
    ierr = MatHeaderReplace(P,&B); CHKERRQ(ierr);
    if (delete_pA) { delete pA; }
+
+   // When using MATNEST and PCFIELDSPLIT, the second setup of the
+   // preconditioner fails because MatCreateSubMatrix_Nest does not
+   // actually return a matrix. Instead, for efficiency reasons,
+   // it returns a reference to the submatrix. The second time it
+   // is called, MAT_REUSE_MATRIX is used and MatCreateSubMatrix_Nest
+   // aborts since the two submatrices are actually different.
+   // We circumvent this issue by incrementing the nonzero state
+   // (i.e. PETSc thinks the operator sparsity pattern has changed)
+   // This does not impact performances in the case of MATNEST
+   PetscBool isnest;
+   ierr = PetscObjectTypeCompare((PetscObject)P,MATNEST,&isnest);
+   CHKERRQ(ierr);
+   if (isnest) { P->nonzerostate = nonzerostate + 1; }
+
+   // Matrix-free case
    if (A && A != P)
    {
       ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY); CHKERRQ(ierr);
@@ -4443,6 +4528,11 @@ PetscErrorCode MakeShellPC(PC pc, mfem::Solver &precond, bool ownsop)
    ctx->factory = NULL;
    ctx->numprec = 0;
 
+   // In case the PC was already of type SHELL, this will destroy any
+   // previous user-defined data structure
+   // We cannot call PCReset as it will wipe out any operator already set
+   ierr = PCSetType(pc,PCNONE); CHKERRQ(ierr);
+
    ierr = PCSetType(pc,PCSHELL); CHKERRQ(ierr);
    ierr = PCShellSetName(pc,"MFEM Solver (unknown Pmat)"); CHKERRQ(ierr);
    ierr = PCShellSetContext(pc,(void *)ctx); CHKERRQ(ierr);
@@ -4467,6 +4557,11 @@ PetscErrorCode MakeShellPCWithFactory(PC pc,
    ctx->factory = factory;
    ctx->numprec = 0;
 
+   // In case the PC was already of type SHELL, this will destroy any
+   // previous user-defined data structure
+   // We cannot call PCReset as it will wipe out any operator already set
+   ierr = PCSetType(pc,PCNONE); CHKERRQ(ierr);
+
    ierr = PCSetType(pc,PCSHELL); CHKERRQ(ierr);
    ierr = PCShellSetName(pc,factory->GetName()); CHKERRQ(ierr);
    ierr = PCShellSetContext(pc,(void *)ctx); CHKERRQ(ierr);

linalg/petsc.hpp

@@ -170,6 +170,7 @@ public:
    PetscParVector& operator+= (const PetscParVector &y);
    PetscParVector& operator-= (const PetscParVector &y);
    PetscParVector& operator*= (PetscScalar d);
+   PetscParVector& operator+= (PetscScalar d);
 
    /** @brief Temporarily replace the data of the PETSc Vec object. To return to
        the original data array, call ResetArray().
@@ -601,6 +602,7 @@ class PetscLinearSolver : public PetscSolver, public Solver
 private:
    /// Internal flag to handle HypreParMatrix conversion or not.
    bool wrap;
+   void MultKernel(const Vector &b, Vector &x, bool trans) const;
 
 public:
    PetscLinearSolver(MPI_Comm comm, const std::string &prefix = std::string(),
@@ -616,6 +618,8 @@ public:
                      const std::string &prefix = std::string());
    virtual ~PetscLinearSolver();
 
+   /// Sets the operator to be used for mat-vec operations and
+   /// for the construction of the preconditioner
    virtual void SetOperator(const Operator &op);
 
    /// Allows to prescribe a different operator (@a pop) to construct
@@ -623,10 +627,12 @@ public:
    void SetOperator(const Operator &op, const Operator &pop);
 
    /// Sets the solver to perform preconditioning
+   /// preserves the linear operator for the mat-vec
    void SetPreconditioner(Solver &precond);
 
    /// Application of the solver.
    virtual void Mult(const Vector &b, Vector &x) const;
+   virtual void MultTranspose(const Vector &b, Vector &x) const;
 
    /// Conversion function to PETSc's KSP type.
    operator KSP() const { return (KSP)obj; }
@@ -646,6 +652,9 @@ public:
 /// Abstract class for PETSc's preconditioners.
 class PetscPreconditioner : public PetscSolver, public Solver
 {
+private:
+   void MultKernel(const Vector &b, Vector &x, bool trans) const;
+
 public:
    PetscPreconditioner(MPI_Comm comm,
                        const std::string &prefix = std::string());
@@ -659,6 +668,7 @@ public:
 
    /// Application of the preconditioner.
    virtual void Mult(const Vector &b, Vector &x) const;
+   virtual void MultTranspose(const Vector &b, Vector &x) const;
 
    /// Conversion function to PETSc's PC type.
    operator PC() const { return (PC)obj; }

linalg/solvers.cpp

@@ -1311,6 +1311,8 @@ void NewtonSolver::Mult(const Vector &b, Vector &x) const
       }
       add(x, -c_scale, c, x);
 
+      ProcessNewState(x);
+
       oper->Mult(x, r);
       if (have_b)
       {

linalg/solvers.hpp

@@ -283,6 +283,10 @@ public:
        value of 0 indicates a failure, interrupting the Newton iteration. */
    virtual double ComputeScalingFactor(const Vector &x, const Vector &b) const
    { return 1.0; }
+
+   /** @brief This method can be overloaded in derived classes to perform
+       computations that need knowledge of the newest Newton state. */
+   virtual void ProcessNewState(const Vector &x) const { }
 };
 
 /** Adaptive restarted GMRES.

linalg/sparsemat.cpp

@@ -2640,9 +2640,11 @@ SparseMatrix &SparseMatrix::operator=(double a)
 {
    if (Rows == NULL)
    {
-      for (int i = 0, nnz = I[height]; i < nnz; i++)
+      const int nnz = J.Capacity();
+      double *h_A = HostWrite(A, nnz);
+      for (int i = 0; i < nnz; i++)
       {
-         A[i] = a;
+         h_A[i] = a;
       }
    }
    else

linalg/sparsemat.hpp

@@ -152,6 +152,54 @@ public:
    /// Return the element data, i.e. the array #A, const version.
    inline const double *GetData() const { return A; }
 
+   // Memory access methods for the #I array.
+   Memory<int> &GetMemoryI() { return I; }
+   const Memory<int> &GetMemoryI() const { return I; }
+   const int *ReadI(bool on_dev = true) const
+   { return mfem::Read(I, Height()+1, on_dev); }
+   int *WriteI(bool on_dev = true)
+   { return mfem::Write(I, Height()+1, on_dev); }
+   int *ReadWriteI(bool on_dev = true)
+   { return mfem::ReadWrite(I, Height()+1, on_dev); }
+   const int *HostReadI() const
+   { return mfem::Read(I, Height()+1, false); }
+   int *HostWriteI()
+   { return mfem::Write(I, Height()+1, false); }
+   int *HostReadWriteI()
+   { return mfem::ReadWrite(I, Height()+1, false); }
+
+   // Memory access methods for the #J array.
+   Memory<int> &GetMemoryJ() { return J; }
+   const Memory<int> &GetMemoryJ() const { return J; }
+   const int *ReadJ(bool on_dev = true) const
+   { return mfem::Read(J, J.Capacity(), on_dev); }
+   int *WriteJ(bool on_dev = true)
+   { return mfem::Write(J, J.Capacity(), on_dev); }
+   int *ReadWriteJ(bool on_dev = true)
+   { return mfem::ReadWrite(J, J.Capacity(), on_dev); }
+   const int *HostReadJ() const
+   { return mfem::Read(J, J.Capacity(), false); }
+   int *HostWriteJ()
+   { return mfem::Write(J, J.Capacity(), false); }
+   int *HostReadWriteJ()
+   { return mfem::ReadWrite(J, J.Capacity(), false); }
+
+   // Memory access methods for the #A array.
+   Memory<double> &GetMemoryData() { return A; }
+   const Memory<double> &GetMemoryData() const { return A; }
+   const double *ReadData(bool on_dev = true) const
+   { return mfem::Read(A, A.Capacity(), on_dev); }
+   double *WriteData(bool on_dev = true)
+   { return mfem::Write(A, A.Capacity(), on_dev); }
+   double *ReadWriteData(bool on_dev = true)
+   { return mfem::ReadWrite(A, A.Capacity(), on_dev); }
+   const double *HostReadData() const
+   { return mfem::Read(A, A.Capacity(), false); }
+   double *HostWriteData()
+   { return mfem::Write(A, A.Capacity(), false); }
+   double *HostReadWriteData()
+   { return mfem::ReadWrite(A, A.Capacity(), false); }
+
    /// Returns the number of elements in row @a i.
    int RowSize(const int i) const;
 

linalg/sundials.hpp

@@ -23,384 +23,458 @@
 #include "ode.hpp"
 #include "solvers.hpp"
 
+#include <sundials/sundials_config.h>
+// Check for appropriate SUNDIALS version
+#if !defined(SUNDIALS_VERSION_MAJOR) || (SUNDIALS_VERSION_MAJOR < 5)
+#error MFEM requires SUNDIALS version 5.0.0 or newer!
+#endif
+#include <sundials/sundials_matrix.h>
+#include <sundials/sundials_linearsolver.h>
 #include <cvode/cvode.h>
-#include <arkode/arkode.h>
+#include <arkode/arkode_arkstep.h>
 #include <kinsol/kinsol.h>
 
-struct KINMemRec;
-
 namespace mfem
 {
 
-/** @brief Abstract base class, wrapping the custom linear solvers interface in
-    SUNDIALS' CVODE and ARKODE solvers. */
-/** For a given ODE system
+// ---------------------------------------------------------------------------
+// Base class for interfacing with SUNDIALS packages
+// ---------------------------------------------------------------------------
 
-        dx/dt = f(x,t)
-
-    the purpose of this class is to facilitate the (approximate) solution of
-    linear systems of the form
-
-        (I - γJ) y = b,   J = J(x,t) = df/dx
-
-    for given b, x, t and γ, where γ = GetTimeStep() is a scaled time step. */
-class SundialsODELinearSolver
-{
-public:
-   enum {CVODE, ARKODE} type; ///< Is CVODE or ARKODE using this object?
-
-protected:
-   SundialsODELinearSolver() { }
-   virtual ~SundialsODELinearSolver() { }
-
-   /// Get the current scaled time step, gamma, from @a sundials_mem.
-   double GetTimeStep(void *sundials_mem);
-   /// Get the TimeDependentOperator associated with @a sundials_mem.
-   TimeDependentOperator *GetTimeDependentOperator(void *sundials_mem);
-
-public:
-   /** @name Linear solver interface methods.
-       These four functions and their parameters are documented in Section 7 of
-       http://computation.llnl.gov/sites/default/files/public/cv_guide.pdf
-       and Section 7.4 of
-       http://computation.llnl.gov/sites/default/files/public/ark_guide.pdf
-
-       The first argument, @a sundials_mem, is one of the pointer types,
-       CVodeMem or ARKodeMem, depending on the value of the data member @a type.
-   */
-   ///@{
-   virtual int InitSystem(void *sundials_mem) = 0;
-   virtual int SetupSystem(void *sundials_mem, int conv_fail,
-                           const Vector &y_pred, const Vector &f_pred,
-                           int &jac_cur, Vector &v_temp1,
-                           Vector &v_temp2, Vector &v_temp3) = 0;
-   virtual int SolveSystem(void *sundials_mem, Vector &b, const Vector &w,
-                           const Vector &y_cur, const Vector &f_cur) = 0;
-   virtual int FreeSystem(void *sundials_mem) = 0;
-   ///@}
-};
-
-
-/// A base class for the MFEM classes wrapping SUNDIALS' solvers.
-/** This class defines some common data and functions used by the SUNDIALS
-    solvers, e.g the common @a #sundials_mem pointer and return @a #flag. */
+/// Base class for interfacing with SUNDIALS packages.
 class SundialsSolver
 {
 protected:
-   void *sundials_mem; ///< Pointer to the SUNDIALS mem object.
-   mutable int flag;   ///< Flag returned by the last call to SUNDIALS.
+   void *sundials_mem;     ///< SUNDIALS mem structure.
+   mutable int flag;       ///< Last flag returned from a call to SUNDIALS.
+   bool reinit;            ///< Flag to signal memory reinitialization is need.
+   long saved_global_size; ///< Global vector length on last initialization.
+
+   N_Vector           y;   ///< State vector.
+   SUNMatrix          A;   ///< Linear system A = I - gamma J, M - gamma J, or J.
+   SUNMatrix          M;   ///< Mass matrix M.
+   SUNLinearSolver    LSA; ///< Linear solver for A.
+   SUNLinearSolver    LSM; ///< Linear solver for M.
+   SUNNonlinearSolver NLS; ///< Nonlinear solver.
 
-   N_Vector y;  ///< Auxiliary N_Vector.
 #ifdef MFEM_USE_MPI
    bool Parallel() const
-   { return (y->ops->nvgetvectorid != N_VGetVectorID_Serial); }
+   { return (N_VGetVectorID(y) != SUNDIALS_NVEC_SERIAL); }
 #else
    bool Parallel() const { return false; }
 #endif
 
-   static const double default_rel_tol;
-   static const double default_abs_tol;
+   /// Default scalar relative tolerance.
+   static constexpr double default_rel_tol = 1e-4;
+   /// Default scalar absolute tolerance.
+   static constexpr double default_abs_tol = 1e-9;
 
-   // Computes the action of a time-dependent operator.
-   /// Callback function used in CVODESolver and ARKODESolver.
-   static int ODEMult(realtype t, const N_Vector y,
-                      N_Vector ydot, void *td_oper);
-
-   /// @name The constructors are protected
-   ///@{
-   SundialsSolver() : sundials_mem(NULL) { }
-   SundialsSolver(void *mem) : sundials_mem(mem) { }
-   ///@}
+   /** @brief Protected constructor: objects of this type should be constructed
+       only as part of a derived class. */
+   SundialsSolver() : sundials_mem(NULL), flag(0), reinit(false),
+      saved_global_size(0), y(NULL), A(NULL), M(NULL),
+      LSA(NULL), LSM(NULL), NLS(NULL) { }
 
 public:
-   /// Access the underlying SUNDIALS object.
-   void *SundialsMem() const { return sundials_mem; }
+   /// Access the SUNDIALS memory structure.
+   void *GetMem() const { return sundials_mem; }
 
-   /// Return the flag returned by the last call to a SUNDIALS function.
+   /// Returns the last flag retured by a call to a SUNDIALS function.
    int GetFlag() const { return flag; }
 };
 
-/// Wrapper for SUNDIALS' CVODE library -- Multi-step time integration.
-/**
-   - http://computation.llnl.gov/projects/sundials
-   - http://computation.llnl.gov/sites/default/files/public/cv_guide.pdf
 
-   @note All methods except Step() can be called before Init().
-   To minimize uncertainty, we advise the user to adhere to the given
-   interface, instead of making similar calls by the CVODE's
-   internal CVodeMem object.
-*/
+// ---------------------------------------------------------------------------
+// Interface to the CVODE library -- linear multi-step methods
+// ---------------------------------------------------------------------------
+
+/// Interface to the CVODE library -- linear multi-step methods.
 class CVODESolver : public ODESolver, public SundialsSolver
 {
+protected:
+   int lmm_type;  ///< Linear multistep method type.
+   int step_mode; ///< CVODE step mode (CV_NORMAL or CV_ONE_STEP).
+
+   /// Wrapper to compute the ODE rhs function.
+   static int RHS(realtype t, const N_Vector y, N_Vector ydot, void *user_data);
+
+   /// Setup the linear system \f$ A x = b \f$.
+   static int LinSysSetup(realtype t, N_Vector y, N_Vector fy, SUNMatrix A,
+                          booleantype jok, booleantype *jcur,
+                          realtype gamma, void *user_data, N_Vector tmp1,
+                          N_Vector tmp2, N_Vector tmp3);
+
+   /// Solve the linear system \f$ A x = b \f$.
+   static int LinSysSolve(SUNLinearSolver LS, SUNMatrix A, N_Vector x,
+                          N_Vector b, realtype tol);
+
 public:
-   /// Construct a serial CVODESolver, a wrapper for SUNDIALS' CVODE solver.
-   /** @param[in] lmm   Specifies the linear multistep method, the options are
-                        CV_ADAMS (explicit methods) or CV_BDF (implicit
-                        methods).
-       @param[in] iter  Specifies type of nonlinear solver iteration, the
-                        options are CV_FUNCTIONAL (usually with CV_ADAMS) or
-                        CV_NEWTON (usually with CV_BDF).
-       For parameter desciption, see the CVodeCreate documentation (cvode.h). */
-   CVODESolver(int lmm, int iter);
+   /// Construct a serial wrapper to SUNDIALS' CVODE integrator.
+   /** @param[in] lmm Specifies the linear multistep method, the options are:
+                      - CV_ADAMS - implicit methods for non-stiff systems,
+                      - CV_BDF   - implicit methods for stiff systems. */
+   CVODESolver(int lmm);
 
 #ifdef MFEM_USE_MPI
-   /// Construct a parallel CVODESolver, a wrapper for SUNDIALS' CVODE solver.
-   /** @param[in] comm  The MPI communicator used to partition the ODE system.
-       @param[in] lmm   Specifies the linear multistep method, the options are
-                        CV_ADAMS (explicit methods) or CV_BDF (implicit
-                        methods).
-       @param[in] iter  Specifies type of nonlinear solver iteration, the
-                        options are CV_FUNCTIONAL (usually with CV_ADAMS) or
-                        CV_NEWTON (usually with CV_BDF).
-       For parameter desciption, see the CVodeCreate documentation (cvode.h). */
-   CVODESolver(MPI_Comm comm, int lmm, int iter);
+   /// Construct a parallel wrapper to SUNDIALS' CVODE integrator.
+   /** @param[in] comm The MPI communicator used to partition the ODE system
+       @param[in] lmm  Specifies the linear multistep method, the options are:
+                       - CV_ADAMS - implicit methods for non-stiff systems,
+                       - CV_BDF   - implicit methods for stiff systems. */
+   CVODESolver(MPI_Comm comm, int lmm);
 #endif
 
+   /** @brief Initialize CVODE: calls CVodeCreate() to create the CVODE
+       memory and set some defaults.
+
+       If the CVODE memory has already been created, it checks if the problem
+       size has changed since the last call to Init(). If the problem is the
+       same then CVodeReInit() will be called in the next call to Step(). If
+       the problem size has changed, the CVODE memory is freed and realloced
+       for the new problem size. */
+   /** @param[in] f_ The TimeDependentOperator that defines the ODE system.
+
+       @note All other methods must be called after Init().
+
+       @note If this method is called a second time with a different problem
+       size, then any non-default user-set options will be lost and will need
+       to be set again. */
+   void Init(TimeDependentOperator &f_);
+
+   /// Integrate the ODE with CVODE using the specified step mode.
+   /** @param[in,out] x  On output, the solution vector at the requested output
+                         time tout = @a t + @a dt.
+       @param[in,out] t  On output, the output time reached.
+       @param[in,out] dt On output, the last time step taken.
+
+       @note On input, the values of @a t and @a dt are used to compute desired
+       output time for the integration, tout = @a t + @a dt.
+   */
+   virtual void Step(Vector &x, double &t, double &dt);
+
+   /** @brief Attach the linear system setup and solve methods from the
+       TimeDependentOperator i.e., SUNImplicitSetup() and SUNImplicitSolve() to
+       CVODE.
+   */
+   void UseMFEMLinearSolver();
+
+   /// Attach SUNDIALS GMRES linear solver to CVODE.
+   void UseSundialsLinearSolver();
+
+   /// Select the CVODE step mode: CV_NORMAL (default) or CV_ONE_STEP.
+   /** @param[in] itask  The desired step mode. */
+   void SetStepMode(int itask);
+
    /// Set the scalar relative and scalar absolute tolerances.
    void SetSStolerances(double reltol, double abstol);
 
-   /// Set a custom Jacobian system solver for the CV_NEWTON option usually used
-   /// with implicit CV_BDF.
-   void SetLinearSolver(SundialsODELinearSolver &ls_spec);
+   /// Set the maximum time step.
+   void SetMaxStep(double dt_max);
 
-   /** @brief CVode supports two modes, specified by itask: CV_NORMAL (default)
-       and CV_ONE_STEP. */
-   /** In the CV_NORMAL mode, the solver steps until it reaches or passes
-       tout = t + dt, where t and dt are specified in Step(), and then
-       interpolates to obtain y(tout). In the CV_ONE_STEP mode, it takes one
-       internal step and returns. */
-   void SetStepMode(int itask);
+   /** @brief Set the maximum method order.
 
-   /// Set the maximum order of the linear multistep method.
-   /** The default is 12 (CV_ADAMS) or 5 (CV_BDF).
        CVODE uses adaptive-order integration, based on the local truncation
-       error. Use this if you know a priori that your system is such that
-       higher order integration formulas are unstable.
+       error. The default values for @a max_order are 12 for CV_ADAMS and
+       5 for CV_BDF. Use this if you know a priori that your system is such
+       that higher order integration formulas are unstable.
+
        @note @a max_order can't be higher than the current maximum order. */
    void SetMaxOrder(int max_order);
 
-   /// Set the maximum time step of the linear multistep method.
-   void SetMaxStep(double dt_max)
-   { flag = CVodeSetMaxStep(sundials_mem, dt_max); }
-
-   /// Set the ODE right-hand-side operator.
-   /** The start time of CVODE is initialized from the current time of @a f_.
-       @note This method calls CVodeInit(). Some CVODE parameters can be set
-       (using the handle returned by SundialsMem()) only after this call. */
-   virtual void Init(TimeDependentOperator &f_);
-
-   /// Use CVODE to integrate over [t, t + dt], with the specified step mode.
-   /** Calls CVode(), which is the main driver of the CVODE package.
-       @param[in,out] x  Solution vector to advance. On input/output x=x(t)
-                         for t corresponding to the input/output value of t,
-                         respectively.
-       @param[in,out] t  Input: the starting time value. Output: the time value
-                         of the solution output, as returned by CVode().
-       @param[in,out] dt Input: desired time step. Output: the last incremental
-                         time step used. */
-   virtual void Step(Vector &x, double &t, double &dt);
-
-   /// Print CVODE statistics.
+   /// Print various CVODE statistics.
    void PrintInfo() const;
 
-   /// Destroy the associated CVODE memory.
+   /// Destroy the associated CVODE memory and SUNDIALS objects.
    virtual ~CVODESolver();
 };
 
-/// Wrapper for SUNDIALS' ARKODE library -- Runge-Kutta time integration.
-/**
-  - http://computation.llnl.gov/projects/sundials
-  - http://computation.llnl.gov/sites/default/files/public/ark_guide.pdf
 
-   @note All methods except Step() can be called before Init().
-   To minimize uncertainty, we advise the user to adhere to the given
-   interface, instead of making similar calls by the ARKODE's
-   internal ARKodeMem object.
-*/
-class ARKODESolver : public ODESolver, public SundialsSolver
+// ---------------------------------------------------------------------------
+// Interface to ARKode's ARKStep module -- Additive Runge-Kutta methods
+// ---------------------------------------------------------------------------
+
+/// Interface to ARKode's ARKStep module -- additive Runge-Kutta methods.
+class ARKStepSolver : public ODESolver, public SundialsSolver
 {
-protected:
-   bool use_implicit;
-   int irk_table, erk_table;
-
 public:
    /// Types of ARKODE solvers.
-   enum Type { EXPLICIT, IMPLICIT };
+   enum Type
+   {
+      EXPLICIT, ///< Explicit RK method
+      IMPLICIT, ///< Implicit RK method
+      IMEX      ///< Implicit-explicit ARK method
+   };
 
-   /// Construct a serial ARKODESolver, a wrapper for SUNDIALS' ARKODE solver.
-   /** @param[in] type  Specifies the #Type of ARKODE solver to construct. */
-   ARKODESolver(Type type = EXPLICIT);
+protected:
+   Type rk_type;      ///< Runge-Kutta type.
+   int step_mode;     ///< ARKStep step mode (ARK_NORMAL or ARK_ONE_STEP).
+   bool use_implicit; ///< True for implicit or imex integration.
+
+   /** @name Wrappers to compute the ODE RHS functions.
+       RHS1 is explicit RHS and RHS2 the implicit RHS for IMEX integration. When
+       purely implicit or explicit only RHS1 is used. */
+   ///@{
+   static int RHS1(realtype t, const N_Vector y, N_Vector ydot, void *user_data);
+   static int RHS2(realtype t, const N_Vector y, N_Vector ydot, void *user_data);
+   ///@}
+
+   /// Setup the linear system \f$ A x = b \f$.
+   static int LinSysSetup(realtype t, N_Vector y, N_Vector fy, SUNMatrix A,
+                          SUNMatrix M, booleantype jok, booleantype *jcur,
+                          realtype gamma, void *user_data, N_Vector tmp1,
+                          N_Vector tmp2, N_Vector tmp3);
+
+   /// Solve the linear system \f$ A x = b \f$.
+   static int LinSysSolve(SUNLinearSolver LS, SUNMatrix A, N_Vector x,
+                          N_Vector b, realtype tol);
+
+   /// Setup the linear system \f$ M x = b \f$.
+   static int MassSysSetup(realtype t, SUNMatrix M, void *user_data,
+                           N_Vector tmp1, N_Vector tmp2, N_Vector tmp3);
+
+   /// Solve the linear system \f$ M x = b \f$.
+   static int MassSysSolve(SUNLinearSolver LS, SUNMatrix M, N_Vector x,
+                           N_Vector b, realtype tol);
+
+   /// Compute the matrix-vector product \f$ v = M x \f$.
+   static int MassMult1(SUNMatrix M, N_Vector x, N_Vector v);
+
+   /// Compute the matrix-vector product \f$v = M_t x \f$ at time t.
+   static int MassMult2(N_Vector x, N_Vector v, realtype t,
+                        void* mtimes_data);
+
+public:
+   /// Construct a serial wrapper to SUNDIALS' ARKode integrator.
+   /** @param[in] type Specifies the RK method type:
+                       - EXPLICIT - explicit RK method (default)
+                       - IMPLICIT - implicit RK method
+                       - IMEX     - implicit-explicit ARK method */
+   ARKStepSolver(Type type = EXPLICIT);
 
 #ifdef MFEM_USE_MPI
-   /// Construct a parallel ARKODESolver, a wrapper for SUNDIALS' ARKODE solver.
-   /** @param[in] comm  The MPI communicator used to partition the ODE system.
-       @param[in] type  Specifies the #Type of ARKODE solver to construct. */
-   ARKODESolver(MPI_Comm comm, Type type = EXPLICIT);
+   /// Construct a parallel wrapper to SUNDIALS' ARKode integrator.
+   /** @param[in] comm The MPI communicator used to partition the ODE system.
+       @param[in] type Specifies the RK method type:
+                       - EXPLICIT - explicit RK method (default)
+                       - IMPLICIT - implicit RK method
+                       - IMEX     - implicit-explicit ARK method */
+   ARKStepSolver(MPI_Comm comm, Type type = EXPLICIT);
 #endif
 
-   /// Specify the scalar relative and scalar absolute tolerances.
-   void SetSStolerances(double reltol, double abstol);
+   /** @brief Initialize ARKode: calls ARKStepCreate() to create the ARKStep
+       memory and set some defaults.
 
-   /// Set a custom Jacobian system solver for implicit methods.
-   void SetLinearSolver(SundialsODELinearSolver &ls_spec);
+       If the ARKStep has already been created, it checks if the problem size
+       has changed since the last call to Init(). If the problem is the same
+       then ARKStepReInit() will be called in the next call to Step(). If the
+       problem size has changed, the ARKStep memory is freed and realloced
+       for the new problem size. */
+   /** @param[in] f_ The TimeDependentOperator that defines the ODE system
 
-   /** @brief ARKode supports two modes, specified by itask: ARK_NORMAL
-       (default) and ARK_ONE_STEP. */
-   /** In the ARK_NORMAL mode, the solver steps until it reaches or passes
-       tout = t + dt, where t and dt are specified in Step(), and then
-       interpolates to obtain y(tout). In the ARK_ONE_STEP mode, it takes one
-       internal step and returns. */
+       @note All other methods must be called after Init().
+
+       @note If this method is called a second time with a different problem
+       size, then any non-default user-set options will be lost and will need
+       to be set again. */
+   void Init(TimeDependentOperator &f_);
+
+   /// Integrate the ODE with ARKode using the specified step mode.
+   /**
+       @param[in,out] x  On output, the solution vector at the requested output
+                         time, tout = @a t + @a dt
+       @param[in,out] t  On output, the output time reached
+       @param[in,out] dt On output, the last time step taken
+
+       @note On input, the values of @a t and @a dt are used to compute desired
+       output time for the integration, tout = @a t + @a dt.
+   */
+   virtual void Step(Vector &x, double &t, double &dt);
+
+   /** @brief Attach the linear system setup and solve methods from the
+       TimeDependentOperator i.e., SUNImplicitSetup() and SUNImplicitSolve() to
+       ARKode.
+   */
+   void UseMFEMLinearSolver();
+
+   /// Attach a SUNDIALS GMRES linear solver to ARKode.
+   void UseSundialsLinearSolver();
+
+   /** @brief Attach mass matrix linear system setup, solve, and matrix-vector
+       product methods from the TimeDependentOperator i.e., SUNMassSetup(),
+       SUNMassSolve(), and SUNMassMult() to ARKode.
+
+       @param[in] tdep    An integer flag indicating if the mass matrix is time
+                          dependent (1) or time independent (0)
+   */
+   void UseMFEMMassLinearSolver(int tdep);
+
+   /** @brief Attach the SUNDIALS GMRES linear solver and the mass matrix
+       matrix-vector product method from the TimeDependentOperator i.e.,
+       SUNMassMult() to ARKode to solve mass matrix systems.
+
+       @param[in] tdep    An integer flag indicating if the mass matrix is time
+                          dependent (1) or time independent (0)
+   */
+   void UseSundialsMassLinearSolver(int tdep);
+
+   /// Select the ARKode step mode: ARK_NORMAL (default) or ARK_ONE_STEP.
+   /** @param[in] itask  The desired step mode */
    void SetStepMode(int itask);
 
+   /// Set the scalar relative and scalar absolute tolerances.
+   void SetSStolerances(double reltol, double abstol);
+
+   /// Set the maximum time step.
+   void SetMaxStep(double dt_max);
+
    /// Chooses integration order for all explicit / implicit / IMEX methods.
-   /** The default is 4, and the allowed ranges are: [2, 8] for explicit; [2, 5]
-       for implicit; [3, 5] for IMEX. */
+   /** The default is 4, and the allowed ranges are: [2, 8] for explicit;
+       [2, 5] for implicit; [3, 5] for IMEX. */
    void SetOrder(int order);
 
-   /// Choose a specific Butcher table for implicit RK method.
-   /** See the documentation for all possible options, stability regions, etc.
-       For example, table_num = ARK548L2SA_DIRK_8_4_5 is 8-stage 5th order. */
-   void SetIRKTableNum(int table_num);
-   /// Choose a specific Butcher table for explicit RK method.
-   /** See the documentation for all possible options, stability regions, etc.*/
+   /// Choose a specific Butcher table for an explicit RK method.
+   /** See ARKODE documentation for all possible options, stability regions, etc.
+       For example, table_num = BOGACKI_SHAMPINE_4_2_3 is 4-stage 3rd order. */
    void SetERKTableNum(int table_num);
 
-   /** @brief Use a fixed time step size, instead of performing any form of
-       temporal adaptivity. */
+   /// Choose a specific Butcher table for a diagonally implicit RK method.
+   /** See ARKODE documentation for all possible options, stability regions, etc.
+       For example, table_num = CASH_5_3_4 is 5-stage 4th order. */
+   void SetIRKTableNum(int table_num);
+
+   /// Choose a specific Butcher table for an IMEX RK method.
+   /** See ARKODE documentation for all possible options, stability regions, etc.
+       For example, etable_num = ARK548L2SA_DIRK_8_4_5 and
+       itable_num = ARK548L2SA_ERK_8_4_5 is 8-stage 5th order. */
+   void SetIMEXTableNum(int etable_num, int itable_num);
+
+   /// Use a fixed time step size (disable temporal adaptivity).
    /** Use of this function is not recommended, since there is no assurance of
        the validity of the computed solutions. It is primarily provided for
        code-to-code verification testing purposes. */
    void SetFixedStep(double dt);
 
-   /// Set the maximum time step of the Runge-Kutta method.
-   void SetMaxStep(double dt_max)
-   { flag = ARKodeSetMaxStep(sundials_mem, dt_max); }
-
-   /// Set the ODE right-hand-side operator.
-   /** The start time of ARKODE is initialized from the current time of @a f_.
-       @note This method calls ARKodeInit(). Some ARKODE parameters can be set
-       (using the handle returned by SundialsMem()) only after this call. */
-   virtual void Init(TimeDependentOperator &f_);
-
-   /// Use ARKODE to integrate over [t, t + dt], with the specified step mode.
-   /** Calls ARKode(), which is the main driver of the ARKODE package.
-       @param[in,out] x  Solution vector to advance. On input/output x=x(t)
-                         for t corresponding to the input/output value of t,
-                         respectively.
-       @param[in,out] t  Input: the starting time value. Output: the time value
-                         of the solution output, as returned by CVode().
-       @param[in,out] dt Input: desired time step. Output: the last incremental
-                         time step used. */
-   virtual void Step(Vector &x, double &t, double &dt);
-
-   /// Print ARKODE statistics.
+   /// Print various ARKStep statistics.
    void PrintInfo() const;
 
-   /// Destroy the associated ARKODE memory.
-   virtual ~ARKODESolver();
+   /// Destroy the associated ARKode memory and SUNDIALS objects.
+   virtual ~ARKStepSolver();
 };
 
-/// Wrapper for SUNDIALS' KINSOL library -- Nonlinear solvers.
-/**
-   - http://computation.llnl.gov/projects/sundials
-   - http://computation.llnl.gov/sites/default/files/public/kin_guide.pdf
 
-   @note To minimize uncertainty, we advise the user to adhere to the given
-   interface, instead of making similar calls by the KINSOL's
-   internal KINMem object.
-*/
-class KinSolver : public NewtonSolver, public SundialsSolver
+// ---------------------------------------------------------------------------
+// Interface to the KINSOL library -- nonlinear solver methods
+// ---------------------------------------------------------------------------
+
+/// Interface to the KINSOL library -- nonlinear solver methods.
+class KINSolver : public NewtonSolver, public SundialsSolver
 {
 protected:
-   bool use_oper_grad;
-   mutable N_Vector y_scale, f_scale;
-   const Operator *jacobian; // stores the result of oper->GetGradient()
+   int global_strategy;               ///< KINSOL solution strategy
+   bool use_oper_grad;                ///< use the Jv prod function
+   mutable N_Vector y_scale, f_scale; ///< scaling vectors
+   const Operator *jacobian;          ///< stores oper->GetGradient()
+   int maa;                           ///< number of acceleration vectors
 
-   /// @name Auxiliary callback functions.
-   ///@{
-   // Computes the non-linear operator action F(u).
-   // The real type of user_data is pointer to KinSolver.
+   /// Wrapper to compute the nonlinear residual \f$ F(u) = 0 \f$.
    static int Mult(const N_Vector u, N_Vector fu, void *user_data);
 
-   // Computes J(u)v. The real type of user_data is pointer to KinSolver.
+   /// Wrapper to compute the Jacobian-vector product \f$ J(u) v = Jv \f$.
    static int GradientMult(N_Vector v, N_Vector Jv, N_Vector u,
                            booleantype *new_u, void *user_data);
 
-   static int LinSysSetup(KINMemRec *kin_mem);
+   /// Setup the linear system \f$ J u = b \f$.
+   static int LinSysSetup(N_Vector u, N_Vector fu, SUNMatrix J,
+                          void *user_data, N_Vector tmp1, N_Vector tmp2);
 
-   static int LinSysSolve(KINMemRec *kin_mem, N_Vector x, N_Vector b,
-                          realtype *sJpnorm, realtype *sFdotJp);
-   ///@}
+   /// Solve the linear system \f$ J u = b \f$.
+   static int LinSysSolve(SUNLinearSolver LS, SUNMatrix J, N_Vector u,
+                          N_Vector b, realtype tol);
 
 public:
-   /// Construct a serial KinSolver, a wrapper for SUNDIALS' KINSOL solver.
+
+   /// Construct a serial wrapper to SUNDIALS' KINSOL nonlinear solver.
    /** @param[in] strategy   Specifies the nonlinear solver strategy:
                              KIN_NONE / KIN_LINESEARCH / KIN_PICARD / KIN_FP.
        @param[in] oper_grad  Specifies whether the solver should use its
                              Operator's GetGradient() method to compute the
                              Jacobian of the system. */
-   KinSolver(int strategy, bool oper_grad = true);
+   KINSolver(int strategy, bool oper_grad = true);
 
 #ifdef MFEM_USE_MPI
-   /// Construct a parallel KinSolver, a wrapper for SUNDIALS' KINSOL solver.
+   /// Construct a parallel wrapper to SUNDIALS' KINSOL nonlinear solver.
    /** @param[in] comm       The MPI communicator used to partition the system.
        @param[in] strategy   Specifies the nonlinear solver strategy:
                              KIN_NONE / KIN_LINESEARCH / KIN_PICARD / KIN_FP.
        @param[in] oper_grad  Specifies whether the solver should use its
                              Operator's GetGradient() method to compute the
                              Jacobian of the system. */
-   KinSolver(MPI_Comm comm, int strategy, bool oper_grad = true);
+   KINSolver(MPI_Comm comm, int strategy, bool oper_grad = true);
 #endif
 
    /// Destroy the associated KINSOL memory.
-   virtual ~KinSolver();
+   virtual ~KINSolver();
 
-   /// Set the nonlinear Operator of the system. This method calls KINInit().
+   /// Set the nonlinear Operator of the system and initialize KINSOL.
+   /** @note If this method is called a second time with a different problem
+       size, then non-default KINSOL-specific options will be lost and will need
+       to be set again. */
    virtual void SetOperator(const Operator &op);
 
    /// Set the linear solver for inverting the Jacobian.
    /** @note This function assumes that Operator::GetGradient(const Vector &)
              is implemented by the Operator specified by
-             SetOperator(const Operator &). */
+             SetOperator(const Operator &).
+
+             This method must be called after SetOperator(). */
    virtual void SetSolver(Solver &solver);
-   /// Equivalent to SetSolver(Solver).
+
+   /// Equivalent to SetSolver(solver).
    virtual void SetPreconditioner(Solver &solver) { SetSolver(solver); }
 
    /// Set KINSOL's scaled step tolerance.
-   /** The default tolerance is U^(2/3), where U = machine unit roundoff. */
+   /** The default tolerance is \f$ U^\frac{2}{3} \f$ , where
+       U = machine unit roundoff.
+       @note This method must be called after SetOperator(). */
    void SetScaledStepTol(double sstol);
-   /// Set KINSOL's functional norm tolerance.
-   /** The default tolerance is U^(1/3), where U = machine unit roundoff.
-        @note This function is equivalent to SetAbsTol(double). */
-   void SetFuncNormTol(double ftol) { abs_tol = ftol; }
 
    /// Set maximum number of nonlinear iterations without a Jacobian update.
-   /** The default is 10. */
+   /** The default is 10.
+       @note This method must be called after SetOperator(). */
    void SetMaxSetupCalls(int max_calls);
 
-   /// Solve the nonlinear system F(x) = 0.
-   /** Calls the other Mult(Vector&, Vector&, Vector&) const method with
-       `x_scale = 1`. The values of 'fx_scale' are determined by comparing
+   /// Set the number of acceleration vectors to use with KIN_FP or KIN_PICARD.
+   /** The default is 0.
+       @ note This method must be called before SetOperator() to set the
+       maximum size of the acceleration space. The value of @a maa can be
+       altered after SetOperator() is called but it can't be higher than initial
+       maximum. */
+   void SetMAA(int maa);
+
+   /// Solve the nonlinear system \f$ F(x) = 0 \f$.
+   /** This method computes the x_scale and fx_scale vectors and calls the
+       other Mult(Vector&, Vector&, Vector&) const method. The x_scale vector
+       is a vector of ones and values of fx_scale are determined by comparing
        the chosen relative and functional norm (i.e. absolute) tolerances.
-       @param[in]     b  Not used, KINSol always assumes zero RHS.
+       @param[in]     b  Not used, KINSOL always assumes zero RHS
        @param[in,out] x  On input, initial guess, if @a #iterative_mode = true,
                          otherwise the initial guess is zero; on output, the
-                         solution. */
+                         solution */
    virtual void Mult(const Vector &b, Vector &x) const;
 
-   /// Solve the nonlinear system F(x) = 0.
+   /// Solve the nonlinear system \f$ F(x) = 0 \f$.
    /** Calls KINSol() to solve the nonlinear system. Before calling KINSol(),
        this functions uses the data members inherited from class IterativeSolver
        to set corresponding KINSOL options.
-       @param[in,out] x        On input, initial guess, if @a #iterative_mode =
-                               true, otherwise the initial guess is zero; on
-                               output, the solution.
-       @param[in]     x_scale  Elements of a diagonal scaling matrix D, s.t.
-                               D*x has all elements roughly the same when
-                               x is close to a solution.
-       @param[in]    fx_scale  Elements of a diagonal scaling matrix E, s.t.
-                               D*F(x) has all elements roughly the same when
-                               x is not too close to a solution. */
+       @param[in,out] x         On input, initial guess, if @a #iterative_mode =
+                                true, otherwise the initial guess is zero; on
+                                output, the solution
+       @param[in]     x_scale   Elements of a diagonal scaling matrix D, s.t.
+                                D*x has all elements roughly the same when
+                                x is close to a solution
+       @param[in]     fx_scale  Elements of a diagonal scaling matrix E, s.t.
+                                D*F(x) has all elements roughly the same when
+                                x is not too close to a solution */
    void Mult(Vector &x, const Vector &x_scale, const Vector &fx_scale) const;
 };
 

linalg/superlu.cpp

@@ -20,6 +20,10 @@
 #include "superlu_defs.h"
 #include "superlu_ddefs.h"
 
+#if XSDK_INDEX_SIZE == 64
+#error "SuperLUDist has been built with 64bit integers. This is not supported"
+#endif
+
 using namespace std;
 
 namespace mfem
@@ -130,6 +134,11 @@ SuperLURowLocMatrix::SuperLURowLocMatrix( const HypreParMatrix & hypParMat )
    // hypre_CSRMatrix.
    hypre_CSRMatrix * csr_op = hypre_MergeDiagAndOffd(parcsr_op);
    hypre_CSRMatrixSetDataOwner(csr_op,0);
+#if MFEM_HYPRE_VERSION >= 21600
+   MFEM_VERIFY(csr_op->num_rows < INT_MAX,"SuperLU: number of local rows "
+               "is too large to store as an integer.");
+   hypre_CSRMatrixBigJtoJ(csr_op);
+#endif
 
    int m         = parcsr_op->global_num_rows;
    int n         = parcsr_op->global_num_cols;

linalg/vector.cpp

@@ -834,9 +834,10 @@ double Vector::Sum() const
 {
    double sum = 0.0;
 
+   const double *h_data = this->HostRead();
    for (int i = 0; i < size; i++)
    {
-      sum += data[i];
+      sum += h_data[i];
    }
 
    return sum;
@@ -1043,13 +1044,6 @@ vector_min_cpu:
 
 #ifdef MFEM_USE_SUNDIALS
 
-#ifndef SUNTRUE
-#define SUNTRUE TRUE
-#endif
-#ifndef SUNFALSE
-#define SUNFALSE FALSE
-#endif
-
 Vector::Vector(N_Vector nv)
 {
    N_Vector_ID nvid = N_VGetVectorID(nv);

linalg/vector.hpp

@@ -140,6 +140,13 @@ public:
        @sa NewDataAndSize(). */
    inline void NewMemoryAndSize(const Memory<double> &mem, int s, bool own_mem);
 
+   /// Reset the Vector to be a reference to a sub-vector of @a base.
+   inline void MakeRef(Vector &base, int offset, int size);
+
+   /** @brief Reset the Vector to be a reference to a sub-vector of @a base
+       without changing its current size. */
+   inline void MakeRef(Vector &base, int offset);
+
    /// Set the Vector data (host pointer) ownership flag.
    void MakeDataOwner() const { data.SetHostPtrOwner(true); }
 
@@ -455,6 +462,19 @@ inline void Vector::NewMemoryAndSize(const Memory<double> &mem, int s,
    if (!own_mem) { data.ClearOwnerFlags(); }
 }
 
+inline void Vector::MakeRef(Vector &base, int offset, int s)
+{
+   data.Delete();
+   size = s;
+   data.MakeAlias(base.GetMemory(), offset, s);
+}
+
+inline void Vector::MakeRef(Vector &base, int offset)
+{
+   data.Delete();
+   data.MakeAlias(base.GetMemory(), offset, size);
+}
+
 inline void Vector::Destroy()
 {
    const bool use_dev = data.UseDevice();

mesh/element.hpp

@@ -74,8 +74,13 @@ public:
 
    virtual const int *GetEdgeVertices(int) const = 0;
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const = 0;
 
+   virtual int GetNFaces() const = 0;
+
+   virtual int GetNFaceVertices(int fi) const = 0;
+
    virtual const int *GetFaceVertices(int fi) const = 0;
 
    /// Mark the longest edge by assuming/changing the order of the vertices.

mesh/hexahedron.hpp

@@ -51,9 +51,14 @@ public:
    virtual const int *GetEdgeVertices(int ei) const
    { return geom_t::Edges[ei]; }
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const
    { nFaceVertices = 4; return 6; }
 
+   virtual int GetNFaces() const { return 6; }
+
+   virtual int GetNFaceVertices(int) const { return 4; }
+
    virtual const int *GetFaceVertices(int fi) const
    { return geom_t::FaceVert[fi]; }
 

mesh/mesh.cpp

@@ -430,6 +430,8 @@ void Mesh::GetBdrElementTransformation(int i, IsoparametricTransformation* ElTr)
    else
    {
       const FiniteElement *bdr_el = Nodes->FESpace()->GetBE(i);
+      Nodes->HostRead();
+      const GridFunction &nodes = *Nodes;
       if (bdr_el)
       {
          Array<int> vdofs;
@@ -440,7 +442,7 @@ void Mesh::GetBdrElementTransformation(int i, IsoparametricTransformation* ElTr)
          {
             for (int j = 0; j < n; j++)
             {
-               pm(k,j) = (*Nodes)(vdofs[n*k+j]);
+               pm(k,j) = nodes(vdofs[n*k+j]);
             }
          }
          ElTr->SetFE(bdr_el);
@@ -492,6 +494,8 @@ void Mesh::GetFaceTransformation(int FaceNo, IsoparametricTransformation *FTr)
    else // curved mesh
    {
       const FiniteElement *face_el = Nodes->FESpace()->GetFaceElement(FaceNo);
+      Nodes->HostRead();
+      const GridFunction &nodes = *Nodes;
       if (face_el)
       {
          Array<int> vdofs;
@@ -502,7 +506,7 @@ void Mesh::GetFaceTransformation(int FaceNo, IsoparametricTransformation *FTr)
          {
             for (int j = 0; j < n; j++)
             {
-               pm(i, j) = (*Nodes)(vdofs[n*i+j]);
+               pm(i, j) = nodes(vdofs[n*i+j]);
             }
          }
          FTr->SetFE(face_el);
@@ -3298,6 +3302,10 @@ Mesh::Mesh(Mesh *orig_mesh, int ref_factor, int ref_type)
    Array<int> rdofs;
    DenseMatrix phys_pts;
    int max_nv = 0;
+
+   DenseMatrix node_coordinates(spaceDim*pow(2, Dim), r_num_elem);
+   H1_FECollection vertex_fec(1, Dim);
+
    for (int el = 0; el < orig_mesh->GetNE(); el++)
    {
       Geometry::Type geom = orig_mesh->GetElementBaseGeometry(el);
@@ -3312,6 +3320,7 @@ Mesh::Mesh(Mesh *orig_mesh, int ref_factor, int ref_type)
       orig_mesh->GetElementTransformation(el)->Transform(rfe->GetNodes(),
                                                          phys_pts);
       const int *c2h_map = rfec.GetDofMap(geom);
+      const int *vertex_map = vertex_fec.GetDofMap(geom);
       for (int i = 0; i < phys_pts.Width(); i++)
       {
          vertices[rdofs[i]].SetCoords(spaceDim, phys_pts.GetColumn(i));
@@ -3326,9 +3335,24 @@ Mesh::Mesh(Mesh *orig_mesh, int ref_factor, int ref_type)
             int cid = RG.RefGeoms[k+nvert*j]; // local Cartesian index
             v[k] = rdofs[c2h_map[cid]];
          }
+         for (int k = 0; k < nvert; k++)
+         {
+            for (int j = 0; j < spaceDim; ++j)
+            {
+               node_coordinates(k*spaceDim + j, NumOfElements)
+                  = vertices[v[vertex_map[k]]](j);
+            }
+         }
          AddElement(elem);
       }
    }
+
+   SetCurvature(1, true, spaceDim);
+   Vector node_coordinates_vec(
+      node_coordinates.Data(),
+      node_coordinates.Width()*node_coordinates.Height());
+   SetNodes(node_coordinates_vec);
+
    // Add refined boundary elements
    for (int el = 0; el < orig_mesh->GetNBE(); el++)
    {
@@ -3429,6 +3453,28 @@ void Mesh::KnotInsert(Array<KnotVector *> &kv)
    UpdateNURBS();
 }
 
+void Mesh::KnotInsert(Array<Vector *> &kv)
+{
+   if (NURBSext == NULL)
+   {
+      mfem_error("Mesh::KnotInsert : Not a NURBS mesh!");
+   }
+
+   if (kv.Size() != NURBSext->GetNKV())
+   {
+      mfem_error("Mesh::KnotInsert : KnotVector array size mismatch!");
+   }
+
+   NURBSext->ConvertToPatches(*Nodes);
+
+   NURBSext->KnotInsert(kv);
+
+   last_operation = Mesh::NONE; // FiniteElementSpace::Update is not supported
+   sequence++;
+
+   UpdateNURBS();
+}
+
 void Mesh::NURBSUniformRefinement()
 {
    // do not check for NURBSext since this method is protected
@@ -4759,7 +4805,13 @@ void Mesh::GenerateNCFaceInfo()
    for (unsigned i = 0; i < list.slaves.size(); i++)
    {
       const NCMesh::Slave &slave = list.slaves[i];
-      if (slave.index >= nfaces || slave.master >= nfaces) { continue; }
+
+      if (slave.index < 0 || // degenerate slave face
+          slave.index >= nfaces || // ghost slave
+          slave.master >= nfaces) // has ghost master
+      {
+         continue;
+      }
 
       FaceInfo &slave_fi = faces_info[slave.index];
       FaceInfo &master_fi = faces_info[slave.master];
@@ -4912,15 +4964,39 @@ STable3D *Mesh::GetElementToFaceTable(int ret_ftbl)
    return NULL;
 }
 
+// shift cyclically 3 integers so that the smallest is first
+static inline
+void Rotate3(int &a, int &b, int &c)
+{
+   if (a < b)
+   {
+      if (a > c)
+      {
+         ShiftRight(a, b, c);
+      }
+   }
+   else
+   {
+      if (b < c)
+      {
+         ShiftRight(c, b, a);
+      }
+      else
+      {
+         ShiftRight(a, b, c);
+      }
+   }
+}
+
 void Mesh::ReorientTetMesh()
 {
-   int *v;
-
    if (Dim != 3 || !(meshgen & 1))
    {
       return;
    }
 
+   DeleteLazyTables();
+
    DSTable *old_v_to_v = NULL;
    Table *old_elem_vert = NULL;
 
@@ -4933,7 +5009,7 @@ void Mesh::ReorientTetMesh()
    {
       if (GetElementType(i) == Element::TETRAHEDRON)
       {
-         v = elements[i]->GetVertices();
+         int *v = elements[i]->GetVertices();
 
          Rotate3(v[0], v[1], v[2]);
          if (v[0] < v[3])
@@ -4942,7 +5018,7 @@ void Mesh::ReorientTetMesh()
          }
          else
          {
-            ShiftL2R(v[0], v[1], v[3]);
+            ShiftRight(v[0], v[1], v[3]);
          }
       }
    }
@@ -4951,7 +5027,7 @@ void Mesh::ReorientTetMesh()
    {
       if (GetBdrElementType(i) == Element::TRIANGLE)
       {
-         v = boundary[i]->GetVertices();
+         int *v = boundary[i]->GetVertices();
 
          Rotate3(v[0], v[1], v[2]);
       }
@@ -5783,7 +5859,7 @@ void Mesh::SetVertices(const Vector &vert_coord)
       }
 }
 
-void Mesh::GetNode(int i, double *coord)
+void Mesh::GetNode(int i, double *coord) const
 {
    if (Nodes)
    {
@@ -5934,16 +6010,19 @@ void Mesh::UniformRefinement2D()
    const int oedge = NumOfVertices;
    const int oelem = oedge + NumOfEdges;
 
+   Array<Element*> new_elements;
+   Array<Element*> new_boundary;
+
    vertices.SetSize(oelem + quad_counter);
-   elements.SetSize(4 * NumOfElements);
+   new_elements.SetSize(4 * NumOfElements);
    quad_counter = 0;
-   for (int i = 0; i < NumOfElements; i++)
+
+   for (int i = 0, j = 0; i < NumOfElements; i++)
    {
       const Element::Type el_type = elements[i]->GetType();
       const int attr = elements[i]->GetAttribute();
       int *v = elements[i]->GetVertices();
       const int *e = el_to_edge->GetRow(i);
-      const int j = NumOfElements + 3 * i;
       int vv[2];
 
       if (el_type == Element::TRIANGLE)
@@ -5957,12 +6036,14 @@ void Mesh::UniformRefinement2D()
             AverageVertices(vv, 2, oedge+e[ei]);
          }
 
-         elements[j+0] = new Triangle(oedge+e[1], oedge+e[2], oedge+e[0], attr);
-         elements[j+1] = new Triangle(oedge+e[0], v[1], oedge+e[1], attr);
-         elements[j+2] = new Triangle(oedge+e[2], oedge+e[1], v[2], attr);
-
-         v[1] = oedge+e[0];
-         v[2] = oedge+e[2];
+         new_elements[j++] =
+            new Triangle(v[0], oedge+e[0], oedge+e[2], attr);
+         new_elements[j++] =
+            new Triangle(oedge+e[1], oedge+e[2], oedge+e[0], attr);
+         new_elements[j++] =
+            new Triangle(oedge+e[0], v[1], oedge+e[1], attr);
+         new_elements[j++] =
+            new Triangle(oedge+e[2], oedge+e[1], v[2], attr);
       }
       else if (el_type == Element::QUADRILATERAL)
       {
@@ -5979,34 +6060,36 @@ void Mesh::UniformRefinement2D()
             AverageVertices(vv, 2, oedge+e[ei]);
          }
 
-         elements[j+0] = new Quadrilateral(oedge+e[0], v[1], oedge+e[1],
-                                           oelem+qe, attr);
-         elements[j+1] = new Quadrilateral(oelem+qe, oedge+e[1],
-                                           v[2], oedge+e[2], attr);
-         elements[j+2] = new Quadrilateral(oedge+e[3], oelem+qe,
-                                           oedge+e[2], v[3], attr);
-
-         v[1] = oedge+e[0];
-         v[2] = oelem+qe;
-         v[3] = oedge+e[3];
+         new_elements[j++] =
+            new Quadrilateral(v[0], oedge+e[0], oelem+qe, oedge+e[3], attr);
+         new_elements[j++] =
+            new Quadrilateral(oedge+e[0], v[1], oedge+e[1], oelem+qe, attr);
+         new_elements[j++] =
+            new Quadrilateral(oelem+qe, oedge+e[1], v[2], oedge+e[2], attr);
+         new_elements[j++] =
+            new Quadrilateral(oedge+e[3], oelem+qe, oedge+e[2], v[3], attr);
       }
       else
       {
          MFEM_ABORT("unknown element type: " << el_type);
       }
+      FreeElement(elements[i]);
    }
+   mfem::Swap(elements, new_elements);
 
-   boundary.SetSize(2 * NumOfBdrElements);
-   for (int i = 0; i < NumOfBdrElements; i++)
+   // refine boundary elements
+   new_boundary.SetSize(2 * NumOfBdrElements);
+   for (int i = 0, j = 0; i < NumOfBdrElements; i++)
    {
       const int attr = boundary[i]->GetAttribute();
       int *v = boundary[i]->GetVertices();
-      const int j = NumOfBdrElements + i;
 
-      boundary[j] = new Segment(oedge+be_to_edge[i], v[1], attr);
+      new_boundary[j++] = new Segment(v[0], oedge+be_to_edge[i], attr);
+      new_boundary[j++] = new Segment(oedge+be_to_edge[i], v[1], attr);
 
-      v[1] = oedge+be_to_edge[i];
+      FreeElement(boundary[i]);
    }
+   mfem::Swap(boundary, new_boundary);
 
    static const double A = 0.0, B = 0.5, C = 1.0;
    static double tri_children[2*3*4] =
@@ -6024,17 +6107,17 @@ void Mesh::UniformRefinement2D()
       A,B, B,B, B,C, A,C  // upper-left
    };
 
-   CoarseFineTr.point_matrices[Geometry::TRIANGLE].
-   UseExternalData(tri_children, 2, 3, 4);
-   CoarseFineTr.point_matrices[Geometry::SQUARE].
-   UseExternalData(quad_children, 2, 4, 4);
+   CoarseFineTr.point_matrices[Geometry::TRIANGLE]
+   .UseExternalData(tri_children, 2, 3, 4);
+   CoarseFineTr.point_matrices[Geometry::SQUARE]
+   .UseExternalData(quad_children, 2, 4, 4);
    CoarseFineTr.embeddings.SetSize(elements.Size());
 
    for (int i = 0; i < elements.Size(); i++)
    {
       Embedding &emb = CoarseFineTr.embeddings[i];
-      emb.parent = (i < NumOfElements) ? i : (i - NumOfElements) / 3;
-      emb.matrix = (i < NumOfElements) ? 0 : (i - NumOfElements) % 3 + 1;
+      emb.parent = i / 4;
+      emb.matrix = i % 4;
    }
 
    NumOfVertices    = vertices.Size();
@@ -6166,17 +6249,20 @@ void Mesh::UniformRefinement3D_base(Array<int> *f2qf_ptr, DSTable *v_to_v_p)
    const int oface = oedge + NumOfEdges;
    const int oelem = oface + NumOfQuadFaces;
 
+   Array<Element*> new_elements;
+   Array<Element*> new_boundary;
+
    vertices.SetSize(oelem + hex_counter);
-   elements.SetSize(8 * NumOfElements);
-   CoarseFineTr.embeddings.SetSize(elements.Size());
+   new_elements.SetSize(8 * NumOfElements);
+   CoarseFineTr.embeddings.SetSize(new_elements.Size());
+
    hex_counter = 0;
-   for (int i = 0; i < NumOfElements; i++)
+   for (int i = 0, j = 0; i < NumOfElements; i++)
    {
       const Element::Type el_type = elements[i]->GetType();
       const int attr = elements[i]->GetAttribute();
       int *v = elements[i]->GetVertices();
       const int *e = el_to_edge->GetRow(i);
-      const int j = NumOfElements + 7 * i;
       int vv[4], ev[12];
 
       if (e2v.Size())
@@ -6322,51 +6408,54 @@ void Mesh::UniformRefinement3D_base(Array<int> *f2qf_ptr, DSTable *v_to_v_p)
             const int (&mv)[4][4] = mv_all[rt];
 
 #ifndef MFEM_USE_MEMALLOC
-            elements[j+0] = new Tetrahedron(oedge+e[0], v[1],
-                                            oedge+e[3], oedge+e[4], attr);
-            elements[j+1] = new Tetrahedron(oedge+e[1], oedge+e[3],
-                                            v[2], oedge+e[5], attr);
-            elements[j+2] = new Tetrahedron(oedge+e[2], oedge+e[4],
-                                            oedge+e[5], v[3], attr);
+            new_elements[j+0] =
+               new Tetrahedron(v[0], oedge+e[0], oedge+e[1], oedge+e[2], attr);
+            new_elements[j+1] =
+               new Tetrahedron(oedge+e[0], v[1], oedge+e[3], oedge+e[4], attr);
+            new_elements[j+2] =
+               new Tetrahedron(oedge+e[1], oedge+e[3], v[2], oedge+e[5], attr);
+            new_elements[j+3] =
+               new Tetrahedron(oedge+e[2], oedge+e[4], oedge+e[5], v[3], attr);
+
             for (int k = 0; k < 4; k++)
             {
-               elements[j+k+3] =
+               new_elements[j+4+k] =
                   new Tetrahedron(oedge+e[mv[k][0]], oedge+e[mv[k][1]],
                                   oedge+e[mv[k][2]], oedge+e[mv[k][3]], attr);
             }
 #else
             Tetrahedron *tet;
-            elements[j+0] = tet = TetMemory.Alloc();
+            new_elements[j+0] = tet = TetMemory.Alloc();
+            tet->Init(v[0], oedge+e[0], oedge+e[1], oedge+e[2], attr);
+
+            new_elements[j+1] = tet = TetMemory.Alloc();
             tet->Init(oedge+e[0], v[1], oedge+e[3], oedge+e[4], attr);
-            elements[j+1] = tet = TetMemory.Alloc();
+
+            new_elements[j+2] = tet = TetMemory.Alloc();
             tet->Init(oedge+e[1], oedge+e[3], v[2], oedge+e[5], attr);
-            elements[j+2] = tet = TetMemory.Alloc();
+
+            new_elements[j+3] = tet = TetMemory.Alloc();
             tet->Init(oedge+e[2], oedge+e[4], oedge+e[5], v[3], attr);
+
             for (int k = 0; k < 4; k++)
             {
-               elements[j+k+3] = tet = TetMemory.Alloc();
+               new_elements[j+4+k] = tet = TetMemory.Alloc();
                tet->Init(oedge+e[mv[k][0]], oedge+e[mv[k][1]],
                          oedge+e[mv[k][2]], oedge+e[mv[k][3]], attr);
             }
 #endif
-
-            v[1] = oedge+e[0];
-            v[2] = oedge+e[1];
-            v[3] = oedge+e[2];
-            ((Tetrahedron*)elements[i])->SetRefinementFlag(0);
-
-            CoarseFineTr.embeddings[i].parent = i;
-            CoarseFineTr.embeddings[i].matrix = 0;
-            for (int k = 0; k < 3; k++)
+            for (int k = 0; k < 4; k++)
             {
                CoarseFineTr.embeddings[j+k].parent = i;
-               CoarseFineTr.embeddings[j+k].matrix = k+1;
+               CoarseFineTr.embeddings[j+k].matrix = k;
             }
             for (int k = 0; k < 4; k++)
             {
-               CoarseFineTr.embeddings[j+k+3].parent = i;
-               CoarseFineTr.embeddings[j+k+3].matrix = 4*(rt+1)+k;
+               CoarseFineTr.embeddings[j+4+k].parent = i;
+               CoarseFineTr.embeddings[j+4+k].matrix = 4*(rt+1)+k;
             }
+
+            j += 8;
          }
          break;
 
@@ -6396,33 +6485,37 @@ void Mesh::UniformRefinement3D_base(Array<int> *f2qf_ptr, DSTable *v_to_v_p)
             const int qf3 = f2qf[f[3]];
             const int qf4 = f2qf[f[4]];
 
-            elements[j+0] = new Wedge(oedge+e[1], oedge+e[2], oedge+e[0],
-                                      oface+qf3, oface+qf4, oface+qf2,
-                                      attr);
-            elements[j+1] = new Wedge(oedge+e[0], v[1], oedge+e[1],
-                                      oface+qf2, oedge+e[7], oface+qf3,
-                                      attr);
-            elements[j+2] = new Wedge(oedge+e[2], oedge+e[1], v[2],
-                                      oface+qf4, oface+qf3, oedge+e[8],
-                                      attr);
-            elements[j+3] = new Wedge(oedge+e[6], oface+qf2, oface+qf4,
-                                      v[3], oedge+e[3], oedge+e[5],
-                                      attr);
-            elements[j+4] = new Wedge(oface+qf3, oface+qf4, oface+qf2,
-                                      oedge+e[4], oedge+e[5], oedge+e[3],
-                                      attr);
-            elements[j+5] = new Wedge(oface+qf2, oedge+e[7], oface+qf3,
-                                      oedge+e[3], v[4], oedge+e[4],
-                                      attr);
-            elements[j+6] = new Wedge(oface+qf4, oface+qf3, oedge+e[8],
-                                      oedge+e[5], oedge+e[4], v[5],
-                                      attr);
+            new_elements[j++] =
+               new Wedge(v[0], oedge+e[0], oedge+e[2],
+                         oedge+e[6], oface+qf2, oface+qf4, attr);
 
-            v[1] = oedge+e[0];
-            v[2] = oedge+e[2];
-            v[3] = oedge+e[6];
-            v[4] = oface+qf2;
-            v[5] = oface+qf4;
+            new_elements[j++] =
+               new Wedge(oedge+e[1], oedge+e[2], oedge+e[0],
+                         oface+qf3, oface+qf4, oface+qf2, attr);
+
+            new_elements[j++] =
+               new Wedge(oedge+e[0], v[1], oedge+e[1],
+                         oface+qf2, oedge+e[7], oface+qf3, attr);
+
+            new_elements[j++] =
+               new Wedge(oedge+e[2], oedge+e[1], v[2],
+                         oface+qf4, oface+qf3, oedge+e[8], attr);
+
+            new_elements[j++] =
+               new Wedge(oedge+e[6], oface+qf2, oface+qf4,
+                         v[3], oedge+e[3], oedge+e[5], attr);
+
+            new_elements[j++] =
+               new Wedge(oface+qf3, oface+qf4, oface+qf2,
+                         oedge+e[4], oedge+e[5], oedge+e[3], attr);
+
+            new_elements[j++] =
+               new Wedge(oface+qf2, oedge+e[7], oface+qf3,
+                         oedge+e[3], v[4], oedge+e[4], attr);
+
+            new_elements[j++] =
+               new Wedge(oface+qf4, oface+qf3, oedge+e[8],
+                         oedge+e[5], oedge+e[4], v[5], attr);
          }
          break;
 
@@ -6464,35 +6557,38 @@ void Mesh::UniformRefinement3D_base(Array<int> *f2qf_ptr, DSTable *v_to_v_p)
                AverageVertices(vv, 2, oedge+e[ei]);
             }
 
-            elements[j+0] = new Hexahedron(oedge+e[0], v[1], oedge+e[1],
-                                           oface+qf[0], oface+qf[1], oedge+e[9],
-                                           oface+qf[2], oelem+he, attr);
-            elements[j+1] = new Hexahedron(oface+qf[0], oedge+e[1], v[2],
-                                           oedge+e[2], oelem+he, oface+qf[2],
-                                           oedge+e[10], oface+qf[3], attr);
-            elements[j+2] = new Hexahedron(oedge+e[3], oface+qf[0], oedge+e[2],
-                                           v[3], oface+qf[4], oelem+he,
-                                           oface+qf[3], oedge+e[11], attr);
-            elements[j+3] = new Hexahedron(oedge+e[8], oface+qf[1], oelem+he,
-                                           oface+qf[4], v[4], oedge+e[4],
-                                           oface+qf[5], oedge+e[7], attr);
-            elements[j+4] = new Hexahedron(oface+qf[1], oedge+e[9], oface+qf[2],
-                                           oelem+he, oedge+e[4], v[5],
-                                           oedge+e[5], oface+qf[5], attr);
-            elements[j+5] = new Hexahedron(oelem+he, oface+qf[2], oedge+e[10],
-                                           oface+qf[3], oface+qf[5], oedge+e[5],
-                                           v[6], oedge+e[6], attr);
-            elements[j+6] = new Hexahedron(oface+qf[4], oelem+he, oface+qf[3],
-                                           oedge+e[11], oedge+e[7], oface+qf[5],
-                                           oedge+e[6], v[7], attr);
-
-            v[1] = oedge+e[0];
-            v[2] = oface+qf[0];
-            v[3] = oedge+e[3];
-            v[4] = oedge+e[8];
-            v[5] = oface+qf[1];
-            v[6] = oelem+he;
-            v[7] = oface+qf[4];
+            new_elements[j++] =
+               new Hexahedron(v[0], oedge+e[0], oface+qf[0],
+                              oedge+e[3], oedge+e[8], oface+qf[1],
+                              oelem+he, oface+qf[4], attr);
+            new_elements[j++] =
+               new Hexahedron(oedge+e[0], v[1], oedge+e[1],
+                              oface+qf[0], oface+qf[1], oedge+e[9],
+                              oface+qf[2], oelem+he, attr);
+            new_elements[j++] =
+               new Hexahedron(oface+qf[0], oedge+e[1], v[2],
+                              oedge+e[2], oelem+he, oface+qf[2],
+                              oedge+e[10], oface+qf[3], attr);
+            new_elements[j++] =
+               new Hexahedron(oedge+e[3], oface+qf[0], oedge+e[2],
+                              v[3], oface+qf[4], oelem+he,
+                              oface+qf[3], oedge+e[11], attr);
+            new_elements[j++] =
+               new Hexahedron(oedge+e[8], oface+qf[1], oelem+he,
+                              oface+qf[4], v[4], oedge+e[4],
+                              oface+qf[5], oedge+e[7], attr);
+            new_elements[j++] =
+               new Hexahedron(oface+qf[1], oedge+e[9], oface+qf[2],
+                              oelem+he, oedge+e[4], v[5],
+                              oedge+e[5], oface+qf[5], attr);
+            new_elements[j++] =
+               new Hexahedron(oelem+he, oface+qf[2], oedge+e[10],
+                              oface+qf[3], oface+qf[5], oedge+e[5],
+                              v[6], oedge+e[6], attr);
+            new_elements[j++] =
+               new Hexahedron(oface+qf[4], oelem+he, oface+qf[3],
+                              oedge+e[11], oedge+e[7], oface+qf[5],
+                              oedge+e[6], v[7], attr);
          }
          break;
 
@@ -6500,16 +6596,18 @@ void Mesh::UniformRefinement3D_base(Array<int> *f2qf_ptr, DSTable *v_to_v_p)
             MFEM_ABORT("Unknown 3D element type \"" << el_type << "\"");
             break;
       }
+      FreeElement(elements[i]);
    }
+   mfem::Swap(elements, new_elements);
 
-   boundary.SetSize(4 * NumOfBdrElements);
-   for (int i = 0; i < NumOfBdrElements; i++)
+   // refine boundary elements
+   new_boundary.SetSize(4 * NumOfBdrElements);
+   for (int i = 0, j = 0; i < NumOfBdrElements; i++)
    {
       const Element::Type bdr_el_type = boundary[i]->GetType();
       const int attr = boundary[i]->GetAttribute();
       int *v = boundary[i]->GetVertices();
       const int *e = bel_to_edge->GetRow(i);
-      const int j = NumOfBdrElements + 3 * i;
       int ev[4];
 
       if (e2v.Size())
@@ -6521,34 +6619,36 @@ void Mesh::UniformRefinement3D_base(Array<int> *f2qf_ptr, DSTable *v_to_v_p)
 
       if (bdr_el_type == Element::TRIANGLE)
       {
-         boundary[j+0] = new Triangle(oedge+e[1], oedge+e[2], oedge+e[0], attr);
-         boundary[j+1] = new Triangle(oedge+e[0], v[1], oedge+e[1], attr);
-         boundary[j+2] = new Triangle(oedge+e[2], oedge+e[1], v[2], attr);
-
-         v[1] = oedge+e[0];
-         v[2] = oedge+e[2];
+         new_boundary[j++] =
+            new Triangle(v[0], oedge+e[0], oedge+e[2], attr);
+         new_boundary[j++] =
+            new Triangle(oedge+e[1], oedge+e[2], oedge+e[0], attr);
+         new_boundary[j++] =
+            new Triangle(oedge+e[0], v[1], oedge+e[1], attr);
+         new_boundary[j++] =
+            new Triangle(oedge+e[2], oedge+e[1], v[2], attr);
       }
       else if (bdr_el_type == Element::QUADRILATERAL)
       {
          const int qf =
             (f2qf.Size() == 0) ? be_to_face[i] : f2qf[be_to_face[i]];
 
-         boundary[j+0] = new Quadrilateral(oedge+e[0], v[1], oedge+e[1],
-                                           oface+qf, attr);
-         boundary[j+1] = new Quadrilateral(oface+qf, oedge+e[1], v[2],
-                                           oedge+e[2], attr);
-         boundary[j+2] = new Quadrilateral(oedge+e[3], oface+qf,
-                                           oedge+e[2], v[3], attr);
-
-         v[1] = oedge+e[0];
-         v[2] = oface+qf;
-         v[3] = oedge+e[3];
+         new_boundary[j++] =
+            new Quadrilateral(v[0], oedge+e[0], oface+qf, oedge+e[3], attr);
+         new_boundary[j++] =
+            new Quadrilateral(oedge+e[0], v[1], oedge+e[1], oface+qf, attr);
+         new_boundary[j++] =
+            new Quadrilateral(oface+qf, oedge+e[1], v[2], oedge+e[2], attr);
+         new_boundary[j++] =
+            new Quadrilateral(oedge+e[3], oface+qf, oedge+e[2], v[3], attr);
       }
       else
       {
          MFEM_ABORT("boundary Element is not a triangle or a quad!");
       }
+      FreeElement(boundary[i]);
    }
+   mfem::Swap(boundary, new_boundary);
 
    static const double A = 0.0, B = 0.5, C = 1.0;
    static double tet_children[3*4*16] =
@@ -6599,20 +6699,21 @@ void Mesh::UniformRefinement3D_base(Array<int> *f2qf_ptr, DSTable *v_to_v_p)
       A,B,B, B,B,B, B,C,B, A,C,B, A,B,C, B,B,C, B,C,C, A,C,C
    };
 
-   CoarseFineTr.point_matrices[Geometry::TETRAHEDRON].
-   UseExternalData(tet_children, 3, 4, 16);
-   CoarseFineTr.point_matrices[Geometry::PRISM].
-   UseExternalData(pri_children, 3, 6, 8);
-   CoarseFineTr.point_matrices[Geometry::CUBE].
-   UseExternalData(hex_children, 3, 8, 8);
+   CoarseFineTr.point_matrices[Geometry::TETRAHEDRON]
+   .UseExternalData(tet_children, 3, 4, 16);
+   CoarseFineTr.point_matrices[Geometry::PRISM]
+   .UseExternalData(pri_children, 3, 6, 8);
+   CoarseFineTr.point_matrices[Geometry::CUBE]
+   .UseExternalData(hex_children, 3, 8, 8);
 
    for (int i = 0; i < elements.Size(); i++)
    {
-      // Tetrahedron elements are handled above:
+      // tetrahedron elements are handled above:
       if (elements[i]->GetType() == Element::TETRAHEDRON) { continue; }
+
       Embedding &emb = CoarseFineTr.embeddings[i];
-      emb.parent = (i < NumOfElements) ? i : (i - NumOfElements) / 7;
-      emb.matrix = (i < NumOfElements) ? 0 : (i - NumOfElements) % 7 + 1;
+      emb.parent = i / 8;
+      emb.matrix = i % 8;
    }
 
    NumOfVertices    = vertices.Size();
@@ -7196,13 +7297,13 @@ void Mesh::GeneralRefinement(const Array<Refinement> &refinements,
    else if (nonconforming < 0)
    {
       // determine if nonconforming refinement is suitable
-      if (meshgen & 2)
+      if ((meshgen & 2) || (meshgen & 4))
       {
-         nonconforming = 1;
+         nonconforming = 1; // tensor product elements and wedges
       }
       else
       {
-         nonconforming = 0;
+         nonconforming = 0; // simplices
       }
    }
 
@@ -7257,8 +7358,9 @@ void Mesh::EnsureNCMesh(bool triangles_nonconforming)
 
    if (!ncmesh)
    {
-      if ((meshgen & 2) /* quads/hexes */ ||
-          (triangles_nonconforming && Dim == 2 && (meshgen & 1)))
+      if ((meshgen & 0x2) /* quads/hexes */ ||
+          (meshgen & 0x4) /* wedges */ ||
+          (triangles_nonconforming && Dim == 2 && (meshgen & 0x1)))
       {
          MFEM_VERIFY(GetNumGeometries(Dim) <= 1,
                      "mixed meshes are not supported");
@@ -7681,12 +7783,7 @@ void Mesh::UniformRefinement(int i, const DSTable &v_to_v,
 void Mesh::InitRefinementTransforms()
 {
    // initialize CoarseFineTr
-   map<Geometry::Type,DenseTensor> &pms = CoarseFineTr.point_matrices;
-   map<Geometry::Type,DenseTensor>::iterator pms_iter;
-   for (pms_iter = pms.begin(); pms_iter != pms.end(); ++pms_iter)
-   {
-      pms_iter->second.SetSize(0, 0, 0);
-   }
+   CoarseFineTr.Clear();
    CoarseFineTr.embeddings.SetSize(NumOfElements);
    for (int i = 0; i < NumOfElements; i++)
    {
@@ -10016,4 +10113,51 @@ Mesh *Extrude2D(Mesh *mesh, const int nz, const double sz)
    return mesh3d;
 }
 
+#ifdef MFEM_DEBUG
+void Mesh::DebugDump(std::ostream &out) const
+{
+   // dump vertices and edges (NCMesh "nodes")
+   out << NumOfVertices + NumOfEdges << "\n";
+   for (int i = 0; i < NumOfVertices; i++)
+   {
+      const double *v = GetVertex(i);
+      out << i << " " << v[0] << " " << v[1] << " " << v[2]
+          << " 0 0 " << i << " -1 0\n";
+   }
+
+   Array<int> ev;
+   for (int i = 0; i < NumOfEdges; i++)
+   {
+      GetEdgeVertices(i, ev);
+      double mid[3] = {0, 0, 0};
+      for (int j = 0; j < 2; j++)
+      {
+         for (int k = 0; k < spaceDim; k++)
+         {
+            mid[k] += GetVertex(ev[j])[k];
+         }
+      }
+      out << NumOfVertices+i << " "
+          << mid[0]/2 << " " << mid[1]/2 << " " << mid[2]/2 << " "
+          << ev[0] << " " << ev[1] << " -1 " << i << " 0\n";
+   }
+
+   // dump elements
+   out << NumOfElements << "\n";
+   for (int i = 0; i < NumOfElements; i++)
+   {
+      const Element* e = elements[i];
+      out << e->GetNVertices() << " ";
+      for (int j = 0; j < e->GetNVertices(); j++)
+      {
+         out << e->GetVertices()[j] << " ";
+      }
+      out << e->GetAttribute() << " 0 " << i << "\n";
+   }
+
+   // dump faces
+   out << "0\n";
+}
+#endif
+
 }

mesh/mesh.hpp

@@ -386,11 +386,6 @@ protected:
       return FaceIsInterior(FaceNo) || (faces_info[FaceNo].Elem2Inf >= 0);
    }
 
-   // shift cyclically 3 integers left-to-right
-   inline static void ShiftL2R(int &, int &, int &);
-   // shift cyclically 3 integers so that the smallest is first
-   inline static void Rotate3(int &, int &, int &);
-
    void FreeElement(Element *E);
 
    void GenerateFaces();
@@ -1014,7 +1009,7 @@ public:
    // Nodes are only active for higher order meshes, and share locations with
    // the vertices, plus all the higher- order control points within the element
    // and along the edges and on the faces.
-   void GetNode(int i, double *coord);
+   void GetNode(int i, double *coord) const;
    void SetNode(int i, const double *coord);
 
    // Node operations for curved mesh.
@@ -1122,6 +1117,7 @@ public:
 
    ///@{ @name NURBS mesh refinement methods
    void KnotInsert(Array<KnotVector *> &kv);
+   void KnotInsert(Array<Vector *> &kv);
    /* For each knot vector:
          new_degree = max(old_degree, min(old_degree + rel_degree, degree)). */
    void DegreeElevate(int rel_degree, int degree = 16);
@@ -1270,6 +1266,11 @@ public:
 
    /// Destroys Mesh.
    virtual ~Mesh() { DestroyPointers(); }
+
+#ifdef MFEM_DEBUG
+   /// Output an NCMesh-compatible debug dump.
+   void DebugDump(std::ostream &out) const;
+#endif
 };
 
 /** Overload operator<< for std::ostream and Mesh; valid also for the derived
@@ -1358,35 +1359,13 @@ public:
 };
 
 
-// inline functions
-inline void Mesh::ShiftL2R(int &a, int &b, int &c)
+// shift cyclically 3 integers left-to-right
+inline void ShiftRight(int &a, int &b, int &c)
 {
    int t = a;
    a = c;  c = b;  b = t;
 }
 
-inline void Mesh::Rotate3(int &a, int &b, int &c)
-{
-   if (a < b)
-   {
-      if (a > c)
-      {
-         ShiftL2R(a, b, c);
-      }
-   }
-   else
-   {
-      if (b < c)
-      {
-         ShiftL2R(c, b, a);
-      }
-      else
-      {
-         ShiftL2R(a, b, c);
-      }
-   }
-}
-
 }
 
 #endif

mesh/ncmesh.hpp

@@ -15,6 +15,7 @@
 #include "../config/config.hpp"
 #include "../general/hash.hpp"
 #include "../general/globals.hpp"
+#include "../general/sort_pairs.hpp"
 #include "../linalg/densemat.hpp"
 #include "element.hpp"
 #include "vertex.hpp"
@@ -59,24 +60,23 @@ struct Embedding
 struct CoarseFineTransformations
 {
    /// Matrices for IsoparametricTransformation organized by Geometry::Type
-   std::map<Geometry::Type, DenseTensor> point_matrices;
+   DenseTensor point_matrices[Geometry::NumGeom];
    /// Fine element positions in their parents.
    Array<Embedding> embeddings;
 
-   const DenseTensor &GetPointMatrices(Geometry::Type geom) const;
-
    void GetCoarseToFineMap(const Mesh &fine_mesh,
                            Table &coarse_to_fine,
                            Array<int> &coarse_to_ref_type,
                            Table &ref_type_to_matrix,
                            Array<Geometry::Type> &ref_type_to_geom) const;
 
-   void Clear() { point_matrices.clear(); embeddings.DeleteAll(); }
+   void Clear();
+   bool IsInitialized() const;
    long MemoryUsage() const;
 };
 
 
-/** \brief A class for non-conforming AMR on higher-order hexahedral,
+/** \brief A class for non-conforming AMR on higher-order hexahedral, prismatic,
  *  quadrilateral or triangular meshes.
  *
  *  The class is used as follows:
@@ -152,10 +152,13 @@ public:
    {
       int index;   ///< Mesh number
       int element; ///< NCMesh::Element containing this vertex/edge/face
-      int local;   ///< local number within 'element'
+      char local;  ///< local number within 'element'
+      char geom;   ///< Geometry::Type (faces only) (char storage to save RAM)
 
-      MeshId(int index = -1, int element = -1, int local = -1)
-         : index(index), element(element), local(local) {}
+      MeshId(int index = -1, int element = -1, char local = -1, char geom = -1)
+         : index(index), element(element), local(local), geom(geom) {}
+
+      Geometry::Type Geom() const { return Geometry::Type(geom); }
    };
 
    /** Nonconforming edge/face that has more than one neighbor. The neighbors
@@ -164,8 +167,9 @@ public:
    {
       int slaves_begin, slaves_end; ///< slave faces
 
-      Master(int index, int element, int local, int sb, int se)
-         : MeshId(index, element, local), slaves_begin(sb), slaves_end(se) {}
+      Master(int index, int element, char local, char geom, int sb, int se)
+         : MeshId(index, element, local, geom)
+         , slaves_begin(sb), slaves_end(se) {}
    };
 
    /// Nonconforming edge/face within a bigger edge/face.
@@ -175,8 +179,9 @@ public:
       int edge_flags; ///< edge orientation flags
       DenseMatrix point_matrix; ///< position within the master edge/face
 
-      Slave(int index, int element, int local)
-         : MeshId(index, element, local), master(-1), edge_flags(0) {}
+      Slave(int index, int element, char local, char geom)
+         : MeshId(index, element, local, geom)
+         , master(-1), edge_flags(0) {}
 
       /// Return the point matrix oriented according to the master and slave edges
       void OrientedPointMatrix(DenseMatrix &oriented_matrix) const;
@@ -286,10 +291,11 @@ public:
        processor independent. TODO: this seems only partially true? */
    int GetEdgeNCOrientation(const MeshId &edge_id) const;
 
-   /// Return Mesh vertex and edge indices of a face identified by 'face_id'.
-   void GetFaceVerticesEdges(const MeshId &face_id,
-                             int vert_index[4], int edge_index[4],
-                             int edge_orientation[4]) const;
+   /** Return Mesh vertex and edge indices of a face identified by 'face_id'.
+       The return value is the number of face vertices. */
+   int GetFaceVerticesEdges(const MeshId &face_id,
+                            int vert_index[4], int edge_index[4],
+                            int edge_orientation[4]) const;
 
    /** Given an edge (by its vertex indices v1 and v2) return the first
        (geometric) parent edge that exists in the Mesh or -1 if there is no such
@@ -305,14 +311,29 @@ public:
                                    Array<int> &bdr_vertices,
                                    Array<int> &bdr_edges);
 
-   /// Return the type of elements in the mesh.
-   Geometry::Type GetElementGeometry() const { return elements[0].geom; }
+   /// Return element geometry type. @a index is the Mesh element number.
+   Geometry::Type GetElementGeometry(int index) const
+   { return elements[leaf_elements[index]].Geom(); }
 
-   Geometry::Type GetFaceGeometry() const { return Geometry::SQUARE; }
+   /// Return face geometry type. @a index is the Mesh face number.
+   Geometry::Type GetFaceGeometry(int index) const
+   { return Geometry::Type(face_geom[index]); }
+
+   /// Return the number of root elements.
+   int GetNumRootElements() { return root_state.Size(); }
 
    /// Return the distance of leaf 'i' from the root.
    int GetElementDepth(int i) const;
 
+   /** Return the size reduction compared to the root element (ignoring local
+       stretching and curvature). */
+   int GetElementSizeReduction(int i) const;
+
+   /// Return the faces and face attributes of leaf element 'i'.
+   void GetElementFacesAttributes(int i, Array<int> &faces,
+                                  Array<int> &fattr) const;
+
+
    /// I/O: Print the "vertex_parents" section of the mesh file (ver. >= 1.1).
    void PrintVertexParents(std::ostream &out) const;
 
@@ -339,15 +360,17 @@ public:
 
    void PrintStats(std::ostream &out = mfem::out) const;
 
+   typedef int64_t RefCoord;
+
 
 protected: // interface for Mesh to be able to construct itself from NCMesh
 
    friend class Mesh;
 
    /// Return the basic Mesh arrays for the current finest level.
-   void GetMeshComponents(Array<mfem::Vertex>& mvertices,
-                          Array<mfem::Element*>& melements,
-                          Array<mfem::Element*>& mboundary) const;
+   void GetMeshComponents(Array<mfem::Vertex> &mvertices,
+                          Array<mfem::Element*> &melements,
+                          Array<mfem::Element*> &mboundary) const;
 
    /** Get edge and face numbering from 'mesh' (i.e., set all Edge::index and
        Face::index) after a new mesh was created from us. */
@@ -358,6 +381,7 @@ protected: // implementation
 
    int Dim, spaceDim; ///< dimensions of the elements and the vertex coordinates
    bool Iso; ///< true if the mesh only contains isotropic refinements
+   int Geoms; ///< bit mask of element geometries present, see InitGeomFlags()
 
    /** A Node can hold a vertex, an edge, or both. Elements directly point to
        their corner nodes, but edge nodes also exist and can be accessed using
@@ -411,7 +435,7 @@ protected: // implementation
        to its vertex nodes. */
    struct Element
    {
-      Geometry::Type geom; ///< Geometry::Type of the element
+      char geom;     ///< Geometry::Type of the element (char for storage only)
       char ref_type; ///< bit mask of X,Y,Z refinements (bits 0,1,2 respectively)
       char flag;     ///< generic flag/marker, can be used by algorithms
       int index;     ///< element number in the Mesh, -1 if refined
@@ -425,6 +449,8 @@ protected: // implementation
       int parent; ///< parent element, -1 if this is a root element, -2 if free
 
       Element(Geometry::Type geom, int attr);
+
+      Geometry::Type Geom() const { return Geometry::Type(geom); }
    };
 
    // primary data
@@ -470,6 +496,7 @@ protected: // implementation
    NCList vertex_list; ///< lazy-initialized list of vertices, see GetVertexList
 
    Array<int> boundary_faces; ///< subset of all faces, set by BuildFaceList
+   Array<char> face_geom; ///< face geometry by face index, set by OnMeshUpdated
 
    Table element_vertex; ///< leaf-element to vertex table, see FindSetNeighbors
 
@@ -491,10 +518,15 @@ protected: // implementation
    virtual int GetNumGhostElements() const { return 0; }
    virtual int GetNumGhostVertices() const { return 0; }
 
+   void InitGeomFlags();
+   bool HavePrisms() const { return Geoms & (1 << Geometry::PRISM); }
+
 
    // refinement/derefinement
 
    Array<Refinement> ref_stack; ///< stack of scheduled refinements (temporary)
+   HashTable<Node> shadow; ///< temporary storage for reparented nodes
+   Array<Triple<int, int, int> > reparents; ///< scheduled node reparents (tmp)
 
    Table derefinements; ///< possible derefinements, see GetDerefinementTable
 
@@ -519,13 +551,16 @@ protected: // implementation
    }
 
    int NewHexahedron(int n0, int n1, int n2, int n3,
-                     int n4, int n5, int n6, int n7,
-                     int attr,
+                     int n4, int n5, int n6, int n7, int attr,
                      int fattr0, int fattr1, int fattr2,
                      int fattr3, int fattr4, int fattr5);
 
-   int NewQuadrilateral(int n0, int n1, int n2, int n3,
-                        int attr,
+   int NewWedge(int n0, int n1, int n2,
+                int n3, int n4, int n5, int attr,
+                int fattr0, int fattr1,
+                int fattr2, int fattr3, int fattr4);
+
+   int NewQuadrilateral(int n0, int n1, int n2, int n3, int attr,
                         int eattr0, int eattr1, int eattr2, int eattr3);
 
    int NewTriangle(int n0, int n1, int n2,
@@ -533,57 +568,62 @@ protected: // implementation
 
    mfem::Element* NewMeshElement(int geom) const;
 
-   int GetMidEdgeNode(int vn1, int vn2);
-   int GetMidFaceNode(int en1, int en2, int en3, int en4);
+   int QuadFaceSplitType(int v1, int v2, int v3, int v4, int mid[5]
+                         = NULL /*optional output of mid-edge nodes*/) const;
 
-   int FaceSplitType(int v1, int v2, int v3, int v4, int mid[4]
-                     = NULL /*optional output of mid-edge nodes*/) const;
+   bool TriFaceSplit(int v1, int v2, int v3, int mid[3] = NULL) const;
 
    void ForceRefinement(int vn1, int vn2, int vn3, int vn4);
 
+   void FindEdgeElements(int vn1, int vn2, int vn3, int vn4,
+                         Array<MeshId> &prisms) const;
+
+   void CheckAnisoPrism(int vn1, int vn2, int vn3, int vn4,
+                        const Refinement *refs, int nref);
+
    void CheckAnisoFace(int vn1, int vn2, int vn3, int vn4,
                        int mid12, int mid34, int level = 0);
 
    void CheckIsoFace(int vn1, int vn2, int vn3, int vn4,
                      int en1, int en2, int en3, int en4, int midf);
 
-   void RefElement(int elem);
-   void UnrefElement(int elem, Array<int> &elemFaces);
+   void ReparentNode(int node, int new_p1, int new_p2);
+
+   int FindMidEdgeNode(int node1, int node2) const;
+   int GetMidEdgeNode(int node1, int node2);
+
+   int GetMidFaceNode(int en1, int en2, int en3, int en4);
+
+   void ReferenceElement(int elem);
+   void UnreferenceElement(int elem, Array<int> &elemFaces);
 
    Face* GetFace(Element &elem, int face_no);
    void RegisterFaces(int elem, int *fattr = NULL);
    void DeleteUnusedFaces(const Array<int> &elemFaces);
 
-   int FindAltParents(int node1, int node2);
-
-   bool NodeSetX1(int node, int* n);
-   bool NodeSetX2(int node, int* n);
-   bool NodeSetY1(int node, int* n);
-   bool NodeSetY2(int node, int* n);
-   bool NodeSetZ1(int node, int* n);
-   bool NodeSetZ2(int node, int* n);
-
    void CollectDerefinements(int elem, Array<Connection> &list);
 
    /// Return el.node[index] correctly, even if the element is refined.
    int RetrieveNode(const Element &el, int index);
 
-   /// Extended version of find_node: works if 'el' is refined; optional abort.
-   int FindNodeExt(const Element &el, int node, bool abort = false);
+   /// Extended version of find_node: works if 'el' is refined.
+   int FindNodeExt(const Element &el, int node, bool abort = true);
 
 
    // face/edge lists
 
    static int find_node(const Element &el, int node);
-   static int find_element_edge(const Element &el, int vn0, int vn1);
-   static int find_hex_face(int a, int b, int c);
+   static int find_element_edge(const Element &el, int vn0, int vn1,
+                                bool abort = true);
+   static int find_local_face(int geom, int a, int b, int c);
 
    int ReorderFacePointMat(int v0, int v1, int v2, int v3,
                            int elem, DenseMatrix& mat) const;
    struct PointMatrix;
-   void TraverseFace(int vn0, int vn1, int vn2, int vn3,
-                     const PointMatrix& pm, int level);
-
+   void TraverseQuadFace(int vn0, int vn1, int vn2, int vn3,
+                         const PointMatrix& pm, int level, Face* eface[4]);
+   void TraverseTriFace(int vn0, int vn1, int vn2,
+                        const PointMatrix& pm, int level);
    void TraverseEdge(int vn0, int vn1, double t0, double t1, int flags,
                      int level);
 
@@ -626,8 +666,9 @@ protected: // implementation
 
 
    void CollectEdgeVertices(int v0, int v1, Array<int> &indices);
-   void CollectFaceVertices(int v0, int v1, int v2, int v3,
-                            Array<int> &indices);
+   void CollectTriFaceVertices(int v0, int v1, int v2, Array<int> &indices);
+   void CollectQuadFaceVertices(int v0, int v1, int v2, int v3,
+                                Array<int> &indices);
    void BuildElementToVertexTable();
 
    void UpdateElementToVertexTable()
@@ -635,6 +676,15 @@ protected: // implementation
       if (element_vertex.Size() < 0) { BuildElementToVertexTable(); }
    }
 
+   int GetVertexRootCoord(int elem, RefCoord coord[3]) const;
+   void CollectIncidentElements(int elem, const RefCoord coord[3],
+                                Array<int> &list) const;
+
+   /** Return elements neighboring to a local vertex of element 'elem'. Only
+       elements from within the same refinement tree ('cousins') are returned.
+       Complexity is proportional to the depth of elem's refinement tree. */
+   void FindVertexCousins(int elem, int local, Array<int> &cousins) const;
+
 
    // coarse/fine transformations
 
@@ -690,6 +740,13 @@ protected: // implementation
       PointMatrix(const Point& p0, const Point& p1, const Point& p2, const Point& p3)
       { np = 4; points[0] = p0; points[1] = p1; points[2] = p2; points[3] = p3; }
 
+      PointMatrix(const Point& p0, const Point& p1, const Point& p2,
+                  const Point& p3, const Point& p4, const Point& p5)
+      {
+         np = 6;
+         points[0] = p0; points[1] = p1; points[2] = p2;
+         points[3] = p3; points[4] = p4; points[5] = p5;
+      }
       PointMatrix(const Point& p0, const Point& p1, const Point& p2,
                   const Point& p3, const Point& p4, const Point& p5,
                   const Point& p6, const Point& p7)
@@ -707,11 +764,13 @@ protected: // implementation
 
    static PointMatrix pm_tri_identity;
    static PointMatrix pm_quad_identity;
+   static PointMatrix pm_prism_identity;
    static PointMatrix pm_hex_identity;
 
-   static const PointMatrix& GetGeomIdentity(int geom);
+   static const PointMatrix& GetGeomIdentity(Geometry::Type geom);
 
-   void GetPointMatrix(int geom, const char* ref_path, DenseMatrix& matrix);
+   void GetPointMatrix(Geometry::Type geom, const char* ref_path,
+                       DenseMatrix& matrix);
 
    typedef std::map<std::string, int> RefPathMap;
 
@@ -748,9 +807,10 @@ protected: // implementation
 
    void FindFaceNodes(int face, int node[4]);
 
-   int  EdgeSplitLevel(int vn1, int vn2) const;
-   void FaceSplitLevel(int vn1, int vn2, int vn3, int vn4,
-                       int& h_level, int& v_level) const;
+   int EdgeSplitLevel(int vn1, int vn2) const;
+   int TriFaceSplitLevel(int vn1, int vn2, int vn3) const;
+   void QuadFaceSplitLevel(int vn1, int vn2, int vn3, int vn4,
+                           int& h_level, int& v_level) const;
 
    void CountSplits(int elem, int splits[3]) const;
    void GetLimitRefinements(Array<Refinement> &refinements, int max_level);
@@ -765,9 +825,10 @@ protected: // implementation
        (triangles, quads, cubes) */
    struct GeomInfo
    {
-      int nv, ne, nf, nfv; // number of: vertices, edges, faces, face vertices
-      int edges[12][2];    // edge vertices (up to 12 edges)
-      int faces[6][4];     // face vertices (up to 6 faces)
+      int nv, ne, nf;   // number of: vertices, edges, faces
+      int edges[12][2]; // edge vertices (up to 12 edges)
+      int faces[6][4];  // face vertices (up to 6 faces)
+      int nfv[6];       // number of face vertices
 
       bool initialized;
       GeomInfo() : initialized(false) {}
@@ -776,7 +837,7 @@ protected: // implementation
 
    static GeomInfo GI[Geometry::NumGeom];
 
-   static GeomInfo &gi_hex, &gi_quad, &gi_tri;
+   static GeomInfo &gi_hex, &gi_wedge, &gi_quad, &gi_tri;
 
 #ifdef MFEM_DEBUG
 public:

mesh/ncmesh_tables.hpp

@@ -0,0 +1,469 @@
+// Copyright (c) 2010, Lawrence Livermore National Security, LLC. Produced at
+// the Lawrence Livermore National Laboratory. LLNL-CODE-443211. All Rights
+// reserved. See file COPYRIGHT for details.
+//
+// This file is part of the MFEM library. For more information and source code
+// availability see http://mfem.org.
+//
+// MFEM is free software; you can redistribute it and/or modify it under the
+// terms of the GNU Lesser General Public License (as published by the Free
+// Software Foundation) version 2.1 dated February 1999.
+
+namespace mfem
+{
+
+static int ref_type_num_children[8] = { 0, 2, 2, 4, 2, 4, 4, 8 };
+
+
+// derefinement tables
+
+static int quad_deref_table[3][4 + 4] =
+{
+   { 0, 1, 1, 0, /**/ 1, 1, 0, 0 }, // 1 - X
+   { 0, 0, 1, 1, /**/ 0, 0, 1, 1 }, // 2 - Y
+   { 0, 1, 2, 3, /**/ 1, 1, 3, 3 }  // 3 - iso
+};
+
+static int hex_deref_table[7][8 + 6] =
+{
+   { 0, 1, 1, 0, 0, 1, 1, 0, /**/ 1, 1, 1, 0, 0, 0 }, // 1 - X
+   { 0, 0, 1, 1, 0, 0, 1, 1, /**/ 0, 0, 0, 1, 1, 1 }, // 2 - Y
+   { 0, 1, 2, 3, 0, 1, 2, 3, /**/ 1, 1, 1, 3, 3, 3 }, // 3 - XY
+   { 0, 0, 0, 0, 1, 1, 1, 1, /**/ 0, 0, 0, 1, 1, 1 }, // 4 - Z
+   { 0, 1, 1, 0, 3, 2, 2, 3, /**/ 1, 1, 1, 3, 3, 3 }, // 5 - XZ
+   { 0, 0, 1, 1, 2, 2, 3, 3, /**/ 0, 0, 0, 3, 3, 3 }, // 6 - YZ
+   { 0, 1, 2, 3, 4, 5, 6, 7, /**/ 1, 1, 1, 7, 7, 7 }  // 7 - iso
+};
+
+static int prism_deref_table[7][6 + 5] =
+{
+   {-1,-1,-1,-1,-1,-1, /**/ -1,-1,-1,-1,-1 }, // 1
+   {-1,-1,-1,-1,-1,-1, /**/ -1,-1,-1,-1,-1 }, // 2
+   { 0, 1, 2, 0, 1, 2, /**/  0, 0, 0, 1, 0 }, // 3 - XY
+   { 0, 0, 0, 1, 1, 1, /**/  0, 1, 0, 0, 0 }, // 4 - Z
+   {-1,-1,-1,-1,-1,-1, /**/ -1,-1,-1,-1,-1 }, // 5
+   {-1,-1,-1,-1,-1,-1, /**/ -1,-1,-1,-1,-1 }, // 6
+   { 0, 1, 2, 4, 5, 6, /**/  0, 5, 0, 5, 0 }  // 7 - iso
+};
+
+
+// child ordering tables
+
+static char quad_hilbert_child_order[8][4] =
+{
+   {0,1,2,3}, {0,3,2,1}, {1,2,3,0}, {1,0,3,2},
+   {2,3,0,1}, {2,1,0,3}, {3,0,1,2}, {3,2,1,0}
+};
+
+static char quad_hilbert_child_state[8][4] =
+{
+   {1,0,0,5}, {0,1,1,4}, {3,2,2,7}, {2,3,3,6},
+   {5,4,4,1}, {4,5,5,0}, {7,6,6,3}, {6,7,7,2}
+};
+
+static char hex_hilbert_child_order[24][8] =
+{
+   {0,1,2,3,7,6,5,4}, {0,3,7,4,5,6,2,1}, {0,4,5,1,2,6,7,3},
+   {1,0,3,2,6,7,4,5}, {1,2,6,5,4,7,3,0}, {1,5,4,0,3,7,6,2},
+   {2,1,5,6,7,4,0,3}, {2,3,0,1,5,4,7,6}, {2,6,7,3,0,4,5,1},
+   {3,0,4,7,6,5,1,2}, {3,2,1,0,4,5,6,7}, {3,7,6,2,1,5,4,0},
+   {4,0,1,5,6,2,3,7}, {4,5,6,7,3,2,1,0}, {4,7,3,0,1,2,6,5},
+   {5,1,0,4,7,3,2,6}, {5,4,7,6,2,3,0,1}, {5,6,2,1,0,3,7,4},
+   {6,2,3,7,4,0,1,5}, {6,5,1,2,3,0,4,7}, {6,7,4,5,1,0,3,2},
+   {7,3,2,6,5,1,0,4}, {7,4,0,3,2,1,5,6}, {7,6,5,4,0,1,2,3}
+};
+
+static char hex_hilbert_child_state[24][8] =
+{
+   {1,2,2,7,7,21,21,17},     {2,0,0,22,22,16,16,8},    {0,1,1,15,15,6,6,23},
+   {4,5,5,10,10,18,18,14},   {5,3,3,19,19,13,13,11},   {3,4,4,12,12,9,9,20},
+   {8,7,7,17,17,23,23,2},    {6,8,8,0,0,15,15,22},     {7,6,6,21,21,1,1,16},
+   {11,10,10,14,14,20,20,5}, {9,11,11,3,3,12,12,19},   {10,9,9,18,18,4,4,13},
+   {13,14,14,5,5,19,19,10},  {14,12,12,20,20,11,11,4}, {12,13,13,9,9,3,3,18},
+   {16,17,17,2,2,22,22,7},   {17,15,15,23,23,8,8,1},   {15,16,16,6,6,0,0,21},
+   {20,19,19,11,11,14,14,3}, {18,20,20,4,4,10,10,12},  {19,18,18,13,13,5,5,9},
+   {23,22,22,8,8,17,17,0},   {21,23,23,1,1,7,7,15},    {22,21,21,16,16,2,2,6}
+};
+
+
+// child/parent reference domain transforms
+
+typedef NCMesh::RefCoord RefCoord;
+
+// reference domain coordinates as fixed point numbers
+const RefCoord T_HALF = (1ll << 59);
+const RefCoord T_ONE = (1ll << 60);
+const RefCoord T_TWO = (1ll << 61);
+
+// (scaling factors have a different fixed point multiplier)
+const RefCoord S_HALF = 1;
+const RefCoord S_ONE = 2;
+const RefCoord S_TWO = 4;
+
+static RefCoord tri_corners[3][3] =
+{
+   {    0,     0, 0},
+   {T_ONE,     0, 0},
+   {    0, T_ONE, 0}
+};
+
+static RefCoord quad_corners[4][3] =
+{
+   {    0,     0, 0},
+   {T_ONE,     0, 0},
+   {T_ONE, T_ONE, 0},
+   {    0, T_ONE, 0}
+};
+
+static RefCoord hex_corners[8][3] =
+{
+   {    0,     0,     0},
+   {T_ONE,     0,     0},
+   {T_ONE, T_ONE,     0},
+   {    0, T_ONE,     0},
+   {    0,     0, T_ONE},
+   {T_ONE,     0, T_ONE},
+   {T_ONE, T_ONE, T_ONE},
+   {    0, T_ONE, T_ONE}
+};
+
+static RefCoord prism_corners[6][3] =
+{
+   {    0,     0,     0},
+   {T_ONE,     0,     0},
+   {    0, T_ONE,     0},
+   {    0,     0, T_ONE},
+   {T_ONE,     0, T_ONE},
+   {    0, T_ONE, T_ONE}
+};
+
+typedef RefCoord RefPoint[3];
+static RefPoint* geom_corners[7] =
+{
+   NULL, // point
+   NULL, // segment
+   tri_corners,
+   quad_corners,
+   NULL, // tetrahedron
+   hex_corners,
+   prism_corners
+};
+
+// reference domain transform: 3 scales, 3 translations
+struct RefTrf
+{
+   RefCoord s[3], t[3];
+
+   void Apply(const RefCoord src[3], RefCoord dst[3]) const;
+};
+
+static RefTrf quad_parent_rt1[2] =
+{
+   { {S_HALF, S_ONE, 0}, {     0, 0, 0} },
+   { {S_HALF, S_ONE, 0}, {T_HALF, 0, 0} }
+};
+
+static RefTrf quad_child_rt1[2] =
+{
+   { {S_TWO, S_ONE, 0}, {     0, 0, 0} },
+   { {S_TWO, S_ONE, 0}, {-T_ONE, 0, 0} }
+};
+
+static RefTrf quad_parent_rt2[2] =
+{
+   { {S_ONE, S_HALF, 0}, {0,      0, 0} },
+   { {S_ONE, S_HALF, 0}, {0, T_HALF, 0} }
+};
+
+static RefTrf quad_child_rt2[2] =
+{
+   { {S_ONE, S_TWO, 0}, {0,      0, 0} },
+   { {S_ONE, S_TWO, 0}, {0, -T_ONE, 0} }
+};
+
+static RefTrf quad_parent_rt3[4] =
+{
+   { {S_HALF, S_HALF, 0}, {     0,      0, 0} },
+   { {S_HALF, S_HALF, 0}, {T_HALF,      0, 0} },
+   { {S_HALF, S_HALF, 0}, {T_HALF, T_HALF, 0} },
+   { {S_HALF, S_HALF, 0}, {     0, T_HALF, 0} }
+};
+
+static RefTrf quad_child_rt3[4] =
+{
+   { {S_TWO, S_TWO, 0}, {     0,      0, 0} },
+   { {S_TWO, S_TWO, 0}, {-T_ONE,      0, 0} },
+   { {S_TWO, S_TWO, 0}, {-T_ONE, -T_ONE, 0} },
+   { {S_TWO, S_TWO, 0}, {     0, -T_ONE, 0} }
+};
+
+static RefTrf* quad_parent[4] =
+{
+   NULL,
+   quad_parent_rt1,
+   quad_parent_rt2,
+   quad_parent_rt3
+};
+
+static RefTrf* quad_child[4] =
+{
+   NULL,
+   quad_child_rt1,
+   quad_child_rt2,
+   quad_child_rt3
+};
+
+static RefTrf hex_parent_rt1[2] =
+{
+   { {S_HALF, S_ONE, S_ONE}, {     0, 0, 0} },
+   { {S_HALF, S_ONE, S_ONE}, {T_HALF, 0, 0} }
+};
+
+static RefTrf hex_child_rt1[2] =
+{
+   { {S_TWO, S_ONE, S_ONE}, {     0, 0, 0} },
+   { {S_TWO, S_ONE, S_ONE}, {-T_ONE, 0, 0} }
+};
+
+static RefTrf hex_parent_rt2[2] =
+{
+   { {S_ONE, S_HALF, S_ONE}, {0,      0, 0} },
+   { {S_ONE, S_HALF, S_ONE}, {0, T_HALF, 0} }
+};
+
+static RefTrf hex_child_rt2[2] =
+{
+   { {S_ONE, S_TWO, S_ONE}, {0,      0, 0} },
+   { {S_ONE, S_TWO, S_ONE}, {0, -T_ONE, 0} }
+};
+
+static RefTrf hex_parent_rt3[4] =
+{
+   { {S_HALF, S_HALF, S_ONE}, {     0,      0, 0} },
+   { {S_HALF, S_HALF, S_ONE}, {T_HALF,      0, 0} },
+   { {S_HALF, S_HALF, S_ONE}, {T_HALF, T_HALF, 0} },
+   { {S_HALF, S_HALF, S_ONE}, {     0, T_HALF, 0} }
+};
+
+static RefTrf hex_child_rt3[4] =
+{
+   { {S_TWO, S_TWO, S_ONE}, {     0,      0, 0} },
+   { {S_TWO, S_TWO, S_ONE}, {-T_ONE,      0, 0} },
+   { {S_TWO, S_TWO, S_ONE}, {-T_ONE, -T_ONE, 0} },
+   { {S_TWO, S_TWO, S_ONE}, {     0, -T_ONE, 0} }
+};
+
+static RefTrf hex_parent_rt4[2] =
+{
+   { {S_ONE, S_ONE, S_HALF}, {0, 0,      0} },
+   { {S_ONE, S_ONE, S_HALF}, {0, 0, T_HALF} }
+};
+
+static RefTrf hex_child_rt4[2] =
+{
+   { {S_ONE, S_ONE, S_TWO}, {0, 0,      0} },
+   { {S_ONE, S_ONE, S_TWO}, {0, 0, -T_ONE} }
+};
+
+static RefTrf hex_parent_rt5[4] =
+{
+   { {S_HALF, S_ONE, S_HALF}, {     0, 0,      0} },
+   { {S_HALF, S_ONE, S_HALF}, {T_HALF, 0,      0} },
+   { {S_HALF, S_ONE, S_HALF}, {T_HALF, 0, T_HALF} },
+   { {S_HALF, S_ONE, S_HALF}, {     0, 0, T_HALF} }
+};
+
+static RefTrf hex_child_rt5[4] =
+{
+   { {S_TWO, S_ONE, S_TWO}, {     0, 0,      0} },
+   { {S_TWO, S_ONE, S_TWO}, {-T_ONE, 0,      0} },
+   { {S_TWO, S_ONE, S_TWO}, {-T_ONE, 0, -T_ONE} },
+   { {S_TWO, S_ONE, S_TWO}, {     0, 0, -T_ONE} }
+};
+
+static RefTrf hex_parent_rt6[4] =
+{
+   { {S_ONE, S_HALF, S_HALF}, {0,      0,      0} },
+   { {S_ONE, S_HALF, S_HALF}, {0, T_HALF,      0} },
+   { {S_ONE, S_HALF, S_HALF}, {0,      0, T_HALF} },
+   { {S_ONE, S_HALF, S_HALF}, {0, T_HALF, T_HALF} }
+};
+
+static RefTrf hex_child_rt6[4] =
+{
+   { {S_ONE, S_TWO, S_TWO}, {0,      0,      0} },
+   { {S_ONE, S_TWO, S_TWO}, {0, -T_ONE,      0} },
+   { {S_ONE, S_TWO, S_TWO}, {0,      0, -T_ONE} },
+   { {S_ONE, S_TWO, S_TWO}, {0, -T_ONE, -T_ONE} }
+};
+
+static RefTrf hex_parent_rt7[8] =
+{
+   { {S_HALF, S_HALF, S_HALF}, {     0,      0,      0} },
+   { {S_HALF, S_HALF, S_HALF}, {T_HALF,      0,      0} },
+   { {S_HALF, S_HALF, S_HALF}, {T_HALF, T_HALF,      0} },
+   { {S_HALF, S_HALF, S_HALF}, {     0, T_HALF,      0} },
+   { {S_HALF, S_HALF, S_HALF}, {     0,      0, T_HALF} },
+   { {S_HALF, S_HALF, S_HALF}, {T_HALF,      0, T_HALF} },
+   { {S_HALF, S_HALF, S_HALF}, {T_HALF, T_HALF, T_HALF} },
+   { {S_HALF, S_HALF, S_HALF}, {     0, T_HALF, T_HALF} }
+};
+
+static RefTrf hex_child_rt7[8] =
+{
+   { {S_TWO, S_TWO, S_TWO}, {     0,      0,      0} },
+   { {S_TWO, S_TWO, S_TWO}, {-T_ONE,      0,      0} },
+   { {S_TWO, S_TWO, S_TWO}, {-T_ONE, -T_ONE,      0} },
+   { {S_TWO, S_TWO, S_TWO}, {     0, -T_ONE,      0} },
+   { {S_TWO, S_TWO, S_TWO}, {     0,      0, -T_ONE} },
+   { {S_TWO, S_TWO, S_TWO}, {-T_ONE,      0, -T_ONE} },
+   { {S_TWO, S_TWO, S_TWO}, {-T_ONE, -T_ONE, -T_ONE} },
+   { {S_TWO, S_TWO, S_TWO}, {     0, -T_ONE, -T_ONE} }
+};
+
+static RefTrf* hex_parent[8] =
+{
+   NULL,
+   hex_parent_rt1,
+   hex_parent_rt2,
+   hex_parent_rt3,
+   hex_parent_rt4,
+   hex_parent_rt5,
+   hex_parent_rt6,
+   hex_parent_rt7
+};
+
+static RefTrf* hex_child[8] =
+{
+   NULL,
+   hex_child_rt1,
+   hex_child_rt2,
+   hex_child_rt3,
+   hex_child_rt4,
+   hex_child_rt5,
+   hex_child_rt6,
+   hex_child_rt7
+};
+
+static RefTrf tri_parent_rt3[4] =
+{
+   { { S_HALF,  S_HALF, 0}, {     0,      0, 0} },
+   { { S_HALF,  S_HALF, 0}, {T_HALF,      0, 0} },
+   { { S_HALF,  S_HALF, 0}, {     0, T_HALF, 0} },
+   { {-S_HALF, -S_HALF, 0}, {T_HALF, T_HALF, 0} }
+};
+
+static RefTrf tri_child_rt3[4] =
+{
+   { { S_TWO,  S_TWO, 0}, {     0,      0, 0} },
+   { { S_TWO,  S_TWO, 0}, {-T_ONE,      0, 0} },
+   { { S_TWO,  S_TWO, 0}, {     0, -T_ONE, 0} },
+   { {-S_TWO, -S_TWO, 0}, { T_ONE,  T_ONE, 0} }
+};
+
+static RefTrf* tri_parent[4] =
+{
+   NULL, NULL, NULL,
+   tri_parent_rt3
+};
+
+static RefTrf* tri_child[4] =
+{
+   NULL, NULL, NULL,
+   tri_child_rt3
+};
+
+static RefTrf prism_parent_rt3[4] =
+{
+   { { S_HALF,  S_HALF, S_ONE}, {     0,      0, 0} },
+   { { S_HALF,  S_HALF, S_ONE}, {T_HALF,      0, 0} },
+   { { S_HALF,  S_HALF, S_ONE}, {     0, T_HALF, 0} },
+   { {-S_HALF, -S_HALF, S_ONE}, {T_HALF, T_HALF, 0} }
+};
+
+static RefTrf prism_child_rt3[4] =
+{
+   { { S_TWO,  S_TWO, S_ONE}, {     0,      0, 0} },
+   { { S_TWO,  S_TWO, S_ONE}, {-T_ONE,      0, 0} },
+   { { S_TWO,  S_TWO, S_ONE}, {     0, -T_ONE, 0} },
+   { {-S_TWO, -S_TWO, S_ONE}, { T_ONE,  T_ONE, 0} }
+};
+
+static RefTrf prism_parent_rt4[2] =
+{
+   { {S_ONE, S_ONE, S_HALF}, {0, 0,      0} },
+   { {S_ONE, S_ONE, S_HALF}, {0, 0, T_HALF} }
+};
+
+static RefTrf prism_child_rt4[2] =
+{
+   { {S_ONE, S_ONE, S_TWO}, {0, 0,      0} },
+   { {S_ONE, S_ONE, S_TWO}, {0, 0, -T_ONE} }
+};
+
+static RefTrf prism_parent_rt7[8] =
+{
+   { { S_HALF,  S_HALF, S_HALF}, {     0,      0,      0} },
+   { { S_HALF,  S_HALF, S_HALF}, {T_HALF,      0,      0} },
+   { { S_HALF,  S_HALF, S_HALF}, {     0, T_HALF,      0} },
+   { {-S_HALF, -S_HALF, S_HALF}, {T_HALF, T_HALF,      0} },
+   { { S_HALF,  S_HALF, S_HALF}, {     0,      0, T_HALF} },
+   { { S_HALF,  S_HALF, S_HALF}, {T_HALF,      0, T_HALF} },
+   { { S_HALF,  S_HALF, S_HALF}, {     0, T_HALF, T_HALF} },
+   { {-S_HALF, -S_HALF, S_HALF}, {T_HALF, T_HALF, T_HALF} }
+};
+
+static RefTrf prism_child_rt7[8] =
+{
+   { { S_TWO,  S_TWO, S_TWO}, {     0,      0,      0} },
+   { { S_TWO,  S_TWO, S_TWO}, {-T_ONE,      0,      0} },
+   { { S_TWO,  S_TWO, S_TWO}, {     0, -T_ONE,      0} },
+   { {-S_TWO, -S_TWO, S_TWO}, { T_ONE,  T_ONE,      0} },
+   { { S_TWO,  S_TWO, S_TWO}, {     0,      0, -T_ONE} },
+   { { S_TWO,  S_TWO, S_TWO}, {-T_ONE,      0, -T_ONE} },
+   { { S_TWO,  S_TWO, S_TWO}, {     0, -T_ONE, -T_ONE} },
+   { {-S_TWO, -S_TWO, S_TWO}, { T_ONE,  T_ONE, -T_ONE} }
+};
+
+static RefTrf* prism_parent[8] =
+{
+   NULL, NULL, NULL,
+   prism_parent_rt3,
+   prism_parent_rt4,
+   NULL, NULL,
+   prism_parent_rt7
+};
+
+static RefTrf* prism_child[8] =
+{
+   NULL, NULL, NULL,
+   prism_child_rt3,
+   prism_child_rt4,
+   NULL, NULL,
+   prism_child_rt7
+};
+
+static RefTrf** geom_parent[7] =
+{
+   NULL,
+   NULL,
+   tri_parent,
+   quad_parent,
+   NULL,
+   hex_parent,
+   prism_parent
+};
+
+static RefTrf** geom_child[7] =
+{
+   NULL,
+   NULL,
+   tri_child,
+   quad_child,
+   NULL,
+   hex_child,
+   prism_child
+};
+
+
+} // namespace mfem

mesh/nurbs.cpp

@@ -582,9 +582,19 @@ void NURBSPatch::KnotInsert(int dir, const KnotVector &newkv)
    }
 }
 
+void NURBSPatch::KnotInsert(Array<Vector *> &newkv)
+{
+   for (int dir = 0; dir < kv.Size(); dir++)
+   {
+      KnotInsert(dir, *newkv[dir]);
+   }
+}
+
 // Routine from "The NURBS book" - 2nd ed - Piegl and Tiller
 void NURBSPatch::KnotInsert(int dir, const Vector &knot)
 {
+   if (knot.Size() == 0 ) { return; }
+
    if (dir >= kv.Size() || dir < 0)
    {
       mfem_error("NURBSPatch::KnotInsert : Incorrect direction!");
@@ -2982,6 +2992,34 @@ void NURBSExtension::KnotInsert(Array<KnotVector *> &kv)
    }
 }
 
+void NURBSExtension::KnotInsert(Array<Vector *> &kv)
+{
+   Array<int> edges;
+   Array<int> orient;
+
+   Array<Vector *> pkv(Dimension());
+
+   for (int p = 0; p < patches.Size(); p++)
+   {
+      patchTopo->GetElementEdges(p, edges, orient);
+
+      if (Dimension()==2)
+      {
+         pkv[0] = kv[KnotInd(edges[0])];
+         pkv[1] = kv[KnotInd(edges[1])];
+      }
+      else
+      {
+         pkv[0] = kv[KnotInd(edges[0])];
+         pkv[1] = kv[KnotInd(edges[3])];
+         pkv[2] = kv[KnotInd(edges[8])];
+      }
+
+      patches[p]->KnotInsert(pkv);
+   }
+}
+
+
 void NURBSExtension::GetPatchNets(const Vector &coords, int vdim)
 {
    if (Dimension() == 2)

mesh/nurbs.hpp

@@ -119,7 +119,9 @@ public:
    void KnotInsert   (int dir, const KnotVector &knot);
    void KnotInsert   (int dir, const Vector     &knot);
 
+   void KnotInsert(Array<Vector *> &knot);
    void KnotInsert(Array<KnotVector *> &knot);
+
    void DegreeElevate(int t);
    void UniformRefinement();
 
@@ -389,6 +391,7 @@ public:
    void DegreeElevate(int rel_degree, int degree = 16);
    void UniformRefinement();
    void KnotInsert(Array<KnotVector *> &kv);
+   void KnotInsert(Array<Vector *> &kv);
 };
 
 

mesh/pmesh.cpp

@@ -2399,6 +2399,32 @@ int ParMesh::GetSharedFace(int sface) const
    }
 }
 
+// shift cyclically 3 integers a, b, c, so that the smallest of
+// order[a], order[b], order[c] is first
+static inline
+void Rotate3Indirect(int &a, int &b, int &c,
+                     const Array<std::int64_t> &order)
+{
+   if (order[a] < order[b])
+   {
+      if (order[a] > order[c])
+      {
+         ShiftRight(a, b, c);
+      }
+   }
+   else
+   {
+      if (order[b] < order[c])
+      {
+         ShiftRight(c, b, a);
+      }
+      else
+      {
+         ShiftRight(a, b, c);
+      }
+   }
+}
+
 void ParMesh::ReorientTetMesh()
 {
    if (Dim != 3 || !(meshgen & 1))
@@ -2406,7 +2432,109 @@ void ParMesh::ReorientTetMesh()
       return;
    }
 
-   Mesh::ReorientTetMesh();
+   DeleteLazyTables();
+
+   DSTable *old_v_to_v = NULL;
+   Table *old_elem_vert = NULL;
+
+   if (Nodes)
+   {
+      PrepareNodeReorder(&old_v_to_v, &old_elem_vert);
+   }
+
+   // create a GroupCommunicator over shared vertices
+   GroupCommunicator svert_comm(gtopo);
+   {
+      // initialize svert_comm
+      Table &gr_svert = svert_comm.GroupLDofTable();
+      // gr_svert differs from group_svert - the latter does not store gr. 0
+      gr_svert.SetDims(GetNGroups(), svert_lvert.Size());
+      gr_svert.GetI()[0] = 0;
+      for (int gr = 1; gr <= GetNGroups(); gr++)
+      {
+         gr_svert.GetI()[gr] = group_svert.GetI()[gr-1];
+      }
+      for (int k = 0; k < svert_lvert.Size(); k++)
+      {
+         gr_svert.GetJ()[k] = group_svert.GetJ()[k];
+      }
+      svert_comm.Finalize();
+   }
+
+   // communicate the local index of each shared vertex from the group master to
+   // other ranks in the group
+   Array<int> svert_master_rank(svert_lvert.Size());
+   Array<int> svert_master_index(svert_lvert);
+   {
+      for (int i = 0; i < group_svert.Size(); i++)
+      {
+         int rank = gtopo.GetGroupMasterRank(i+1);
+         for (int j = 0; j < group_svert.RowSize(i); j++)
+         {
+            svert_master_rank[group_svert.GetRow(i)[j]] = rank;
+         }
+      }
+      svert_comm.Bcast(svert_master_index);
+   }
+
+   // the pairs (master rank, master local index) define a globally consistent
+   // vertex ordering
+   Array<std::int64_t> glob_vert_order(vertices.Size());
+   {
+      Array<int> lvert_svert(vertices.Size());
+      lvert_svert = -1;
+      for (int i = 0; i < svert_lvert.Size(); i++)
+      {
+         lvert_svert[svert_lvert[i]] = i;
+      }
+
+      for (int i = 0; i < vertices.Size(); i++)
+      {
+         int s = lvert_svert[i];
+         if (s >= 0)
+         {
+            glob_vert_order[i] =
+               (std::int64_t(svert_master_rank[s]) << 32) + svert_master_index[s];
+         }
+         else
+         {
+            glob_vert_order[i] = (std::int64_t(MyRank) << 32) + i;
+         }
+      }
+   }
+
+   // rotate tetrahedra so that vertex zero is the lowest (global) index vertex,
+   // vertex 1 is the second lowest (global) index and vertices 2 and 3 preserve
+   // positive orientation of the element
+   for (int i = 0; i < NumOfElements; i++)
+   {
+      if (GetElementType(i) == Element::TETRAHEDRON)
+      {
+         int *v = elements[i]->GetVertices();
+
+         Rotate3Indirect(v[0], v[1], v[2], glob_vert_order);
+
+         if (glob_vert_order[v[0]] < glob_vert_order[v[3]])
+         {
+            Rotate3Indirect(v[1], v[2], v[3], glob_vert_order);
+         }
+         else
+         {
+            ShiftRight(v[0], v[1], v[3]);
+         }
+      }
+   }
+
+   // rotate also boundary triangles
+   for (int i = 0; i < NumOfBdrElements; i++)
+   {
+      if (GetBdrElementType(i) == Element::TRIANGLE)
+      {
+         int *v = boundary[i]->GetVertices();
+
+         Rotate3Indirect(v[0], v[1], v[2], glob_vert_order);
+      }
+   }
 
    const bool check_consistency = true;
    if (check_consistency)
@@ -2433,37 +2561,56 @@ void ParMesh::ReorientTetMesh()
       for (int i = 0; i < stria_flag.Size(); i++)
       {
          const int *v = shared_trias[i].v;
-         if (v[0] < v[1])
+         if (glob_vert_order[v[0]] < glob_vert_order[v[1]])
          {
-            stria_flag[i] = (v[0] < v[2]) ? 0 : 2;
+            stria_flag[i] = (glob_vert_order[v[0]] < glob_vert_order[v[2]]) ? 0 : 2;
          }
          else // v[1] < v[0]
          {
-            stria_flag[i] = (v[1] < v[2]) ? 1 : 2;
+            stria_flag[i] = (glob_vert_order[v[1]] < glob_vert_order[v[2]]) ? 1 : 2;
          }
       }
+
       Array<int> stria_master_flag(stria_flag);
       stria_comm.Bcast(stria_master_flag);
       for (int i = 0; i < stria_flag.Size(); i++)
       {
+         const int *v = shared_trias[i].v;
          MFEM_VERIFY(stria_flag[i] == stria_master_flag[i],
-                     "inconsistent vertex ordering found");
+                     "inconsistent vertex ordering found, shared triangle "
+                     << i << ": ("
+                     << v[0] << ", " << v[1] << ", " << v[2] << "), "
+                     << "local flag: " << stria_flag[i]
+                     << ", master flag: " << stria_master_flag[i]);
       }
    }
 
-   // Rotate shared triangle faces.
-   // Note that no communication is needed to ensure that the shared
-   // faces are rotated in the same way in both processors. This is
-   // automatic due to various things, e.g. the global to local vertex
-   // mapping preserves the global order; also the way new vertices
-   // are introduced during refinement is essential.
+   // rotate shared triangle faces
    for (int i = 0; i < shared_trias.Size(); i++)
    {
       int *v = shared_trias[i].v;
-      Rotate3(v[0], v[1], v[2]);
+
+      Rotate3Indirect(v[0], v[1], v[2], glob_vert_order);
    }
 
-   // The local edge and face numbering is changed therefore we need to
+   // finalize
+   if (!Nodes)
+   {
+      GetElementToFaceTable();
+      GenerateFaces();
+      if (el_to_edge)
+      {
+         NumOfEdges = GetElementToEdgeTable(*el_to_edge, be_to_edge);
+      }
+   }
+   else
+   {
+      DoNodeReorder(old_v_to_v, old_elem_vert);
+      delete old_elem_vert;
+      delete old_v_to_v;
+   }
+
+   // the local edge and face numbering is changed therefore we need to
    // update sedge_ledge and sface_lface.
    FinalizeParTopo();
 }
@@ -3055,6 +3202,9 @@ bool ParMesh::NonconformingDerefinement(Array<double> &elem_error,
    long glob_size = ReduceInt(derefs.Size());
    if (!glob_size) { return false; }
 
+   // Destroy face-neighbor data only when actually de-refining.
+   DeleteFaceNbrData();
+
    pncmesh->Derefine(derefs);
 
    ParMesh* mesh2 = new ParMesh(*pncmesh);
@@ -3082,7 +3232,18 @@ bool ParMesh::NonconformingDerefinement(Array<double> &elem_error,
    return true;
 }
 
+
 void ParMesh::Rebalance()
+{
+   RebalanceImpl(NULL); // default SFC-based partition
+}
+
+void ParMesh::Rebalance(const Array<int> &partition)
+{
+   RebalanceImpl(&partition);
+}
+
+void ParMesh::RebalanceImpl(const Array<int> *partition)
 {
    if (Conforming())
    {
@@ -3109,7 +3270,7 @@ void ParMesh::Rebalance()
 
    DeleteFaceNbrData();
 
-   pncmesh->Rebalance();
+   pncmesh->Rebalance(partition);
 
    ParMesh* pmesh2 = new ParMesh(*pncmesh);
    pncmesh->OnMeshUpdated(pmesh2);
@@ -4010,6 +4171,7 @@ void ParMesh::PrintAsOne(std::ostream &out)
           "# SQUARE      = 3\n"
           "# TETRAHEDRON = 4\n"
           "# CUBE        = 5\n"
+          "# PRISM       = 6\n"
           "#\n";
 
       out << "\ndimension\n" << Dim;
@@ -4108,6 +4270,15 @@ void ParMesh::PrintAsOne(std::ostream &out)
    {
       switch (Dim)
       {
+         case 1:
+            for (i = 0; i < svert_lvert.Size(); i++)
+            {
+               ints.Append(Geometry::POINT);
+               ints.Append(svert_lvert[i]);
+               ne++;
+            }
+            break;
+
          case 2:
             for (i = 0; i < shared_edges.Size(); i++)
             {

mesh/pmesh.hpp

@@ -141,6 +141,9 @@ protected:
    virtual bool NonconformingDerefinement(Array<double> &elem_error,
                                           double threshold, int nc_limit = 0,
                                           int op = 1);
+
+   void RebalanceImpl(const Array<int> *partition);
+
    void DeleteFaceNbrData();
 
    bool WantSkipSharedMaster(const NCMesh::Master &master) const;
@@ -286,9 +289,15 @@ public:
    /// Utility function: sum integers from all processors (Allreduce).
    virtual long ReduceInt(int value) const;
 
-   /// Load balance the mesh. NC meshes only.
+   /** Load balance the mesh by equipartitioning the global space-filling
+       sequence of elements. Works for nonconforming meshes only. */
    void Rebalance();
 
+   /** Load balance a nonconforming mesh using a user-defined partition.
+       Each local element 'i' is migrated to processor rank 'partition[i]',
+       for 0 <= i < GetNE(). */
+   void Rebalance(const Array<int> &partition);
+
    /** Print the part of the mesh in the calling processor adding the interface
        as boundary (for visualization purposes) using the mfem v1.0 format. */
    virtual void Print(std::ostream &out = mfem::out) const;

mesh/pncmesh.cpp

@@ -156,7 +156,7 @@ void ParNCMesh::UpdateVertices()
       Element &el = elements[leaf_elements[i]];
       if (el.rank == MyRank)
       {
-         for (int j = 0; j < GI[(int) el.geom].nv; j++)
+         for (int j = 0; j < GI[el.Geom()].nv; j++)
          {
             int &vindex = nodes[el.node[j]].vert_index;
             if (vindex < 0) { vindex = NVertices++; }
@@ -227,6 +227,30 @@ void ParNCMesh::OnMeshUpdated(Mesh *mesh)
       MFEM_ASSERT(NFaces == NEdges, "");
       MFEM_ASSERT(NGhostFaces == NGhostEdges, "");
    }
+
+   // update face_geom for ghost faces
+   face_geom.SetSize(NFaces + NGhostFaces, Geometry::SQUARE);
+   for (int i = 0; i < NGhostElements; i++)
+   {
+      Element &el = elements[leaf_elements[NElements + i]]; // ghost element
+      GeomInfo &gi = GI[el.Geom()];
+
+      for (int j = 0; j < gi.nf; j++)
+      {
+         const int *fv = gi.faces[j];
+         Face* face = faces.Find(el.node[fv[0]], el.node[fv[1]],
+                                 el.node[fv[2]], el.node[fv[3]]);
+         MFEM_ASSERT(face, "face not found!");
+
+         static const Geometry::Type types[5] =
+         {
+            Geometry::INVALID, Geometry::INVALID,
+            Geometry::SEGMENT, Geometry::TRIANGLE, Geometry::SQUARE
+         };
+
+         face_geom[face->index] = types[gi.nfv[j]];
+      }
+   }
 }
 
 void ParNCMesh::ElementSharesFace(int elem, int local, int face)
@@ -257,6 +281,9 @@ void ParNCMesh::BuildFaceList()
    // This is an extension of NCMesh::BuildFaceList() which also determines
    // face ownership and prepares face processor groups.
 
+   // (special case for prisms: to be able to handle edge-face constraints)
+   if (HavePrisms()) { GetEdgeList(); }
+
    int nfaces = NFaces + NGhostFaces;
 
    tmp_owner.SetSize(nfaces);
@@ -416,13 +443,25 @@ void ParNCMesh::MakeSharedList(const NCList &list, NCList &shared)
    for (unsigned i = 0; i < list.masters.size(); i++)
    {
       const Master &master = list.masters[i];
-      char master_old_flag = tmp_shared_flag[master.index];
+      char &master_flag = tmp_shared_flag[master.index];
+      char master_old_flag = master_flag;
 
       for (int j = master.slaves_begin; j < master.slaves_end; j++)
       {
-         char &slave_flag = tmp_shared_flag[list.slaves[j].index];
-         tmp_shared_flag[master.index] |= slave_flag;
-         slave_flag |= master_old_flag;
+         int si = list.slaves[j].index;
+         if (si >= 0)
+         {
+            char &slave_flag = tmp_shared_flag[si];
+            master_flag |= slave_flag;
+            slave_flag |= master_old_flag;
+         }
+         else // special case: prism edge-face constraint
+         {
+            if (entity_owner[1][-1-si] != MyRank)
+            {
+               master_flag |= 0x2;
+            }
+         }
       }
    }
 
@@ -444,7 +483,8 @@ void ParNCMesh::MakeSharedList(const NCList &list, NCList &shared)
    }
    for (unsigned i = 0; i < list.slaves.size(); i++)
    {
-      if (tmp_shared_flag[list.slaves[i].index] == 0x3)
+      int si = list.slaves[i].index;
+      if (si >= 0 && tmp_shared_flag[si] == 0x3)
       {
          shared.slaves.push_back(list.slaves[i]);
       }
@@ -594,7 +634,9 @@ void ParNCMesh::CalculatePMatrixGroups()
       ranks.SetSize(0);
       for (int j = master_face.slaves_begin; j < master_face.slaves_end; j++)
       {
-         int owner = entity_owner[2][face_list.slaves[j].index];
+         int si = face_list.slaves[j].index;
+         int owner = (si >= 0) ? entity_owner[2][si] // standard face dependency
+                     /*     */ : entity_owner[1][-1 - si]; // prism edge-face dep
          ranks.Append(groups[owner][0]);
       }
       ranks.Sort();
@@ -602,8 +644,8 @@ void ParNCMesh::CalculatePMatrixGroups()
 
       AddConnections(2, master_face.index, ranks);
 
-      GetFaceVerticesEdges(master_face, v, e, eo);
-      for (int j = 0; j < 4; j++)
+      int nfv = GetFaceVerticesEdges(master_face, v, e, eo);
+      for (int j = 0; j < nfv; j++)
       {
          AddConnections(0, v[j], ranks);
          AddConnections(1, e[j], ranks);
@@ -635,20 +677,23 @@ int ParNCMesh::get_face_orientation(Face &face, Element &e1, Element &e2,
    {
       // get local face number (remember that p1, p2, p3 are not in order, and
       // p4 is not stored)
-      int lf = find_hex_face(find_node(*e[i], face.p1),
-                             find_node(*e[i], face.p2),
-                             find_node(*e[i], face.p3));
+      int lf = find_local_face(e[i]->Geom(),
+                               find_node(*e[i], face.p1),
+                               find_node(*e[i], face.p2),
+                               find_node(*e[i], face.p3));
       // optional output
       if (local) { local[i] = lf; }
 
       // get node IDs for the face as seen from e[i]
-      const int* fv = GI[Geometry::CUBE].faces[lf];
+      const int* fv = GI[e[i]->Geom()].faces[lf];
       for (int j = 0; j < 4; j++)
       {
          ids[i][j] = e[i]->node[fv[j]];
       }
    }
-   return Mesh::GetQuadOrientation(ids[0], ids[1]);
+
+   return (ids[0][3] >= 0) ? Mesh::GetQuadOrientation(ids[0], ids[1])
+          /*            */ : Mesh::GetTriOrientation(ids[0], ids[1]);
 }
 
 void ParNCMesh::CalcFaceOrientations()
@@ -793,6 +838,8 @@ void ParNCMesh::NeighborProcessors(Array<int> &neighbors)
 {
    UpdateLayers();
 
+   // TODO: look at groups instead?
+
    std::set<int> ranks;
    for (int i = 0; i < ghost_layer.Size(); i++)
    {
@@ -804,30 +851,9 @@ void ParNCMesh::NeighborProcessors(Array<int> &neighbors)
 
 //// ParMesh compatibility /////////////////////////////////////////////////////
 
-struct CompareShared // TODO: use lambda when C++11 available
-{
-   const Array<int> &elem_local, &leaf_glob_order, &shared_local;
-
-   CompareShared
-   (const Array<int> &el, const Array<int> &lgo, const Array<int> &sl)
-      : elem_local(el), leaf_glob_order(lgo), shared_local(sl) {}
-
-   inline bool operator()(const int a, const int b)
-   {
-      int el_loc_a = elem_local[shared_local[a]];
-      int el_loc_b = elem_local[shared_local[b]];
-
-      int lgo_a = leaf_glob_order[el_loc_a >> 4];
-      int lgo_b = leaf_glob_order[el_loc_b >> 4];
-
-      if (lgo_a != lgo_b) { return lgo_a < lgo_b; }
-
-      return (el_loc_a & 0xf) < (el_loc_b & 0xf);
-   }
-};
-
 void ParNCMesh::MakeSharedTable(int ngroups, int ent, Array<int> &shared_local,
-                                Table &group_shared)
+                                Table &group_shared, Array<char> *entity_geom,
+                                char geom)
 {
    const Array<GroupId> &conf_group = entity_conf_group[ent];
 
@@ -839,6 +865,8 @@ void ParNCMesh::MakeSharedTable(int ngroups, int ent, Array<int> &shared_local,
    {
       if (conf_group[i])
       {
+         if (entity_geom && (*entity_geom)[i] != geom) { continue; }
+
          num_shared++;
          group_shared.AddAColumnInRow(conf_group[i]-1);
       }
@@ -852,6 +880,8 @@ void ParNCMesh::MakeSharedTable(int ngroups, int ent, Array<int> &shared_local,
    {
       if (conf_group[i])
       {
+         if (entity_geom && (*entity_geom)[i] != geom) { continue; }
+
          shared_local[j] = i;
          group_shared.AddConnection(conf_group[i]-1, j);
          j++;
@@ -866,8 +896,18 @@ void ParNCMesh::MakeSharedTable(int ngroups, int ent, Array<int> &shared_local,
       int *row = group_shared.GetRow(i);
 
       Array<int> ref_row(row, size);
-      ref_row.Sort(
-         CompareShared(entity_elem_local[ent], leaf_glob_order, shared_local));
+      ref_row.Sort([&](const int a, const int b)
+      {
+         int el_loc_a = entity_elem_local[ent][shared_local[a]];
+         int el_loc_b = entity_elem_local[ent][shared_local[b]];
+
+         int lgo_a = leaf_glob_order[el_loc_a >> 4];
+         int lgo_b = leaf_glob_order[el_loc_b >> 4];
+
+         if (lgo_a != lgo_b) { return lgo_a < lgo_b; }
+
+         return (el_loc_a & 0xf) < (el_loc_b & 0xf);
+      });
    }
 }
 
@@ -913,15 +953,16 @@ void ParNCMesh::GetConformingSharedStructures(ParMesh &pmesh)
    }
 
    // create shared to local index mappings and group tables
-   int ngroups = pmesh.gtopo.NGroups();
-   MakeSharedTable(ngroups, 0, pmesh.svert_lvert, pmesh.group_svert);
-   MakeSharedTable(ngroups, 1, pmesh.sedge_ledge, pmesh.group_sedge);
-   MakeSharedTable(ngroups, 2, pmesh.sface_lface, pmesh.group_squad);
+   int ng = pmesh.gtopo.NGroups();
+   MakeSharedTable(ng, 0, pmesh.svert_lvert, pmesh.group_svert);
+   MakeSharedTable(ng, 1, pmesh.sedge_ledge, pmesh.group_sedge);
 
-   // create an empty group_stria (we currently don't have triangle faces)
-   pmesh.group_stria.MakeI(ngroups-1);
-   pmesh.group_stria.MakeJ();
-   pmesh.group_stria.ShiftUpI();
+   Array<int> slt, slq;
+   MakeSharedTable(ng, 2, slt, pmesh.group_stria, &face_geom, Geometry::TRIANGLE);
+   MakeSharedTable(ng, 2, slq, pmesh.group_squad, &face_geom, Geometry::SQUARE);
+
+   pmesh.sface_lface = slt;
+   pmesh.sface_lface.Append(slq);
 
    // create shared_edges
    for (int i = 0; i < pmesh.shared_edges.Size(); i++)
@@ -939,11 +980,23 @@ void ParNCMesh::GetConformingSharedStructures(ParMesh &pmesh)
       pmesh.shared_edges[i] = new Segment(v, 1);
    }
 
-   // create shared_faces
-   pmesh.shared_quads.SetSize(pmesh.sface_lface.Size());
-   for (int i = 0; i < pmesh.shared_quads.Size(); i++)
+   // create shared_trias
+   pmesh.shared_trias.SetSize(slt.Size());
+   for (int i = 0; i < slt.Size(); i++)
    {
-      int el_loc = entity_elem_local[2][pmesh.sface_lface[i]];
+      int el_loc = entity_elem_local[2][slt[i]];
+      MeshId face_id(-1, leaf_elements[(el_loc >> 4)], (el_loc & 0xf));
+
+      int v[4], e[4], eo[4];
+      GetFaceVerticesEdges(face_id, v, e, eo);
+      pmesh.shared_trias[i].Set(v);
+   }
+
+   // create shared_quads
+   pmesh.shared_quads.SetSize(slq.Size());
+   for (int i = 0; i < slq.Size(); i++)
+   {
+      int el_loc = entity_elem_local[2][slq[i]];
       MeshId face_id(-1, leaf_elements[(el_loc >> 4)], (el_loc & 0xf));
 
       int e[4], eo[4];
@@ -959,12 +1012,6 @@ void ParNCMesh::GetConformingSharedStructures(ParMesh &pmesh)
    leaf_glob_order.DeleteAll();
 }
 
-bool ParNCMesh::compare_ranks_indices(const Element* a, const Element* b)
-{
-   return (a->rank != b->rank) ? a->rank < b->rank
-          /*                */ : a->index < b->index;
-}
-
 void ParNCMesh::GetFaceNeighbors(ParMesh &pmesh)
 {
    ClearAuxPM();
@@ -1003,6 +1050,7 @@ void ParNCMesh::GetFaceNeighbors(ParMesh &pmesh)
       for (int j = mf.slaves_begin; j < mf.slaves_end; j++)
       {
          const Slave &sf = full_list.slaves[j];
+         if (sf.index < 0) { continue; }
 
          MFEM_ASSERT(mf.element >= 0 && sf.element >= 0, "");
          Element* e[2] = { &elements[mf.element], &elements[sf.element] };
@@ -1024,7 +1072,11 @@ void ParNCMesh::GetFaceNeighbors(ParMesh &pmesh)
    // same on different processors, this is important for ExchangeFaceNbrData)
    fnbr.Sort();
    fnbr.Unique();
-   fnbr.Sort(compare_ranks_indices);
+   fnbr.Sort([](const Element* a, const Element* b)
+   {
+      return (a->rank != b->rank) ? a->rank < b->rank
+             /*                */ : a->index < b->index;
+   });
 
    // put the ranks into 'face_nbr_group'
    for (int i = 0; i < fnbr.Size(); i++)
@@ -1156,6 +1208,7 @@ void ParNCMesh::GetFaceNeighbors(ParMesh &pmesh)
          for (int j = mf.slaves_begin; j < mf.slaves_end; j++)
          {
             const Slave &sf = full_list.slaves[j];
+            if (sf.index < 0) { continue; }
 
             MFEM_ASSERT(sf.element >= 0 && mf.element >= 0, "");
             Element &sfe = elements[sf.element];
@@ -1303,13 +1356,18 @@ void ParNCMesh::Prune()
    {
       if (PruneTree(i)) { DerefineElement(i); }
    }
-
    Update();
 }
 
 
 void ParNCMesh::Refine(const Array<Refinement> &refinements)
 {
+   if (NRanks == 1)
+   {
+      NCMesh::Refine(refinements);
+      return;
+   }
+
    for (int i = 0; i < refinements.Size(); i++)
    {
       const Refinement &ref = refinements[i];
@@ -1708,7 +1766,7 @@ void ParNCMesh::CheckDerefinementNCLevel(const Table &deref_table,
 
 //// Rebalance /////////////////////////////////////////////////////////////////
 
-void ParNCMesh::Rebalance()
+void ParNCMesh::Rebalance(const Array<int> *custom_partition)
 {
    send_rebalance_dofs.clear();
    recv_rebalance_dofs.clear();
@@ -1716,30 +1774,46 @@ void ParNCMesh::Rebalance()
    Array<int> old_elements;
    leaf_elements.GetSubArray(0, NElements, old_elements);
 
-   // figure out new assignments for Element::rank
-   long local_elems = NElements, total_elems = 0;
-   MPI_Allreduce(&local_elems, &total_elems, 1, MPI_LONG, MPI_SUM, MyComm);
-
-   long first_elem_global = 0;
-   MPI_Scan(&local_elems, &first_elem_global, 1, MPI_LONG, MPI_SUM, MyComm);
-   first_elem_global -= local_elems;
-
-   Array<int> new_ranks(leaf_elements.Size());
-   new_ranks = -1;
-
-   for (int i = 0, j = 0; i < leaf_elements.Size(); i++)
+   if (!custom_partition) // SFC based partitioning
    {
-      if (elements[leaf_elements[i]].rank == MyRank)
+      Array<int> new_ranks(leaf_elements.Size());
+      new_ranks = -1;
+
+      // figure out new assignments for Element::rank
+      long local_elems = NElements, total_elems = 0;
+      MPI_Allreduce(&local_elems, &total_elems, 1, MPI_LONG, MPI_SUM, MyComm);
+
+      long first_elem_global = 0;
+      MPI_Scan(&local_elems, &first_elem_global, 1, MPI_LONG, MPI_SUM, MyComm);
+      first_elem_global -= local_elems;
+
+      for (int i = 0, j = 0; i < leaf_elements.Size(); i++)
       {
-         new_ranks[i] = Partition(first_elem_global + (j++), total_elems);
+         if (elements[leaf_elements[i]].rank == MyRank)
+         {
+            new_ranks[i] = Partition(first_elem_global + (j++), total_elems);
+         }
       }
+
+      int target_elements = PartitionFirstIndex(MyRank+1, total_elems)
+                            - PartitionFirstIndex(MyRank, total_elems);
+
+      // assign the new ranks and send elements (plus ghosts) to new owners
+      RedistributeElements(new_ranks, target_elements, true);
    }
+   else // whatever partitioning the user has passed
+   {
+      MFEM_VERIFY(custom_partition->Size() == NElements,
+                  "Size of the partition array must match the number "
+                  "of local mesh elements (ParMesh::GetNE()).");
 
-   int target_elements = PartitionFirstIndex(MyRank+1, total_elems)
-                         - PartitionFirstIndex(MyRank, total_elems);
+      Array<int> new_ranks;
+      custom_partition->Copy(new_ranks);
 
-   // assign the new ranks and send elements (plus ghosts) to new owners
-   RedistributeElements(new_ranks, target_elements, true);
+      new_ranks.SetSize(leaf_elements.Size(), -1); // make room for ghosts
+
+      RedistributeElements(new_ranks, -1, true);
+   }
 
    // set up the old index array
    old_index_or_rank.SetSize(NElements);
@@ -1754,28 +1828,22 @@ void ParNCMesh::Rebalance()
    Prune();
 }
 
-struct CompareRanks // TODO: use lambda when C++11 available
-{
-   typedef BlockArray<NCMesh::Element> ElemArray;
-   const ElemArray &elements;
-   CompareRanks(const ElemArray &elements) : elements(elements) {}
-
-   inline bool operator()(const int a, const int b)
-   {
-      return elements[a].rank < elements[b].rank;
-   }
-};
-
 void ParNCMesh::RedistributeElements(Array<int> &new_ranks, int target_elements,
                                      bool record_comm)
 {
+   bool sfc = (target_elements >= 0);
+
    UpdateLayers();
 
    // *** STEP 1: communicate new rank assignments for the ghost layer ***
 
    NeighborElementRankMessage::Map send_ghost_ranks, recv_ghost_ranks;
 
-   ghost_layer.Sort(CompareRanks(elements));
+   ghost_layer.Sort([&](const int a, const int b)
+   {
+      return elements[a].rank < elements[b].rank;
+   });
+
    {
       Array<int> rank_neighbors;
 
@@ -1853,12 +1921,17 @@ void ParNCMesh::RedistributeElements(Array<int> &new_ranks, int target_elements,
       el.rank = new_ranks[i];
    }
 
+   int nsent = 0, nrecv = 0; // for debug check
+
    RebalanceMessage::Map send_elems;
    {
       // sort elements we own by the new rank
       Array<int> owned_elements;
       owned_elements.MakeRef(leaf_elements.GetData(), NElements);
-      owned_elements.Sort(CompareRanks(elements));
+      owned_elements.Sort([&](const int a, const int b)
+      {
+         return elements[a].rank < elements[b].rank;
+      });
 
       Array<int> batch;
       batch.Reserve(1024);
@@ -1900,7 +1973,16 @@ void ParNCMesh::RedistributeElements(Array<int> &new_ranks, int target_elements,
                // disrupting the termination mechanism in Step 4.
             }
 
-            msg.Isend(rank, MyComm);
+            if (sfc)
+            {
+               msg.Isend(rank, MyComm);
+            }
+            else
+            {
+               // custom partitioning needs synchronous sends
+               msg.Issend(rank, MyComm);
+            }
+            nsent++;
 
             // also: record what elements we sent (excluding the ghosts)
             // so that SendRebalanceDofs can later send data for them
@@ -1916,41 +1998,108 @@ void ParNCMesh::RedistributeElements(Array<int> &new_ranks, int target_elements,
 
    // *** STEP 3: receive elements from others ***
 
-   /* We don't know from whom we're going to receive so we need to probe.
-      Fortunately, we do know how many elements we're going to own eventually
-      so the termination condition is easy. */
-
    RebalanceMessage msg;
    msg.SetNCMesh(this);
 
-   while (received_elements < target_elements)
+   if (sfc)
    {
-      int rank, size;
-      RebalanceMessage::Probe(rank, size, MyComm);
+      /* We don't know from whom we're going to receive, so we need to probe.
+         However, for the default SFC partitioning, we do know how many elements
+         we're going to own eventually, so the termination condition is easy. */
 
-      // receive message; note: elements are created as the message is decoded
-      msg.Recv(rank, size, MyComm);
-
-      for (int i = 0; i < msg.Size(); i++)
+      while (received_elements < target_elements)
       {
-         int elem_rank = msg.values[i];
-         elements[msg.elements[i]].rank = elem_rank;
+         int rank, size;
+         RebalanceMessage::Probe(rank, size, MyComm);
 
-         if (elem_rank == MyRank) { received_elements++; }
+         // receive message; note: elements are created as the message is decoded
+         msg.Recv(rank, size, MyComm);
+         nrecv++;
+
+         for (int i = 0; i < msg.Size(); i++)
+         {
+            int elem_rank = msg.values[i];
+            elements[msg.elements[i]].rank = elem_rank;
+
+            if (elem_rank == MyRank) { received_elements++; }
+         }
+
+         // save the ranks we received from, for later use in RecvRebalanceDofs
+         if (record_comm)
+         {
+            recv_rebalance_dofs[rank].SetNCMesh(this);
+         }
       }
 
-      // save the ranks we received from, for later use in RecvRebalanceDofs
-      if (record_comm)
+      Update();
+
+      RebalanceMessage::WaitAllSent(send_elems);
+   }
+   else
+   {
+      /* The case (target_elements < 0) is used for custom partitioning.
+         Here we need to employ the "non-blocking consensus" algorithm
+         (https://scorec.rpi.edu/REPORTS/2015-9.pdf) to determine when the
+         element exchange is finished. The algorithm uses a non-blocking
+         barrier. */
+
+      MPI_Request barrier = MPI_REQUEST_NULL;
+      int done = 0;
+
+      while (!done)
       {
-         recv_rebalance_dofs[rank].SetNCMesh(this);
+         int rank, size;
+         while (RebalanceMessage::IProbe(rank, size, MyComm))
+         {
+            // receive message; note: elements are created as the msg is decoded
+            msg.Recv(rank, size, MyComm);
+            nrecv++;
+
+            for (int i = 0; i < msg.Size(); i++)
+            {
+               elements[msg.elements[i]].rank = msg.values[i];
+            }
+
+            // save the ranks we received from, for later use in RecvRebalanceDofs
+            if (record_comm)
+            {
+               recv_rebalance_dofs[rank].SetNCMesh(this);
+            }
+         }
+
+         if (barrier != MPI_REQUEST_NULL)
+         {
+            MPI_Test(&barrier, &done, MPI_STATUS_IGNORE);
+         }
+         else
+         {
+            if (RebalanceMessage::TestAllSent(send_elems))
+            {
+               int err = MPI_Ibarrier(MyComm, &barrier);
+
+               MFEM_VERIFY(err == MPI_SUCCESS, "");
+               MFEM_VERIFY(barrier != MPI_REQUEST_NULL, "");
+            }
+         }
       }
+
+      Update();
    }
 
-   Update();
-
-   // make sure we can delete all send buffers
    NeighborElementRankMessage::WaitAllSent(send_ghost_ranks);
-   NeighborElementRankMessage::WaitAllSent(send_elems);
+
+#ifdef MFEM_DEBUG
+   int glob_sent, glob_recv;
+   MPI_Reduce(&nsent, &glob_sent, 1, MPI_INT, MPI_SUM, 0, MyComm);
+   MPI_Reduce(&nrecv, &glob_recv, 1, MPI_INT, MPI_SUM, 0, MyComm);
+
+   if (MyRank == 0)
+   {
+      MFEM_ASSERT(glob_sent == glob_recv,
+                  "(glob_sent, glob_recv) = ("
+                  << glob_sent << ", " << glob_recv << ")");
+   }
+#endif
 }
 
 
@@ -2242,8 +2391,8 @@ void ParNCMesh::AdjustMeshIds(Array<MeshId> ids[], int rank)
       if (contains_rank[entity_pmat_group[2][face_id.index]])
       {
          int v[4], e[4], eo[4], pos, k;
-         GetFaceVerticesEdges(face_id, v, e, eo);
-         for (int j = 0; j < 4; j++)
+         int nfv = GetFaceVerticesEdges(face_id, v, e, eo);
+         for (int j = 0; j < nfv; j++)
          {
             if ((pos = find_v.FindSorted(Pair<int, int>(v[j], 0))) != -1)
             {
@@ -2267,7 +2416,7 @@ void ParNCMesh::ChangeVertexMeshIdElement(NCMesh::MeshId &id, int elem)
    Element &el = elements[elem];
    MFEM_ASSERT(el.ref_type == 0, "");
 
-   GeomInfo& gi = GI[(int) el.geom];
+   GeomInfo& gi = GI[el.Geom()];
    for (int i = 0; i < gi.nv; i++)
    {
       if (nodes[el.node[i]].vert_index == id.index)
@@ -2283,14 +2432,14 @@ void ParNCMesh::ChangeVertexMeshIdElement(NCMesh::MeshId &id, int elem)
 void ParNCMesh::ChangeEdgeMeshIdElement(NCMesh::MeshId &id, int elem)
 {
    Element &old = elements[id.element];
-   const int *ev = GI[(int) old.geom].edges[id.local];
+   const int *ev = GI[old.Geom()].edges[(int) id.local];
    Node* node = nodes.Find(old.node[ev[0]], old.node[ev[1]]);
    MFEM_ASSERT(node != NULL, "Edge not found.");
 
    Element &el = elements[elem];
    MFEM_ASSERT(el.ref_type == 0, "");
 
-   GeomInfo& gi = GI[(int) el.geom];
+   GeomInfo& gi = GI[el.Geom()];
    for (int i = 0; i < gi.ne; i++)
    {
       const int* ev = gi.edges[i];
@@ -2389,17 +2538,17 @@ void ParNCMesh::DecodeMeshIds(std::istream &is, Array<MeshId> ids[])
          id.local = read<char>(is);
 
          // find vertex/edge/face index
-         GeomInfo &gi = GI[(int) el.geom];
+         GeomInfo &gi = GI[el.Geom()];
          switch (type)
          {
             case 0:
             {
-               id.index = nodes[el.node[id.local]].vert_index;
+               id.index = nodes[el.node[(int) id.local]].vert_index;
                break;
             }
             case 1:
             {
-               const int* ev = gi.edges[id.local];
+               const int* ev = gi.edges[(int) id.local];
                Node* node = nodes.Find(el.node[ev[0]], el.node[ev[1]]);
                MFEM_ASSERT(node && node->HasEdge(), "edge not found.");
                id.index = node->edge_index;
@@ -2407,7 +2556,7 @@ void ParNCMesh::DecodeMeshIds(std::istream &is, Array<MeshId> ids[])
             }
             default:
             {
-               const int* fv = gi.faces[id.local];
+               const int* fv = gi.faces[(int) id.local];
                Face* face = faces.Find(el.node[fv[0]], el.node[fv[1]],
                                        el.node[fv[2]], el.node[fv[3]]);
                MFEM_ASSERT(face, "face not found.");

mesh/pncmesh.hpp

@@ -87,8 +87,12 @@ public:
    virtual void Derefine(const Array<int> &derefs);
 
    /** Migrate leaf elements of the global refinement hierarchy (including ghost
-       elements) so that each processor owns the same number of leaves (+-1). */
-   void Rebalance();
+       elements) so that each processor owns the same number of leaves (+-1).
+       The default partitioning strategy is based on equal splitting of the
+       space-filling sequence of leaf elements (custom_partition == NULL).
+       Alternatively, a used-defined element-rank assignemnt array can be
+       passed. */
+   void Rebalance(const Array<int> *custom_partition = NULL);
 
 
    // interface for ParFiniteElementSpace
@@ -100,9 +104,6 @@ public:
    int GetNGhostFaces() const { return NGhostFaces; }
    int GetNGhostElements() const { return NGhostElements; }
 
-   Geometry::Type GetGhostFaceGeometry(int ghost_face_id) const
-   { return Geometry::SQUARE; }
-
    // Return a list of vertices/edges/faces shared by this processor and at
    // least one other processor. These are subsets of NCMesh::<entity>_list. */
    const NCList& GetSharedVertices() { GetVertexList(); return shared_vertices; }
@@ -168,6 +169,12 @@ public:
    /// Return true if the specified vertex/edge/face is a ghost.
    bool IsGhost(int entity, int index) const
    {
+      if (index < 0) // special case prism edge-face constraint
+      {
+         MFEM_ASSERT(entity == 2, "");
+         entity = 1;
+         index = -1 - index;
+      }
       switch (entity)
       {
          case 0: return index >= NVertices;
@@ -256,7 +263,7 @@ protected: // implementation
    GroupList groups;  // comm group list; NOTE: groups[0] = { MyRank }
    GroupMap group_id; // search index over groups
 
-   // owner rank for each vertex, edge and face (encoded as singleton groups)
+   // owner rank for each vertex, edge and face (encoded as singleton group)
    Array<GroupId> entity_owner[3];
    // P matrix comm pattern groups for each vertex/edge/face (0/1/2)
    Array<GroupId> entity_pmat_group[3];
@@ -336,7 +343,8 @@ protected: // implementation
    void UpdateLayers();
 
    void MakeSharedTable(int ngroups, int ent, Array<int> &shared_local,
-                        Table &group_shared);
+                        Table &group_shared, Array<char> *entity_geom = NULL,
+                        char geom = 0);
 
    /** Uniquely encodes a set of leaf elements in the refinement hierarchy of
        an NCMesh. Can be dumped to a stream, sent to another processor, loaded,
@@ -506,7 +514,10 @@ protected: // implementation
 
    /** Assign new Element::rank to leaf elements and send them to their new
        owners, keeping the ghost layer up to date. Used by Rebalance() and
-       Derefine(). */
+       Derefine(). 'target_elements' is the number of elements this rank
+       is supposed to own after the exchange. If this number is not known
+       apriori, the parameter can be set to -1, but more expensive communication
+       (synchronous sends and a barrier) will be used in that case. */
    void RedistributeElements(Array<int> &new_ranks, int target_elements,
                              bool record_comm);
 
@@ -526,8 +537,6 @@ protected: // implementation
 
    long GroupsMemoryUsage() const;
 
-   static bool compare_ranks_indices(const Element* a, const Element* b);
-
    friend class NeighborRowMessage;
 };
 

mesh/point.hpp

@@ -46,9 +46,14 @@ public:
 
    virtual const int *GetEdgeVertices(int ei) const { return NULL; }
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const
    { nFaceVertices = 0; return 0; }
 
+   virtual int GetNFaces() const { return 0; }
+
+   virtual int GetNFaceVertices(int) const { return 0; }
+
    virtual const int *GetFaceVertices(int fi) const { return NULL; }
 
    virtual Element *Duplicate(Mesh *m) const

mesh/quadrilateral.hpp

@@ -53,9 +53,14 @@ public:
    virtual const int *GetEdgeVertices(int ei) const
    { return geom_t::Edges[ei]; }
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const
    { nFaceVertices = 0; return 0; }
 
+   virtual int GetNFaces() const { return 0; }
+
+   virtual int GetNFaceVertices(int) const { return 0; }
+
    virtual const int *GetFaceVertices(int fi) const { return NULL; }
 
    virtual Element *Duplicate(Mesh *m) const

mesh/segment.hpp

@@ -52,9 +52,14 @@ public:
 
    virtual const int *GetEdgeVertices(int ei) const { return NULL; }
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const
    { nFaceVertices = 0; return 0; }
 
+   virtual int GetNFaces() const { return 0; }
+
+   virtual int GetNFaceVertices(int) const { return 0; }
+
    virtual const int *GetFaceVertices(int fi) const { return NULL; }
 
    virtual Element *Duplicate(Mesh *m) const

mesh/tetrahedron.cpp

@@ -40,13 +40,16 @@ Tetrahedron::Tetrahedron(int ind1, int ind2, int ind3, int ind4, int attr)
    transform = 0;
 }
 
-void Tetrahedron::Init(int ind1, int ind2, int ind3, int ind4, int attr)
+void Tetrahedron::Init(int ind1, int ind2, int ind3, int ind4, int attr,
+                       int ref_flag)
 {
    attribute  = attr;
    indices[0] = ind1;
    indices[1] = ind2;
    indices[2] = ind3;
    indices[3] = ind4;
+   refinement_flag = ref_flag;
+   transform = 0;
 }
 
 void Tetrahedron::ParseRefinementFlag(int refinement_edges[2], int &type,

mesh/tetrahedron.hpp

@@ -52,7 +52,8 @@ public:
    Tetrahedron(int ind1, int ind2, int ind3, int ind4, int attr = 1);
 
    /// Initialize the vertex indices and the attribute of a Tetrahedron.
-   void Init(int ind1, int ind2, int ind3, int ind4, int attr = 1);
+   void Init(int ind1, int ind2, int ind3, int ind4, int attr = 1,
+             int ref_flag = 0);
 
    /// Return element's type.
    virtual Type GetType() const { return Element::TETRAHEDRON; }
@@ -99,9 +100,14 @@ public:
    virtual const int *GetEdgeVertices(int ei) const
    { return geom_t::Edges[ei]; }
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const
    { nFaceVertices = 3; return 4; }
 
+   virtual int GetNFaces() const { return 4; }
+
+   virtual int GetNFaceVertices(int) const { return 3; }
+
    virtual const int *GetFaceVertices(int fi) const
    { MFEM_ABORT("not implemented"); return NULL; }
 

mesh/triangle.hpp

@@ -80,9 +80,14 @@ public:
    virtual const int *GetEdgeVertices(int ei) const
    { return geom_t::Edges[ei]; }
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const
    { nFaceVertices = 0; return 0; }
 
+   virtual int GetNFaces() const { return 0; }
+
+   virtual int GetNFaceVertices(int) const { return 0; }
+
    virtual const int *GetFaceVertices(int fi) const
    { MFEM_ABORT("not implemented"); return NULL; }
 

mesh/wedge.hpp

@@ -54,10 +54,13 @@ public:
    virtual const int *GetEdgeVertices(int ei) const
    { return geom_t::Edges[ei]; }
 
+   /// @deprecated Use GetNFaces(void) and GetNFaceVertices(int) instead.
    virtual int GetNFaces(int &nFaceVertices) const;
 
-   virtual int GetNFaceVerticess(int fi) const
-   { return ( ( fi < 2 ) ? 3 : 4); }
+   virtual int GetNFaces() const { return 5; }
+
+   virtual int GetNFaceVertices(int fi) const
+   { return (fi < 2) ? 3 : 4; }
 
    virtual const int *GetFaceVertices(int fi) const
    { return geom_t::FaceVert[fi]; }
@@ -69,7 +72,7 @@ public:
 };
 
 // Defined in fe.cpp to ensure construction after 'mfem::poly1d'.
-extern class H1_WedgeElement WedgeFE;
+extern H1_WedgeElement WedgeFE;
 
 }
 

miniapps/electromagnetics/volta_solver.cpp

@@ -41,6 +41,7 @@ VoltaSolver::VoltaSolver(ParMesh & pmesh, int order,
      H1FESpace_(NULL),
      HCurlFESpace_(NULL),
      HDivFESpace_(NULL),
+     L2FESpace_(NULL),
      divEpsGrad_(NULL),
      h1Mass_(NULL),
      h1SurfMass_(NULL),
@@ -49,20 +50,24 @@ VoltaSolver::VoltaSolver(ParMesh & pmesh, int order,
      hCurlHDiv_(NULL),
      weakDiv_(NULL),
      rhod_(NULL),
+     l2_vol_int_(NULL),
+     rt_surf_int_(NULL),
      grad_(NULL),
      phi_(NULL),
+     rho_src_(NULL),
      rho_(NULL),
-     sigma_(NULL),
+     sigma_src_(NULL),
      e_(NULL),
      d_(NULL),
-     p_(NULL),
+     p_src_(NULL),
+     oneCoef_(1.0),
      epsCoef_(&epsCoef),
      phiBCCoef_(NULL),
      rhoCoef_(NULL),
      pCoef_(NULL),
-     phi_bc_(phi_bc),
-     rho_src_(rho_src),
-     p_src_(p_src),
+     phi_bc_func_(phi_bc),
+     rho_src_func_(rho_src),
+     p_src_func_(p_src),
      point_charge_params_(point_charges),
      point_charges_(0)
 {
@@ -76,34 +81,38 @@ VoltaSolver::VoltaSolver(ParMesh & pmesh, int order,
    H1FESpace_    = new H1_ParFESpace(pmesh_,order,pmesh_->Dimension());
    HCurlFESpace_ = new ND_ParFESpace(pmesh_,order,pmesh_->Dimension());
    HDivFESpace_  = new RT_ParFESpace(pmesh_,order,pmesh_->Dimension());
+   L2FESpace_    = new L2_ParFESpace(pmesh_,order-1,pmesh_->Dimension());
 
    // Select surface attributes for Dirichlet BCs
    ess_bdr_.SetSize(pmesh.bdr_attributes.Max());
    ess_bdr_ = 0;   // Deselect all outer surfaces
    for (int i=0; i<dbcs_->Size(); i++)
    {
-      ess_bdr_[(*dbcs_)[i]-1] = 1;
+      if ((*dbcs_)[i] <= ess_bdr_.Size())
+      {
+         ess_bdr_[(*dbcs_)[i]-1] = 1;
+      }
    }
 
    // Setup various coefficients
 
    // Potential on outer surface
-   if ( phi_bc_ != NULL )
+   if ( phi_bc_func_ != NULL )
    {
-      phiBCCoef_ = new FunctionCoefficient(*phi_bc_);
+      phiBCCoef_ = new FunctionCoefficient(*phi_bc_func_);
    }
 
    // Volume Charge Density
-   if ( rho_src_ != NULL )
+   if ( rho_src_func_ != NULL )
    {
-      rhoCoef_ = new FunctionCoefficient(rho_src_);
+      rhoCoef_ = new FunctionCoefficient(rho_src_func_);
    }
 
    // Polarization
-   if ( p_src_ != NULL )
+   if ( p_src_func_ != NULL )
    {
       pCoef_ = new VectorFunctionCoefficient(pmesh_->SpaceDimension(),
-                                             p_src_);
+                                             p_src_func_);
    }
 
    // Bilinear Forms
@@ -116,15 +125,23 @@ VoltaSolver::VoltaSolver(ParMesh & pmesh, int order,
    hCurlHDivEps_ = new ParMixedBilinearForm(HCurlFESpace_,HDivFESpace_);
    hCurlHDivEps_->AddDomainIntegrator(new VectorFEMassIntegrator(*epsCoef_));
 
-   rhod_ = new ParLinearForm(H1FESpace_);
+   rhod_   = new ParLinearForm(H1FESpace_);
 
-   // Discrete Grad operator
+   l2_vol_int_ = new ParLinearForm(L2FESpace_);
+   l2_vol_int_->AddDomainIntegrator(new DomainLFIntegrator(oneCoef_));
+
+   rt_surf_int_ = new ParLinearForm(HDivFESpace_);
+   rt_surf_int_->AddBoundaryIntegrator(new VectorFEBoundaryFluxLFIntegrator);
+
+   // Discrete derivative operator
    grad_ = new ParDiscreteGradOperator(H1FESpace_, HCurlFESpace_);
+   div_  = new ParDiscreteDivOperator(HDivFESpace_, L2FESpace_);
 
    // Build grid functions
    phi_  = new ParGridFunction(H1FESpace_);
    d_    = new ParGridFunction(HDivFESpace_);
    e_    = new ParGridFunction(HCurlFESpace_);
+   rho_  = new ParGridFunction(L2FESpace_);
 
    if ( point_charge_params_.Size() > 0 )
    {
@@ -149,29 +166,28 @@ VoltaSolver::VoltaSolver(ParMesh & pmesh, int order,
       }
    }
 
-   if ( rho_src_ )
+   if ( rho_src_func_ )
    {
-      rho_ = new ParGridFunction(H1FESpace_);
+      rho_src_ = new ParGridFunction(H1FESpace_);
 
       h1Mass_ = new ParBilinearForm(H1FESpace_);
       h1Mass_->AddDomainIntegrator(new MassIntegrator);
    }
 
-   if ( p_src_ )
+   if ( p_src_func_ )
    {
-      p_ = new ParGridFunction(HCurlFESpace_);
+      p_src_ = new ParGridFunction(HCurlFESpace_);
 
       hCurlHDiv_ = new ParMixedBilinearForm(HCurlFESpace_, HDivFESpace_);
       hCurlHDiv_->AddDomainIntegrator(new VectorFEMassIntegrator);
 
       weakDiv_ = new ParMixedBilinearForm(HCurlFESpace_, H1FESpace_);
       weakDiv_->AddDomainIntegrator(new VectorFEWeakDivergenceIntegrator);
-
    }
 
    if ( nbcs_->Size() > 0 )
    {
-      sigma_ = new ParGridFunction(H1FESpace_);
+      sigma_src_ = new ParGridFunction(H1FESpace_);
 
       h1SurfMass_ = new ParBilinearForm(H1FESpace_);
       h1SurfMass_->AddBoundaryIntegrator(new MassIntegrator);
@@ -185,14 +201,18 @@ VoltaSolver::~VoltaSolver()
    delete pCoef_;
 
    delete phi_;
+   delete rho_src_;
    delete rho_;
    delete rhod_;
-   delete sigma_;
+   delete l2_vol_int_;
+   delete rt_surf_int_;
+   delete sigma_src_;
    delete d_;
    delete e_;
-   delete p_;
+   delete p_src_;
 
    delete grad_;
+   delete div_;
 
    delete divEpsGrad_;
    delete h1Mass_;
@@ -205,6 +225,7 @@ VoltaSolver::~VoltaSolver()
    delete H1FESpace_;
    delete HCurlFESpace_;
    delete HDivFESpace_;
+   delete L2FESpace_;
 
    for (unsigned int i=0; i<point_charges_.size(); i++)
    {
@@ -230,11 +251,13 @@ VoltaSolver::PrintSizes()
    HYPRE_Int size_h1 = H1FESpace_->GlobalTrueVSize();
    HYPRE_Int size_nd = HCurlFESpace_->GlobalTrueVSize();
    HYPRE_Int size_rt = HDivFESpace_->GlobalTrueVSize();
+   HYPRE_Int size_l2 = L2FESpace_->GlobalTrueVSize();
    if (myid_ == 0)
    {
       cout << "Number of H1      unknowns: " << size_h1 << endl;
       cout << "Number of H(Curl) unknowns: " << size_nd << endl;
       cout << "Number of H(Div)  unknowns: " << size_rt << endl;
+      cout << "Number of L2      unknowns: " << size_l2 << endl;
    }
 }
 
@@ -254,9 +277,15 @@ void VoltaSolver::Assemble()
    *rhod_ = 0.0;
    rhod_->Assemble();
 
+   l2_vol_int_->Assemble();
+   rt_surf_int_->Assemble();
+
    grad_->Assemble();
    grad_->Finalize();
 
+   div_->Assemble();
+   div_->Finalize();
+
    if ( h1Mass_ )
    {
       h1Mass_->Assemble();
@@ -292,15 +321,19 @@ VoltaSolver::Update()
    H1FESpace_->Update(false);
    HCurlFESpace_->Update(false);
    HDivFESpace_->Update(false);
+   L2FESpace_->Update(false);
 
    // Inform the grid functions that the space has changed.
    phi_->Update();
    rhod_->Update();
+   l2_vol_int_->Update();
+   rt_surf_int_->Update();
    d_->Update();
    e_->Update();
-   if ( rho_   ) { rho_->Update(); }
-   if ( sigma_ ) { sigma_->Update(); }
-   if ( p_     ) { p_->Update(); }
+   rho_->Update();
+   if ( rho_src_   ) { rho_src_->Update(); }
+   if ( sigma_src_ ) { sigma_src_->Update(); }
+   if ( p_src_     ) { p_src_->Update(); }
 
    // Inform the bilinear forms that the space has changed.
    divEpsGrad_->Update();
@@ -314,6 +347,7 @@ VoltaSolver::Update()
 
    // Inform the other objects that the space has changed.
    grad_->Update();
+   div_->Update();
 }
 
 void
@@ -339,40 +373,46 @@ VoltaSolver::Solve()
          {
             ConstantCoefficient voltage((*dbcv_)[i]);
             dbc_bdr_attr = 0;
-            dbc_bdr_attr[(*dbcs_)[i]-1] = 1;
+            if ((*dbcs_)[i] <= dbc_bdr_attr.Size())
+            {
+               dbc_bdr_attr[(*dbcs_)[i]-1] = 1;
+            }
             phi_->ProjectBdrCoefficient(voltage, dbc_bdr_attr);
          }
       }
    }
 
    // Initialize the volumetric charge density
-   if ( rho_ )
+   if ( rho_src_ )
    {
-      rho_->ProjectCoefficient(*rhoCoef_);
-      h1Mass_->AddMult(*rho_, *rhod_);
+      rho_src_->ProjectCoefficient(*rhoCoef_);
+      h1Mass_->AddMult(*rho_src_, *rhod_);
    }
 
    // Initialize the Polarization
-   if ( p_ )
+   if ( p_src_ )
    {
-      p_->ProjectCoefficient(*pCoef_);
-      weakDiv_->AddMult(*p_, *rhod_);
+      p_src_->ProjectCoefficient(*pCoef_);
+      weakDiv_->AddMult(*p_src_, *rhod_);
    }
 
    // Initialize the surface charge density
-   if ( sigma_ )
+   if ( sigma_src_ )
    {
-      *sigma_ = 0.0;
+      *sigma_src_ = 0.0;
 
       Array<int> nbc_bdr_attr(pmesh_->bdr_attributes.Max());
       for (int i=0; i<nbcs_->Size(); i++)
       {
          ConstantCoefficient sigma_coef((*nbcv_)[i]);
          nbc_bdr_attr = 0;
-         nbc_bdr_attr[(*nbcs_)[i]-1] = 1;
-         sigma_->ProjectBdrCoefficient(sigma_coef, nbc_bdr_attr);
+         if ((*nbcs_)[i] <= nbc_bdr_attr.Size())
+         {
+            nbc_bdr_attr[(*nbcs_)[i]-1] = 1;
+         }
+         sigma_src_->ProjectBdrCoefficient(sigma_coef, nbc_bdr_attr);
       }
-      h1SurfMass_->AddMult(*sigma_, *rhod_);
+      h1SurfMass_->AddMult(*sigma_src_, *rhod_);
    }
 
    // Determine the essential BC degrees of freedom
@@ -424,9 +464,9 @@ VoltaSolver::Solve()
 
    ParGridFunction ed(HDivFESpace_);
    hCurlHDivEps_->Mult(*e_, ed);
-   if ( p_ )
+   if ( p_src_ )
    {
-      hCurlHDiv_->AddMult(*p_, ed, -1.0);
+      hCurlHDiv_->AddMult(*p_src_, ed, -1.0);
    }
 
    HypreParMatrix MassHDiv;
@@ -445,8 +485,27 @@ VoltaSolver::Solve()
 
    hDivMass_->RecoverFEMSolution(D, ed, *d_);
 
+   // Compute charge density from rho = Div(D)
+   div_->Mult(*d_, *rho_);
+
    if (myid_ == 0) { cout << "done." << flush; }
 
+   {
+      // Compute total charge as volume integral of rho
+      double charge_rho = (*l2_vol_int_)(*rho_);
+
+      // Compute total charge as surface integral of D
+      double charge_D = (*rt_surf_int_)(*d_);
+
+      if (myid_ == 0)
+      {
+         cout << endl << "Total charge: \n"
+              << "   Volume integral of charge density:   " << charge_rho
+              << "\n   Surface integral of dielectric flux: " << charge_D
+              << endl << flush;
+      }
+   }
+
    if (myid_ == 0) { cout << "Solver done. " << endl; }
 }
 
@@ -480,9 +539,10 @@ VoltaSolver::RegisterVisItFields(VisItDataCollection & visit_dc)
    visit_dc.RegisterField("Phi", phi_);
    visit_dc.RegisterField("D",     d_);
    visit_dc.RegisterField("E",     e_);
-   if ( rho_   ) { visit_dc.RegisterField("Rho",     rho_); }
-   if ( p_     ) { visit_dc.RegisterField("P",         p_); }
-   if ( sigma_ ) { visit_dc.RegisterField("Sigma", sigma_); }
+   visit_dc.RegisterField("Rho", rho_);
+   if ( rho_src_   ) { visit_dc.RegisterField("Rho Source",     rho_src_); }
+   if ( p_src_     ) { visit_dc.RegisterField("P Source",         p_src_); }
+   if ( sigma_src_ ) { visit_dc.RegisterField("Sigma Source", sigma_src_); }
 }
 
 void
@@ -515,20 +575,23 @@ VoltaSolver::InitializeGLVis()
    socks_["E"] = new socketstream;
    socks_["E"]->precision(8);
 
-   if ( rho_)
+   socks_["Rho"] = new socketstream;
+   socks_["Rho"]->precision(8);
+
+   if ( rho_src_ )
    {
-      socks_["Rho"] = new socketstream;
-      socks_["Rho"]->precision(8);
+      socks_["RhoSrc"] = new socketstream;
+      socks_["RhoSrc"]->precision(8);
    }
-   if ( p_)
+   if ( p_src_ )
    {
-      socks_["P"] = new socketstream;
-      socks_["P"]->precision(8);
+      socks_["PSrc"] = new socketstream;
+      socks_["PSrc"]->precision(8);
    }
-   if ( sigma_)
+   if ( sigma_src_ )
    {
-      socks_["Sigma"] = new socketstream;
-      socks_["Sigma"]->precision(8);
+      socks_["SigmaSrc"] = new socketstream;
+      socks_["SigmaSrc"]->precision(8);
    }
 }
 
@@ -548,31 +611,36 @@ VoltaSolver::DisplayToGLVis()
                   *phi_, "Electric Potential (Phi)", Wx, Wy, Ww, Wh);
    Wx += offx;
 
+   VisualizeField(*socks_["E"], vishost, visport,
+                  *e_, "Electric Field (E)", Wx, Wy, Ww, Wh);
+   Wx += offx;
+
    VisualizeField(*socks_["D"], vishost, visport,
                   *d_, "Electric Displacement (D)", Wx, Wy, Ww, Wh);
    Wx += offx;
 
-   VisualizeField(*socks_["E"], vishost, visport,
-                  *e_, "Electric Field (E)", Wx, Wy, Ww, Wh);
-
+   VisualizeField(*socks_["Rho"], vishost, visport,
+                  *rho_, "Charge Density", Wx, Wy, Ww, Wh);
    Wx = 0; Wy += offy; // next line
 
-   if ( rho_ )
+   if ( rho_src_ )
    {
-      VisualizeField(*socks_["Rho"], vishost, visport,
-                     *rho_, "Charge Density (Rho)", Wx, Wy, Ww, Wh);
+      VisualizeField(*socks_["RhoSrc"], vishost, visport,
+                     *rho_src_, "Charge Density Source (Rho)", Wx, Wy, Ww, Wh);
       Wx += offx;
    }
-   if ( p_ )
+   if ( p_src_ )
    {
-      VisualizeField(*socks_["P"], vishost, visport,
-                     *p_, "Electric Polarization (P)", Wx, Wy, Ww, Wh);
+      VisualizeField(*socks_["PSrc"], vishost, visport,
+                     *p_src_, "Electric Polarization Source (P)",
+                     Wx, Wy, Ww, Wh);
       Wx += offx;
    }
-   if ( sigma_ )
+   if ( sigma_src_ )
    {
-      VisualizeField(*socks_["Sigma"], vishost, visport,
-                     *sigma_, "Surface Charge Density (Sigma)", Wx, Wy, Ww, Wh);
+      VisualizeField(*socks_["SigmaSrc"], vishost, visport,
+                     *sigma_src_, "Surface Charge Density Source (Sigma)",
+                     Wx, Wy, Ww, Wh);
       // Wx += offx; // not used
    }
    if (myid_ == 0) { cout << " done." << endl; }

miniapps/electromagnetics/volta_solver.hpp

@@ -26,7 +26,9 @@ namespace mfem
 using miniapps::H1_ParFESpace;
 using miniapps::ND_ParFESpace;
 using miniapps::RT_ParFESpace;
+using miniapps::L2_ParFESpace;
 using miniapps::ParDiscreteGradOperator;
+using miniapps::ParDiscreteDivOperator;
 
 namespace electromagnetics
 {
@@ -84,6 +86,7 @@ private:
    H1_ParFESpace * H1FESpace_;    // Continuous space for phi
    ND_ParFESpace * HCurlFESpace_; // Tangentially continuous space for E
    RT_ParFESpace * HDivFESpace_;  // Normally continuous space for D
+   L2_ParFESpace * L2FESpace_;    // Discontinuous space for rho
 
    ParBilinearForm * divEpsGrad_; // Laplacian operator
    ParBilinearForm * h1Mass_;     // For Volumetric Charge Density Source
@@ -94,26 +97,32 @@ private:
    ParMixedBilinearForm * hCurlHDiv_;    // For computing D from E and P
    ParMixedBilinearForm * weakDiv_;      // For computing the source term from P
 
-   ParLinearForm * rhod_; // Dual of Volumetric Charge Density
+   ParLinearForm * rhod_; // Dual of Volumetric Charge Density Source
+
+   ParLinearForm * l2_vol_int_;  // Integral of L2 field
+   ParLinearForm * rt_surf_int_; // Integral of H(Div) field over boundary
 
    ParDiscreteGradOperator * grad_; // For Computing E from phi
+   ParDiscreteDivOperator  * div_;  // For Computing rho from D
 
-   ParGridFunction * phi_; // Electric Scalar Potential
-   ParGridFunction * rho_; // Volumetric Charge Density
-   ParGridFunction * sigma_; // Surface Charge Density
-   ParGridFunction * e_; // Electric Field
-   ParGridFunction * d_; // Electric Flux Density (aka Dielectric Flux)
-   ParGridFunction * p_; // Polarization Field
+   ParGridFunction * phi_;       // Electric Scalar Potential
+   ParGridFunction * rho_src_;   // Volumetric Charge Density Source
+   ParGridFunction * rho_;       // Volumetric Charge Density (Div(D))
+   ParGridFunction * sigma_src_; // Surface Charge Density Source
+   ParGridFunction * e_;         // Electric Field
+   ParGridFunction * d_;         // Electric Flux Density (aka Dielectric Flux)
+   ParGridFunction * p_src_;     // Polarization Field Source
 
+   ConstantCoefficient oneCoef_;   // Coefficient equal to 1
    Coefficient       * epsCoef_;   // Dielectric Permittivity Coefficient
    Coefficient       * phiBCCoef_; // Scalar Potential Boundary Condition
    Coefficient       * rhoCoef_;   // Charge Density Coefficient
    VectorCoefficient * pCoef_;     // Polarization Vector Field Coefficient
 
    // Source functions
-   double (*phi_bc_ )(const Vector&);          // Scalar Potential BC
-   double (*rho_src_)(const Vector&);          // Volumetric Charge Density
-   void   (*p_src_  )(const Vector&, Vector&); // Polarization Field
+   double (*phi_bc_func_ )(const Vector&);          // Scalar Potential BC
+   double (*rho_src_func_)(const Vector&);          // Volumetric Charge Density
+   void   (*p_src_func_  )(const Vector&, Vector&); // Polarization Field
 
    const Vector & point_charge_params_;
 

miniapps/meshing/amr-quad-q2.mesh

@@ -0,0 +1,418 @@
+MFEM mesh v1.1
+
+#
+# MFEM Geometry Types (see mesh/geom.hpp):
+#
+# POINT       = 0
+# SEGMENT     = 1
+# TRIANGLE    = 2
+# SQUARE      = 3
+# TETRAHEDRON = 4
+# CUBE        = 5
+# PRISM       = 6
+#
+
+dimension
+2
+
+elements
+28
+1 3 0 9 21 12
+1 3 12 21 11 7
+1 3 21 22 37 25
+1 3 25 37 24 11
+1 3 37 23 8 24
+1 3 22 10 23 37
+1 3 9 4 10 21
+1 3 4 13 26 10
+1 3 13 1 14 26
+1 3 26 14 5 15
+1 3 38 28 15 29
+1 3 27 26 28 38
+1 3 10 27 38 23
+1 3 23 38 29 8
+1 3 8 29 39 33
+1 3 33 39 32 18
+1 3 39 31 30 32
+1 3 29 15 31 39
+1 3 15 5 16 30
+1 3 30 16 3 17
+1 3 18 30 17 6
+1 3 34 18 6 19
+1 3 40 33 18 36
+1 3 24 8 33 40
+1 3 11 24 40 35
+1 3 35 40 36 34
+1 3 7 11 34 20
+1 3 20 34 19 2
+
+boundary
+16
+2 1 0 9
+1 1 12 0
+1 1 7 12
+2 1 9 4
+2 1 4 13
+2 1 13 1
+1 1 1 14
+1 1 14 5
+1 1 5 16
+1 1 16 3
+2 1 3 17
+2 1 17 6
+2 1 6 19
+1 1 20 7
+2 1 19 2
+1 1 2 20
+
+vertex_parents
+37
+4 0 1
+5 1 3
+6 2 3
+7 0 2
+8 4 6
+9 0 4
+10 4 8
+11 7 8
+12 0 7
+13 1 4
+14 1 5
+15 5 8
+16 3 5
+17 3 6
+18 6 8
+19 2 6
+20 2 7
+21 9 11
+22 10 21
+23 8 10
+24 8 11
+25 11 21
+26 13 15
+27 10 26
+28 15 26
+29 8 15
+30 15 17
+31 15 30
+32 18 30
+33 8 18
+34 11 19
+35 11 34
+36 18 34
+37 22 24
+38 27 29
+39 29 32
+40 24 36
+
+coarse_elements
+9
+3 2 5 4 3
+3 0 6 28 1
+3 12 11 10 13
+3 7 8 9 30
+3 14 17 16 15
+3 32 18 19 20
+3 24 23 22 25
+3 26 34 21 27
+3 29 31 33 35
+
+vertices
+41
+
+nodes
+FiniteElementSpace
+FiniteElementCollection: H1_2D_P2
+VDim: 2
+Ordering: 0
+
+0
+1
+0
+1
+0.5
+1
+0.5
+0
+0.490917
+0.25
+0.481659
+0.229025
+0
+0.75
+1
+0.75002
+1
+0.75
+0.497731
+0.25
+0
+0.230224
+0.361824
+0.498798
+0.358947
+0.256374
+0.75627
+0.609156
+0.760931
+0.614878
+0.754468
+0.729327
+0.597212
+0.479497
+0.255854
+0.252575
+0.374701
+0.365809
+0.616315
+0.620005
+0.354453
+0.125
+0.241186
+0.113998
+0
+0.256374
+0.0996625
+0
+0.297495
+0.381624
+0.298118
+0.249986
+0.354661
+0.29368
+0.249387
+0.422671
+0.50313
+0.416618
+0.423212
+0.50141
+0.375
+0.498432
+0.361824
+0.625
+0.750551
+0.609156
+0.875
+1
+0.883899
+1
+0.859365
+0.760931
+0.66728
+0.757422
+0.680277
+0.604085
+0.680261
+0.760547
+0.611485
+0.542955
+0.563608
+0.556166
+0.630915
+0.564831
+0.496409
+0.603763
+0.54025
+0.492351
+0.666944
+0.736168
+0.668618
+0.733944
+1
+0.878096
+0.729327
+1
+0.875
+0.731558
+0.597212
+0.625
+0.509544
+0.374701
+0.375
+0.262228
+0.439501
+0.435693
+0.381762
+0.375151
+0.303302
+0.243334
+0.314755
+0.256748
+0.252575
+0.100238
+0
+0.125
+0
+0.0884843
+0.133844
+0.301451
+0.319617
+0.421308
+0.439579
+0.366542
+0.638837
+0.84411
+0.887525
+0.673669
+0.684397
+0.547681
+0.553932
+0.556981
+0.569094
+0.682474
+0.680908
+0.877814
+0.843209
+0.610964
+0.363741
+0.433969
+0.438104
+0.317325
+0.315253
+0.114408
+0.0955121
+0
+0
+1
+1
+0
+0.5
+1
+0.5
+0.500169
+0
+0.230361
+0.506286
+0.25
+0
+0.25
+0.501817
+0.75
+1
+0.751223
+1
+0.75
+0.245442
+0.249774
+0.359053
+0.482625
+0.339768
+0.25622
+0.246257
+0.342759
+0.493736
+0.737081
+0.598044
+0.732223
+0.628407
+0.755429
+0.618147
+0.736346
+0.380657
+0.367339
+0.619478
+0.619469
+0
+0.112152
+0.214709
+0.125
+0.339768
+0.511468
+0.375
+0.250576
+0.312269
+0.354505
+0.283581
+0.425524
+0.495609
+0.414003
+0.367212
+0.438493
+0.482955
+0.243036
+0.314443
+0
+0.130361
+0.249774
+0
+0.1228
+0.246257
+0
+0.125
+0.243197
+0.375
+0.460998
+0.342759
+0.372076
+0.413223
+0.487797
+0.431307
+0.25198
+0.290425
+0.296938
+0.239051
+0.353925
+0.485506
+0.542001
+0.613511
+0.565178
+0.672486
+0.738741
+0.683983
+0.614503
+0.662211
+0.73167
+0.544579
+0.625
+0.717877
+0.598044
+0.875
+1
+0.883366
+0.732223
+1
+0.888992
+0.736346
+1
+0.868768
+0.606623
+0.739539
+0.671004
+0.56527
+0.611998
+0.559039
+0.741642
+0.68361
+0.618147
+0.760055
+0.625
+1
+0.875
+0.110313
+0.337719
+0.306391
+0.440854
+0.431661
+0.308523
+0.0865872
+0.124325
+0.133215
+0.332392
+0.433604
+0.313603
+0.30387
+0.419458
+0.563411
+0.693606
+0.689503
+0.553108
+0.586083
+0.880316
+0.866348
+0.884747
+0.669969
+0.569241
+0.56415
+0.679016
+0.617255
+0.87398

miniapps/meshing/mesh-optimizer.cpp

@@ -32,6 +32,11 @@
 // Compile with: make mesh-optimizer
 //
 // Sample runs:
+//   Adapted analytic Hessian:
+//     mesh-optimizer -m square01.mesh -o 2 -rs 2 -mid 2 -tid 4 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
+//   Adapted discrete size:
+//     mesh-optimizer -m square01.mesh -o 2 -rs 2 -mid 7 -tid 5 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
+//
 //   Blade shape:
 //     mesh-optimizer -m blade.mesh -o 4 -rs 0 -mid 2 -tid 1 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
 //   Blade limited shape:
@@ -48,6 +53,9 @@
 //     mesh-optimizer -o 3 -rs 0 -mid 1 -tid 1 -ni 1000 -ls 2 -li 100 -bnd -qt 1 -qo 8 -cmb
 //   3D pinched sphere shape (the mesh is in the mfem/data GitHub repository):
 //   * mesh-optimizer -m ../../../mfem_data/ball-pert.mesh -o 4 -rs 0 -mid 303 -tid 1 -ni 20 -ls 2 -li 500 -fix-bnd
+//   2D non-conforming shape and equal size:
+//     mesh-optimizer -m ./amr-quad-q2.mesh -o 2 -rs 1 -mid 9 -tid 2 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
+
 
 #include "mfem.hpp"
 #include <fstream>
@@ -58,190 +66,160 @@ using namespace std;
 
 double weight_fun(const Vector &x);
 
-// Metric values are visualized by creating an L2 finite element functions and
-// computing the metric values at the nodes.
-void vis_metric(int order, TMOP_QualityMetric &qm, const TargetConstructor &tc,
-                Mesh &mesh, char *title, int position)
+double ind_values(const Vector &x)
 {
-   L2_FECollection fec(order, mesh.Dimension(), BasisType::GaussLobatto);
-   FiniteElementSpace fes(&mesh, &fec, 1);
-   GridFunction metric(&fes);
-   InterpolateTMOP_QualityMetric(qm, tc, mesh, metric);
-   osockstream sock(19916, "localhost");
-   sock << "solution\n";
-   mesh.Print(sock);
-   metric.Save(sock);
-   sock.send();
-   sock << "window_title '"<< title << "'\n"
-        << "window_geometry "
-        << position << " " << 0 << " " << 600 << " " << 600 << "\n"
-        << "keys jRmclA" << endl;
+   const int opt = 6;
+   const double small = 0.001, big = 0.01;
+
+   // Sine wave.
+   if (opt==1)
+   {
+      const double X = x(0), Y = x(1);
+      const double ind = std::tanh((10*(Y-0.5) + std::sin(4.0*M_PI*X)) + 1) -
+                         std::tanh((10*(Y-0.5) + std::sin(4.0*M_PI*X)) - 1);
+
+      return ind * small + (1.0 - ind) * big;
+   }
+
+   if (opt==2)
+   {
+      // Circle in the middle.
+      double val = 0.;
+      const double xc = x(0) - 0.5, yc = x(1) - 0.5;
+      const double r = sqrt(xc*xc + yc*yc);
+      double r1 = 0.15; double r2 = 0.35; double sf=30.0;
+      val = 0.5*(std::tanh(sf*(r-r1)) - std::tanh(sf*(r-r2)));
+      if (val > 1.) {val = 1;}
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   if (opt == 3)
+   {
+      // cross
+      const double X = x(0), Y = x(1);
+      const double r1 = 0.45, r2 = 0.55;
+      const double sf = 40.0;
+
+      double val = 0.5 * ( std::tanh(sf*(X-r1)) - std::tanh(sf*(X-r2)) +
+                           std::tanh(sf*(Y-r1)) - std::tanh(sf*(Y-r2)) );
+      if (val > 1.) { val = 1.0; }
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   if (opt==4)
+   {
+      // Multiple circles
+      double r1,r2,val,rval;
+      double sf = 10;
+      val = 0.;
+      // circle 1
+      r1= 0.25; r2 = 0.25; rval = 0.1;
+      double xc = x(0) - r1, yc = x(1) - r2;
+      double r = sqrt(xc*xc+yc*yc);
+      val =  0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*
+                                                               (r-rval)));// std::exp(val1);
+      // circle 2
+      r1= 0.75; r2 = 0.75;
+      xc = x(0) - r1, yc = x(1) - r2;
+      r = sqrt(xc*xc+yc*yc);
+      val +=  (0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*
+                                                                 (r-rval))));// std::exp(val1);
+      // circle 3
+      r1= 0.75; r2 = 0.25;
+      xc = x(0) - r1, yc = x(1) - r2;
+      r = sqrt(xc*xc+yc*yc);
+      val +=  0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*
+                                                                (r-rval)));// std::exp(val1);
+      // circle 4
+      r1= 0.25; r2 = 0.75;
+      xc = x(0) - r1, yc = x(1) - r2;
+      r = sqrt(xc*xc+yc*yc);
+      val +=  0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*(r-rval)));
+      if (val > 1.0) {val = 1.;}
+      if (val < 0.0) {val = 0.;}
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   if (opt==5)
+   {
+      // cross
+      double val = 0.;
+      double X = x(0)-0.5, Y = x(1)-0.5;
+      double rval = std::sqrt(X*X + Y*Y);
+      double thval = 60.*M_PI/180.;
+      double Xmod,Ymod;
+      Xmod = X*std::cos(thval) + Y*std::sin(thval);
+      Ymod= -X*std::sin(thval) + Y*std::cos(thval);
+      X = Xmod+0.5; Y = Ymod+0.5;
+      double r1 = 0.45; double r2 = 0.55; double sf=30.0;
+      val = ( 0.5*(1+std::tanh(sf*(X-r1))) - 0.5*(1+std::tanh(sf*(X-r2)))
+              + 0.5*(1+std::tanh(sf*(Y-r1))) - 0.5*(1+std::tanh(sf*(Y-r2))) );
+      if (rval > 0.4) {val = 0.;}
+      if (val > 1.0) {val = 1.;}
+      if (val < 0.0) {val = 0.;}
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   if (opt==6)
+   {
+      double val = 0.;
+      const double xc = x(0) - 0.0, yc = x(1) - 0.5;
+      const double r = sqrt(xc*xc + yc*yc);
+      double r1 = 0.45; double r2 = 0.55; double sf=30.0;
+      val = 0.5*(1+std::tanh(sf*(r-r1))) - 0.5*(1+std::tanh(sf*(r-r2)));
+      if (val > 1.) {val = 1;}
+      if (val < 0.) {val = 0;}
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   return 0.0;
 }
 
-class RelaxedNewtonSolver : public NewtonSolver
+class HessianCoefficient : public MatrixCoefficient
 {
 private:
-   // Quadrature points that are checked for negative Jacobians etc.
-   const IntegrationRule &ir;
-   FiniteElementSpace *fes;
-   mutable GridFunction x_gf;
+   int type;
 
 public:
-   RelaxedNewtonSolver(const IntegrationRule &irule, FiniteElementSpace *f)
-      : ir(irule), fes(f) { }
+   HessianCoefficient(int dim, int type_)
+      : MatrixCoefficient(dim), type(type_) { }
 
-   virtual double ComputeScalingFactor(const Vector &x, const Vector &b) const;
+   virtual void Eval(DenseMatrix &K, ElementTransformation &T,
+                     const IntegrationPoint &ip)
+   {
+      Vector pos(3);
+      T.Transform(ip, pos);
+
+      if (type == 0)
+      {
+         K(0, 0) = 1.0 + 3.0 * std::sin(M_PI*pos(0));
+         K(0, 1) = 0.0;
+         K(1, 0) = 0.0;
+         K(1, 1) = 1.0;
+      }
+      else
+      {
+         const double xc = pos(0) - 0.5, yc = pos(1) - 0.5;
+         const double r = sqrt(xc*xc + yc*yc);
+         double r1 = 0.15; double r2 = 0.35; double sf=30.0;
+         const double eps = 0.5;
+
+         const double tan1 = std::tanh(sf*(r-r1)),
+                      tan2 = std::tanh(sf*(r-r2));
+
+         K(0, 0) = eps + 1.0 * (tan1 - tan2);
+         K(0, 1) = 0.0;
+         K(1, 0) = 0.0;
+         K(1, 1) = 1.0;
+      }
+   }
 };
 
-double RelaxedNewtonSolver::ComputeScalingFactor(const Vector &x,
-                                                 const Vector &b) const
-{
-   const NonlinearForm *nlf = dynamic_cast<const NonlinearForm *>(oper);
-   MFEM_VERIFY(nlf != NULL, "invalid Operator subclass");
-   const bool have_b = (b.Size() == Height());
-
-   const int NE = fes->GetMesh()->GetNE(), dim = fes->GetFE(0)->GetDim(),
-             dof = fes->GetFE(0)->GetDof(), nsp = ir.GetNPoints();
-   Array<int> xdofs(dof * dim);
-   DenseMatrix Jpr(dim), dshape(dof, dim), pos(dof, dim);
-   Vector posV(pos.Data(), dof * dim);
-
-   Vector x_out(x.Size());
-   bool x_out_ok = false;
-   const double energy_in = nlf->GetEnergy(x);
-   double scale = 1.0, energy_out;
-   double norm0 = Norm(r);
-   x_gf.MakeTRef(fes, x_out, 0);
-
-   // Decreases the scaling of the update until the new mesh is valid.
-   for (int i = 0; i < 12; i++)
-   {
-      add(x, -scale, c, x_out);
-      x_gf.SetFromTrueVector();
-
-      energy_out = nlf->GetGridFunctionEnergy(x_gf);
-      if (energy_out > 1.2*energy_in || std::isnan(energy_out) != 0)
-      {
-         if (print_level >= 0)
-         { cout << "Scale = " << scale << " Increasing energy." << endl; }
-         scale *= 0.5; continue;
-      }
-
-      int jac_ok = 1;
-      for (int i = 0; i < NE; i++)
-      {
-         fes->GetElementVDofs(i, xdofs);
-         x_gf.GetSubVector(xdofs, posV);
-         for (int j = 0; j < nsp; j++)
-         {
-            fes->GetFE(i)->CalcDShape(ir.IntPoint(j), dshape);
-            MultAtB(pos, dshape, Jpr);
-            if (Jpr.Det() <= 0.0) { jac_ok = 0; goto break2; }
-         }
-      }
-   break2:
-      if (jac_ok == 0)
-      {
-         if (print_level >= 0)
-         { cout << "Scale = " << scale << " Neg det(J) found." << endl; }
-         scale *= 0.5; continue;
-      }
-
-      oper->Mult(x_out, r);
-      if (have_b) { r -= b; }
-      double norm = Norm(r);
-
-      if (norm > 1.2*norm0)
-      {
-         if (print_level >= 0)
-         { cout << "Scale = " << scale << " Norm increased." << endl; }
-         scale *= 0.5; continue;
-      }
-      else { x_out_ok = true; break; }
-   }
-
-   if (print_level >= 0)
-   {
-      cout << "Energy decrease: "
-           << (energy_in - energy_out) / energy_in * 100.0
-           << "% with " << scale << " scaling." << endl;
-   }
-
-   if (x_out_ok == false) { scale = 0.0; }
-
-   return scale;
-}
-
-// Allows negative Jacobians. Used in untangling metrics.
-class DescentNewtonSolver : public NewtonSolver
-{
-private:
-   // Quadrature points that are checked for negative Jacobians etc.
-   const IntegrationRule &ir;
-   FiniteElementSpace *fes;
-   mutable GridFunction x_gf;
-
-public:
-   DescentNewtonSolver(const IntegrationRule &irule, FiniteElementSpace *f)
-      : ir(irule), fes(f) { }
-
-   virtual double ComputeScalingFactor(const Vector &x, const Vector &b) const;
-};
-
-double DescentNewtonSolver::ComputeScalingFactor(const Vector &x,
-                                                 const Vector &b) const
-{
-   const NonlinearForm *nlf = dynamic_cast<const NonlinearForm *>(oper);
-   MFEM_VERIFY(nlf != NULL, "invalid Operator subclass");
-
-   const int NE = fes->GetMesh()->GetNE(), dim = fes->GetFE(0)->GetDim(),
-             dof = fes->GetFE(0)->GetDof(), nsp = ir.GetNPoints();
-   Array<int> xdofs(dof * dim);
-   DenseMatrix Jpr(dim), dshape(dof, dim), pos(dof, dim);
-   Vector posV(pos.Data(), dof * dim);
-
-   x_gf.MakeTRef(fes, x.GetData());
-   x_gf.SetFromTrueVector();
-
-   double min_detJ = infinity();
-   for (int i = 0; i < NE; i++)
-   {
-      fes->GetElementVDofs(i, xdofs);
-      x_gf.GetSubVector(xdofs, posV);
-      for (int j = 0; j < nsp; j++)
-      {
-         fes->GetFE(i)->CalcDShape(ir.IntPoint(j), dshape);
-         MultAtB(pos, dshape, Jpr);
-         min_detJ = min(min_detJ, Jpr.Det());
-      }
-   }
-   cout << "Minimum det(J) = " << min_detJ << endl;
-
-   Vector x_out(x.Size());
-   bool x_out_ok = false;
-   const double energy_in = nlf->GetGridFunctionEnergy(x_gf);
-   double scale = 1.0, energy_out;
-
-   for (int i = 0; i < 7; i++)
-   {
-      add(x, -scale, c, x_out);
-
-      energy_out = nlf->GetEnergy(x_out);
-      if (energy_out > energy_in || std::isnan(energy_out) != 0)
-      {
-         scale *= 0.5;
-      }
-      else { x_out_ok = true; break; }
-   }
-
-   cout << "Energy decrease: " << (energy_in - energy_out) / energy_in * 100.0
-        << "% with " << scale << " scaling." << endl;
-
-   if (x_out_ok == false) { return 0.0; }
-
-   return scale;
-}
-
 // Additional IntegrationRules that can be used with the --quad-type option.
 IntegrationRules IntRulesLo(0, Quadrature1D::GaussLobatto);
 IntegrationRules IntRulesCU(0, Quadrature1D::ClosedUniform);
@@ -304,7 +282,9 @@ int main (int argc, char *argv[])
                   "Target (ideal element) type:\n\t"
                   "1: Ideal shape, unit size\n\t"
                   "2: Ideal shape, equal size\n\t"
-                  "3: Ideal shape, initial size");
+                  "3: Ideal shape, initial size\n\t"
+                  "4: Given full analytic Jacobian (in physical space)\n\t"
+                  "5: Ideal shape, given size (in physical space)");
    args.AddOption(&lim_const, "-lc", "--limit-const", "Limiting constant.");
    args.AddOption(&quad_type, "-qt", "--quad-type",
                   "Quadrature rule type:\n\t"
@@ -375,7 +355,8 @@ int main (int argc, char *argv[])
    // 6. Get the mesh nodes (vertices and other degrees of freedom in the finite
    //    element space) as a finite element grid function in fespace. Note that
    //    changing x automatically changes the shapes of the mesh elements.
-   GridFunction *x = mesh->GetNodes();
+   GridFunction x(fespace);
+   mesh->SetNodalGridFunction(&x);
 
    // 7. Define a vector representing the minimal local mesh size in the mesh
    //    nodes. We index the nodes using the scalar version of the degrees of
@@ -425,10 +406,9 @@ int main (int argc, char *argv[])
       // Set the boundary values to zero.
       for (int j = 0; j < vdofs.Size(); j++) { rdm(vdofs[j]) = 0.0; }
    }
-   *x -= rdm;
-   // Set the perturbation of all nodes from the true nodes.
-   x->SetTrueVector();
-   x->SetFromTrueVector();
+   x -= rdm;
+   x.SetTrueVector();
+   x.SetFromTrueVector();
 
    // 9. Save the starting (prior to the optimization) mesh to a file. This
    //    output can be viewed later using GLVis: "glvis -m perturbed.mesh".
@@ -439,7 +419,7 @@ int main (int argc, char *argv[])
 
    // 10. Store the starting (prior to the optimization) positions.
    GridFunction x0(fespace);
-   x0 = *x;
+   x0 = x;
 
    // 11. Form the integrator that uses the chosen metric and target.
    double tauval = -0.1;
@@ -468,15 +448,43 @@ int main (int argc, char *argv[])
       default: cout << "Unknown metric_id: " << metric_id << endl; return 3;
    }
    TargetConstructor::TargetType target_t;
+   TargetConstructor *target_c = NULL;
+   HessianCoefficient *adapt_coeff = NULL;
+   H1_FECollection ind_fec(3, dim);
+   FiniteElementSpace ind_fes(mesh, &ind_fec);
+   GridFunction size;
    switch (target_id)
    {
       case 1: target_t = TargetConstructor::IDEAL_SHAPE_UNIT_SIZE; break;
       case 2: target_t = TargetConstructor::IDEAL_SHAPE_EQUAL_SIZE; break;
       case 3: target_t = TargetConstructor::IDEAL_SHAPE_GIVEN_SIZE; break;
-      default: cout << "Unknown target_id: " << target_id << endl;
-         delete metric; return 3;
+      case 4:
+      {
+         target_t = TargetConstructor::GIVEN_FULL;
+         AnalyticAdaptTC *tc = new AnalyticAdaptTC(target_t);
+         adapt_coeff = new HessianCoefficient(dim, 1);
+         tc->SetAnalyticTargetSpec(NULL, NULL, adapt_coeff);
+         target_c = tc;
+         break;
+      }
+      case 5:
+      {
+         target_t = TargetConstructor::IDEAL_SHAPE_GIVEN_SIZE;
+         DiscreteAdaptTC *tc = new DiscreteAdaptTC(target_t);
+         size.SetSpace(&ind_fes);
+         FunctionCoefficient ind_coeff(ind_values);
+         size.ProjectCoefficient(ind_coeff);
+         tc->SetSerialDiscreteTargetSpec(size);
+         target_c = tc;
+         break;
+      }
+      default: cout << "Unknown target_id: " << target_id << endl; return 3;
+   }
+
+   if (target_c == NULL)
+   {
+      target_c = new TargetConstructor(target_t);
    }
-   TargetConstructor *target_c = new TargetConstructor(target_t);
    target_c->SetNodes(x0);
    TMOP_Integrator *he_nlf_integ = new TMOP_Integrator(metric, target_c);
 
@@ -542,13 +550,13 @@ int main (int argc, char *argv[])
    }
    else { a.AddDomainIntegrator(he_nlf_integ); }
 
-   const double init_energy = a.GetGridFunctionEnergy(*x);
+   const double init_energy = a.GetGridFunctionEnergy(x);
 
    // 15. Visualize the starting mesh and metric values.
    if (visualization)
    {
       char title[] = "Initial metric values";
-      vis_metric(mesh_poly_deg, *metric, *target_c, *mesh, title, 0);
+      vis_tmop_metric_s(mesh_poly_deg, *metric, *target_c, *mesh, title, 0);
    }
 
    // 16. Fix all boundary nodes, or fix only a given component depending on the
@@ -652,8 +660,13 @@ int main (int argc, char *argv[])
    if (tauval > 0.0)
    {
       tauval = 0.0;
-      newton = new RelaxedNewtonSolver(*ir, fespace);
-      cout << "The RelaxedNewtonSolver is used (as all det(J)>0)." << endl;
+      TMOPNewtonSolver *tns = new TMOPNewtonSolver(*ir);
+      if (target_id == 5)
+      {
+         tns->SetDiscreteAdaptTC(dynamic_cast<DiscreteAdaptTC *>(target_c));
+      }
+      newton = tns;
+      cout << "TMOPNewtonSolver is used (as all det(J) > 0).\n";
    }
    else
    {
@@ -664,8 +677,8 @@ int main (int argc, char *argv[])
          return 3;
       }
       tauval -= 0.01 * h0.Min(); // Slightly below minJ0 to avoid div by 0.
-      newton = new DescentNewtonSolver(*ir, fespace);
-      cout << "The DescentNewtonSolver is used (as some det(J)<0)." << endl;
+      newton = new TMOPDescentNewtonSolver(*ir);
+      cout << "The TMOPDescentNewtonSolver is used (as some det(J) < 0).\n";
    }
    newton->SetPreconditioner(*S);
    newton->SetMaxIter(newton_iter);
@@ -673,8 +686,9 @@ int main (int argc, char *argv[])
    newton->SetAbsTol(0.0);
    newton->SetPrintLevel(verbosity_level >= 1 ? 1 : -1);
    newton->SetOperator(a);
-   newton->Mult(b, x->GetTrueVector());
-   x->SetFromTrueVector();
+   newton->Mult(b, x.GetTrueVector());
+   x.SetFromTrueVector();
+
    if (newton->GetConverged() == false)
    {
       cout << "NewtonIteration: rtol = " << newton_rtol << " not achieved."
@@ -691,12 +705,12 @@ int main (int argc, char *argv[])
    }
 
    // 21. Compute the amount of energy decrease.
-   const double fin_energy = a.GetGridFunctionEnergy(*x);
+   const double fin_energy = a.GetGridFunctionEnergy(x);
    double metric_part = fin_energy;
    if (lim_const != 0.0)
    {
       lim_coeff.constant = 0.0;
-      metric_part = a.GetGridFunctionEnergy(*x);
+      metric_part = a.GetGridFunctionEnergy(x);
       lim_coeff.constant = lim_const;
    }
    cout << "Initial strain energy: " << init_energy
@@ -712,13 +726,13 @@ int main (int argc, char *argv[])
    if (visualization)
    {
       char title[] = "Final metric values";
-      vis_metric(mesh_poly_deg, *metric, *target_c, *mesh, title, 600);
+      vis_tmop_metric_s(mesh_poly_deg, *metric, *target_c, *mesh, title, 600);
    }
 
    // 23. Visualize the mesh displacement.
    if (visualization)
    {
-      x0 -= *x;
+      x0 -= x;
       osockstream sock(19916, "localhost");
       sock << "solution\n";
       mesh->Print(sock);

miniapps/meshing/pmesh-optimizer.cpp

@@ -32,6 +32,11 @@
 // Compile with: make pmesh-optimizer
 //
 // Sample runs:
+//   Adapted analytic Hessian:
+//     mpirun -np 4 pmesh-optimizer -m square01.mesh -o 2 -rs 2 -mid 2 -tid 4 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
+//   Adapted discrete size:
+//     mpirun -np 4 pmesh-optimizer -m square01.mesh -o 2 -rs 2 -mid 7 -tid 5 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
+//
 //   Blade shape:
 //     mpirun -np 4 pmesh-optimizer -m blade.mesh -o 4 -rs 0 -mid 2 -tid 1 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
 //   Blade limited shape:
@@ -48,223 +53,176 @@
 //     mpirun -np 4 pmesh-optimizer -o 3 -rs 0 -mid 1 -tid 1 -ni 1000 -ls 2 -li 100 -bnd -qt 1 -qo 8 -cmb
 //   3D pinched sphere shape (the mesh is in the mfem/data GitHub repository):
 //   * mpirun -np 4 pmesh-optimizer -m ../../../mfem_data/ball-pert.mesh -o 4 -rs 0 -mid 303 -tid 1 -ni 20 -ls 2 -li 500 -fix-bnd
+//   2D non-conforming shape and equal size:
+//     mpirun -np 4 pmesh-optimizer -m ./amr-quad-q2.mesh -o 2 -rs 1 -mid 9 -tid 2 -ni 200 -ls 2 -li 100 -bnd -qt 1 -qo 8
 
 #include "mfem.hpp"
-#include <fstream>
 #include <iostream>
+#include <fstream>
 
 using namespace mfem;
 using namespace std;
 
 double weight_fun(const Vector &x);
 
-// Metric values are visualized by creating an L2 finite element functions and
-// computing the metric values at the nodes.
-void vis_metric(int order, TMOP_QualityMetric &qm, const TargetConstructor &tc,
-                ParMesh &pmesh, char *title, int position)
+double ind_values(const Vector &x)
 {
-   L2_FECollection fec(order, pmesh.Dimension(), BasisType::GaussLobatto);
-   ParFiniteElementSpace fes(&pmesh, &fec, 1);
-   ParGridFunction metric(&fes);
-   InterpolateTMOP_QualityMetric(qm, tc, pmesh, metric);
-   socketstream sock;
-   if (pmesh.GetMyRank() == 0)
+   const int opt = 6;
+   const double small = 0.001, big = 0.01;
+
+   // Sine wave.
+   if (opt==1)
    {
-      sock.open("localhost", 19916);
-      sock << "solution\n";
+      const double X = x(0), Y = x(1);
+      const double ind = std::tanh((10*(Y-0.5) + std::sin(4.0*M_PI*X)) + 1) -
+                         std::tanh((10*(Y-0.5) + std::sin(4.0*M_PI*X)) - 1);
+
+      return ind * small + (1.0 - ind) * big;
    }
-   pmesh.PrintAsOne(sock);
-   metric.SaveAsOne(sock);
-   if (pmesh.GetMyRank() == 0)
+
+   if (opt==2)
    {
-      sock << "window_title '"<< title << "'\n"
-           << "window_geometry "
-           << position << " " << 0 << " " << 600 << " " << 600 << "\n"
-           << "keys jRmclA" << endl;
+      // Circle in the middle.
+      double val = 0.;
+      const double xc = x(0) - 0.5, yc = x(1) - 0.5;
+      const double r = sqrt(xc*xc + yc*yc);
+      double r1 = 0.15; double r2 = 0.35; double sf=30.0;
+      val = 0.5*(std::tanh(sf*(r-r1)) - std::tanh(sf*(r-r2)));
+      if (val > 1.) {val = 1;}
+
+      return val * small + (1.0 - val) * big;
    }
+
+   if (opt == 3)
+   {
+      // cross
+      const double X = x(0), Y = x(1);
+      const double r1 = 0.45, r2 = 0.55;
+      const double sf = 40.0;
+
+      double val = 0.5 * ( std::tanh(sf*(X-r1)) - std::tanh(sf*(X-r2)) +
+                           std::tanh(sf*(Y-r1)) - std::tanh(sf*(Y-r2)) );
+      if (val > 1.) { val = 1.0; }
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   if (opt==4)
+   {
+      // Multiple circles
+      double r1,r2,val,rval;
+      double sf = 10;
+      val = 0.;
+      // circle 1
+      r1= 0.25; r2 = 0.25; rval = 0.1;
+      double xc = x(0) - r1, yc = x(1) - r2;
+      double r = sqrt(xc*xc+yc*yc);
+      val =  0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*
+                                                               (r-rval)));// std::exp(val1);
+      // circle 2
+      r1= 0.75; r2 = 0.75;
+      xc = x(0) - r1, yc = x(1) - r2;
+      r = sqrt(xc*xc+yc*yc);
+      val +=  (0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*
+                                                                 (r-rval))));// std::exp(val1);
+      // circle 3
+      r1= 0.75; r2 = 0.25;
+      xc = x(0) - r1, yc = x(1) - r2;
+      r = sqrt(xc*xc+yc*yc);
+      val +=  0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*
+                                                                (r-rval)));// std::exp(val1);
+      // circle 4
+      r1= 0.25; r2 = 0.75;
+      xc = x(0) - r1, yc = x(1) - r2;
+      r = sqrt(xc*xc+yc*yc);
+      val +=  0.5*(1+std::tanh(sf*(r+rval))) - 0.5*(1+std::tanh(sf*(r-rval)));
+      if (val > 1.0) {val = 1.;}
+      if (val < 0.0) {val = 0.;}
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   if (opt==5)
+   {
+      // cross
+      double val = 0.;
+      double X = x(0)-0.5, Y = x(1)-0.5;
+      double rval = std::sqrt(X*X + Y*Y);
+      double thval = 60.*M_PI/180.;
+      double Xmod,Ymod;
+      Xmod = X*std::cos(thval) + Y*std::sin(thval);
+      Ymod= -X*std::sin(thval) + Y*std::cos(thval);
+      X = Xmod+0.5; Y = Ymod+0.5;
+      double r1 = 0.45; double r2 = 0.55; double sf=30.0;
+      val = ( 0.5*(1+std::tanh(sf*(X-r1))) - 0.5*(1+std::tanh(sf*(X-r2)))
+              + 0.5*(1+std::tanh(sf*(Y-r1))) - 0.5*(1+std::tanh(sf*(Y-r2))) );
+      if (rval > 0.4) {val = 0.;}
+      if (val > 1.0) {val = 1.;}
+      if (val < 0.0) {val = 0.;}
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   if (opt==6)
+   {
+      double val = 0.;
+      const double xc = x(0) - 0.0, yc = x(1) - 0.5;
+      const double r = sqrt(xc*xc + yc*yc);
+      double r1 = 0.45; double r2 = 0.55; double sf=30.0;
+      val = 0.5*(1+std::tanh(sf*(r-r1))) - 0.5*(1+std::tanh(sf*(r-r2)));
+      if (val > 1.) {val = 1;}
+      if (val < 0.) {val = 0;}
+
+      return val * small + (1.0 - val) * big;
+   }
+
+   return 0.0;
 }
 
-class RelaxedNewtonSolver : public NewtonSolver
+class HessianCoefficient : public MatrixCoefficient
 {
 private:
-   // Quadrature points that are checked for negative Jacobians etc.
-   const IntegrationRule &ir;
-   ParFiniteElementSpace *pfes;
-   mutable ParGridFunction x_gf;
+   int type;
 
 public:
-   RelaxedNewtonSolver(const IntegrationRule &irule, ParFiniteElementSpace *pf)
-      : NewtonSolver(pf->GetComm()), ir(irule), pfes(pf) { }
+   HessianCoefficient(int dim, int type_)
+      : MatrixCoefficient(dim), type(type_) { }
 
-   virtual double ComputeScalingFactor(const Vector &x, const Vector &b) const;
+   virtual void Eval(DenseMatrix &K, ElementTransformation &T,
+                     const IntegrationPoint &ip)
+   {
+      Vector pos(3);
+      T.Transform(ip, pos);
+
+      if (type == 0)
+      {
+         K(0, 0) = 1.0 + 3.0 * std::sin(M_PI*pos(0));
+         K(0, 1) = 0.0;
+         K(1, 0) = 0.0;
+         K(1, 1) = 1.0;
+      }
+      else
+      {
+         const double xc = pos(0) - 0.5, yc = pos(1) - 0.5;
+         const double r = sqrt(xc*xc + yc*yc);
+         double r1 = 0.15; double r2 = 0.35; double sf=30.0;
+         const double eps = 0.5;
+
+         const double tan1 = std::tanh(sf*(r-r1)),
+                      tan2 = std::tanh(sf*(r-r2));
+
+         K(0, 0) = eps + 1.0 * (tan1 - tan2);
+         K(0, 1) = 0.0;
+         K(1, 0) = 0.0;
+         K(1, 1) = 1.0;
+      }
+   }
 };
 
-double RelaxedNewtonSolver::ComputeScalingFactor(const Vector &x,
-                                                 const Vector &b) const
-{
-   const ParNonlinearForm *nlf = dynamic_cast<const ParNonlinearForm *>(oper);
-   MFEM_VERIFY(nlf != NULL, "invalid Operator subclass");
-   const bool have_b = (b.Size() == Height());
-
-   const int NE = pfes->GetParMesh()->GetNE(), dim = pfes->GetFE(0)->GetDim(),
-             dof = pfes->GetFE(0)->GetDof(), nsp = ir.GetNPoints();
-   Array<int> xdofs(dof * dim);
-   DenseMatrix Jpr(dim), dshape(dof, dim), pos(dof, dim);
-   Vector posV(pos.Data(), dof * dim);
-
-   Vector x_out(x.Size());
-   bool x_out_ok = false;
-   const double energy_in = nlf->GetEnergy(x);
-   double scale = 1.0, energy_out;
-   double norm0 = Norm(r);
-   x_gf.MakeTRef(pfes, x_out, 0);
-
-   // Decreases the scaling of the update until the new mesh is valid.
-   for (int i = 0; i < 12; i++)
-   {
-      add(x, -scale, c, x_out);
-      x_gf.SetFromTrueVector();
-
-      energy_out = nlf->GetParGridFunctionEnergy(x_gf);
-      if (energy_out > 1.2*energy_in || std::isnan(energy_out) != 0)
-      {
-         if (print_level >= 0)
-         { cout << "Scale = " << scale << " Increasing energy." << endl; }
-         scale *= 0.5; continue;
-      }
-
-      int jac_ok = 1;
-      for (int i = 0; i < NE; i++)
-      {
-         pfes->GetElementVDofs(i, xdofs);
-         x_gf.GetSubVector(xdofs, posV);
-         for (int j = 0; j < nsp; j++)
-         {
-            pfes->GetFE(i)->CalcDShape(ir.IntPoint(j), dshape);
-            MultAtB(pos, dshape, Jpr);
-            if (Jpr.Det() <= 0.0) { jac_ok = 0; goto break2; }
-         }
-      }
-   break2:
-      int jac_ok_all;
-      MPI_Allreduce(&jac_ok, &jac_ok_all, 1, MPI_INT, MPI_LAND,
-                    pfes->GetComm());
-
-      if (jac_ok_all == 0)
-      {
-         if (print_level >= 0)
-         { cout << "Scale = " << scale << " Neg det(J) found." << endl; }
-         scale *= 0.5; continue;
-      }
-
-      oper->Mult(x_out, r);
-      if (have_b) { r -= b; }
-      double norm = Norm(r);
-
-      if (norm > 1.2*norm0)
-      {
-         if (print_level >= 0)
-         { cout << "Scale = " << scale << " Norm increased." << endl; }
-         scale *= 0.5; continue;
-      }
-      else { x_out_ok = true; break; }
-   }
-
-   if (print_level >= 0)
-   {
-      cout << "Energy decrease: "
-           << (energy_in - energy_out) / energy_in * 100.0
-           << "% with " << scale << " scaling." << endl;
-   }
-
-   if (x_out_ok == false) { scale = 0.0; }
-
-   return scale;
-}
-
-// Allows negative Jacobians. Used in untangling metrics.
-class DescentNewtonSolver : public NewtonSolver
-{
-private:
-   // Quadrature points that are checked for negative Jacobians etc.
-   const IntegrationRule &ir;
-   ParFiniteElementSpace *pfes;
-   mutable ParGridFunction x_gf;
-
-public:
-   DescentNewtonSolver(const IntegrationRule &irule, ParFiniteElementSpace *pf)
-      : NewtonSolver(pf->GetComm()), ir(irule), pfes(pf) { }
-
-   virtual double ComputeScalingFactor(const Vector &x, const Vector &b) const;
-};
-
-double DescentNewtonSolver::ComputeScalingFactor(const Vector &x,
-                                                 const Vector &b) const
-{
-   const ParNonlinearForm *nlf = dynamic_cast<const ParNonlinearForm *>(oper);
-   MFEM_VERIFY(nlf != NULL, "invalid Operator subclass");
-
-   const int NE = pfes->GetParMesh()->GetNE(), dim = pfes->GetFE(0)->GetDim(),
-             dof = pfes->GetFE(0)->GetDof(), nsp = ir.GetNPoints();
-   Array<int> xdofs(dof * dim);
-   DenseMatrix Jpr(dim), dshape(dof, dim), pos(dof, dim);
-   Vector posV(pos.Data(), dof * dim);
-
-   x_gf.MakeTRef(pfes, x.GetData());
-   x_gf.SetFromTrueVector();
-
-   double min_detJ = infinity();
-   for (int i = 0; i < NE; i++)
-   {
-      pfes->GetElementVDofs(i, xdofs);
-      x_gf.GetSubVector(xdofs, posV);
-      for (int j = 0; j < nsp; j++)
-      {
-         pfes->GetFE(i)->CalcDShape(ir.IntPoint(j), dshape);
-         MultAtB(pos, dshape, Jpr);
-         min_detJ = min(min_detJ, Jpr.Det());
-      }
-   }
-   double min_detJ_all;
-   MPI_Allreduce(&min_detJ, &min_detJ_all, 1, MPI_DOUBLE, MPI_MIN,
-                 pfes->GetComm());
-   if (print_level >= 0)
-   { cout << "Minimum det(J) = " << min_detJ_all << endl; }
-
-   Vector x_out(x.Size());
-   bool x_out_ok = false;
-   const double energy_in = nlf->GetParGridFunctionEnergy(x_gf);
-   double scale = 1.0, energy_out;
-
-   for (int i = 0; i < 7; i++)
-   {
-      add(x, -scale, c, x_out);
-
-      energy_out = nlf->GetEnergy(x_out);
-      if (energy_out > energy_in || std::isnan(energy_out) != 0)
-      {
-         scale *= 0.5;
-      }
-      else { x_out_ok = true; break; }
-   }
-
-   if (print_level >= 0)
-   {
-      cout << "Energy decrease: "
-           << (energy_in - energy_out) / energy_in * 100.0
-           << "% with " << scale << " scaling." << endl;
-   }
-
-   if (x_out_ok == false) { return 0.0; }
-
-   return scale;
-}
-
 // Additional IntegrationRules that can be used with the --quad-type option.
 IntegrationRules IntRulesLo(0, Quadrature1D::GaussLobatto);
 IntegrationRules IntRulesCU(0, Quadrature1D::ClosedUniform);
 
-
 int main (int argc, char *argv[])
 {
    // 0. Initialize MPI.
@@ -331,7 +289,9 @@ int main (int argc, char *argv[])
                   "Target (ideal element) type:\n\t"
                   "1: Ideal shape, unit size\n\t"
                   "2: Ideal shape, equal size\n\t"
-                  "3: Ideal shape, initial size");
+                  "3: Ideal shape, initial size\n\t"
+                  "4: Given full analytic Jacobian (in physical space)\n\t"
+                  "5: Ideal shape, given size (in physical space)");
    args.AddOption(&lim_const, "-lc", "--limit-const", "Limiting constant.");
    args.AddOption(&quad_type, "-qt", "--quad-type",
                   "Quadrature rule type:\n\t"
@@ -381,6 +341,7 @@ int main (int argc, char *argv[])
       cout << endl;
    }
    ParMesh *pmesh = new ParMesh(MPI_COMM_WORLD, *mesh);
+
    delete mesh;
    for (int lev = 0; lev < rp_levels; lev++) { pmesh->UniformRefinement(); }
 
@@ -510,20 +471,47 @@ int main (int argc, char *argv[])
          return 3;
    }
    TargetConstructor::TargetType target_t;
+   TargetConstructor *target_c = NULL;
+   HessianCoefficient *adapt_coeff = NULL;
+   H1_FECollection ind_fec(3, dim);
+   ParFiniteElementSpace ind_fes(pmesh, &ind_fec);
+   ParGridFunction size;
    switch (target_id)
    {
       case 1: target_t = TargetConstructor::IDEAL_SHAPE_UNIT_SIZE; break;
       case 2: target_t = TargetConstructor::IDEAL_SHAPE_EQUAL_SIZE; break;
       case 3: target_t = TargetConstructor::IDEAL_SHAPE_GIVEN_SIZE; break;
+      case 4:
+      {
+         target_t = TargetConstructor::GIVEN_FULL;
+         AnalyticAdaptTC *tc = new AnalyticAdaptTC(target_t);
+         adapt_coeff = new HessianCoefficient(dim, 1);
+         tc->SetAnalyticTargetSpec(NULL, NULL, adapt_coeff);
+         target_c = tc;
+         break;
+      }
+      case 5:
+      {
+         target_t = TargetConstructor::IDEAL_SHAPE_GIVEN_SIZE;
+         DiscreteAdaptTC *tc = new DiscreteAdaptTC(target_t);
+         size.SetSpace(&ind_fes);
+         FunctionCoefficient ind_coeff(ind_values);
+         size.ProjectCoefficient(ind_coeff);
+         tc->SetParDiscreteTargetSpec(size);
+         target_c = tc;
+         break;
+      }
       default:
          if (myid == 0) { cout << "Unknown target_id: " << target_id << endl; }
          return 3;
    }
-   TargetConstructor *target_c;
-   target_c = new TargetConstructor(target_t, MPI_COMM_WORLD);
+
+   if (target_c == NULL)
+   {
+      target_c = new TargetConstructor(target_t, MPI_COMM_WORLD);
+   }
    target_c->SetNodes(x0);
-   TMOP_Integrator *he_nlf_integ;
-   he_nlf_integ = new TMOP_Integrator(metric, target_c);
+   TMOP_Integrator *he_nlf_integ= new TMOP_Integrator(metric, target_c);
 
    // 13. Setup the quadrature rule for the non-linear form integrator.
    const IntegrationRule *ir = NULL;
@@ -597,7 +585,7 @@ int main (int argc, char *argv[])
    if (visualization)
    {
       char title[] = "Initial metric values";
-      vis_metric(mesh_poly_deg, *metric, *target_c, *pmesh, title, 0);
+      vis_tmop_metric_p(mesh_poly_deg, *metric, *target_c, *pmesh, title, 0);
    }
 
    // 17. Fix all boundary nodes, or fix only a given component depending on the
@@ -705,9 +693,14 @@ int main (int argc, char *argv[])
    if (tauval > 0.0)
    {
       tauval = 0.0;
-      newton = new RelaxedNewtonSolver(*ir, pfespace);
+      TMOPNewtonSolver *tns = new TMOPNewtonSolver(pfespace->GetComm(), *ir);
+      if (target_id == 5)
+      {
+         tns->SetDiscreteAdaptTC(dynamic_cast<DiscreteAdaptTC *>(target_c));
+      }
+      newton = tns;
       if (myid == 0)
-      { cout << "RelaxedNewtonSolver is used (as all det(J) > 0)." << endl; }
+      { cout << "TMOPNewtonSolver is used (as all det(J) > 0)." << endl; }
    }
    else
    {
@@ -715,15 +708,15 @@ int main (int argc, char *argv[])
            (dim == 3 && metric_id != 352) )
       {
          if (myid == 0)
-         { cout << "The mesh is inverted. Use an untangling metric." << endl; }
+         { cout << "The mesh is inverted. Use an untangling metric.\n"; }
          return 3;
       }
       double h0min = h0.Min(), h0min_all;
       MPI_Allreduce(&h0min, &h0min_all, 1, MPI_DOUBLE, MPI_MIN, MPI_COMM_WORLD);
       tauval -= 0.01 * h0min_all; // Slightly below minJ0 to avoid div by 0.
-      newton = new DescentNewtonSolver(*ir, pfespace);
+      newton = new TMOPDescentNewtonSolver(pfespace->GetComm(), *ir);
       if (myid == 0)
-      { cout << "DescentNewtonSolver is used (as some det(J) < 0)." << endl; }
+      { cout << "TMOPDescentNewtonSolver is used (as some det(J) < 0).\n"; }
    }
    newton->SetPreconditioner(*S);
    newton->SetMaxIter(newton_iter);
@@ -775,7 +768,7 @@ int main (int argc, char *argv[])
    if (visualization)
    {
       char title[] = "Final metric values";
-      vis_metric(mesh_poly_deg, *metric, *target_c, *pmesh, title, 600);
+      vis_tmop_metric_p(mesh_poly_deg, *metric, *target_c, *pmesh, title, 600);
    }
 
    // 23. Visualize the mesh displacement.
@@ -805,6 +798,7 @@ int main (int argc, char *argv[])
    delete metric2;
    delete coeff1;
    delete target_c;
+   delete adapt_coeff;
    delete metric;
    delete pfespace;
    delete fec;