including an obstacle problem variant wherein the essential dofs are not given to the optimizer as free variables that it must choose the values for.

removing bug in the evaluation of the optimality measures wherein all processors now see the same value of the optimality error
clean up... using a GeneralOptimizationProblem class and an OptimizationProblem class and no longer using the specific ContactProblem class as the generality of the Parent class does not have anything per say to do with contact
2024-03-01 14:58:16 -08:00 · 2023-11-10 10:43:20 -08:00 · 2023-10-10 11:33:29 -07:00 · 2023-08-07 10:07:55 -07:00 · 2023-08-07 10:06:35 -07:00 · 2023-07-25 19:20:04 -07:00
467 changed files with 25855 additions and 62337 deletions
@@ -1,6 +1,7 @@
 name: "Docker"

 on:
+
  # Always have a base image ready to go - this is a nightly build
  schedule:
    - cron: 0 3 * * *
@@ -25,6 +26,7 @@ jobs:
    strategy:
      fail-fast: false
      matrix:
+
        # Dockerfiles to build, a matrix supports future expanded builds
        container: [["config/docker/Dockerfile.base", "ghcr.io/mfem/mfem-ubuntu-base"],
                    ["config/docker/Dockerfile", "ghcr.io/mfem/mfem-ubuntu"]]
@@ -32,20 +34,15 @@ jobs:
    runs-on: ubuntu-latest
    name: Build
    steps:
-      - name: Run Actions Cleaner
-        uses: easimon/maximize-build-space@v8
-        with:
-          overprovision-lvm: 'true'
-          remove-dotnet: 'true'
-          remove-android: 'true'
-          remove-haskell: 'true'
-          remove-codeql: 'true'
-          remove-docker-images: 'true'
-
      - name: Checkout
        uses: actions/checkout@v3

-      # It's easier to reference named variables than indexes of the matrix
+      - name: Make Space For Build
+        run: |
+          sudo rm -rf /usr/share/dotnet
+          sudo rm -rf /opt/ghc
+
+       # It's easier to reference named variables than indexes of the matrix
      - name: Set Environment
        env:
          dockerfile: ${{ matrix.container[0] }}
@@ -65,16 +65,13 @@ jobs:
        # - Add a new combination.
        #   'build-system: cmake' and 'hypre-target: int64'
        #
-        # Note: we will gather coverage info for any non-debug run except the
+        # note: we will gather coverage info for any non-debug run except the
        # CMake build.
        include:
          - target: dbg
            codecov: NO
          - target: opt
            codecov: YES
-          - os: ubuntu-latest
-            target: dbg
-            config-opts: 'CPPFLAGS+=-Og'
          - os: windows-latest
            codecov: NO
          - os: windows-latest
@@ -101,189 +98,179 @@ jobs:
    runs-on: ${{ matrix.os }}

    steps:
-      # This external action allows to interrupt a workflow already running on
-      # the same branch to save resources.
-      - name: Cancel Previous Runs
-        uses: styfle/cancel-workflow-action@0.11.0
-        with:
-          access_token: ${{ github.token }}
+    # This external action allows to interrupt a workflow already running on
+    # the same branch to save resource
+    - name: Cancel Previous Runs
+      uses: styfle/cancel-workflow-action@0.11.0
+      with:
+        access_token: ${{ github.token }}

-      # Fix 'No space left on device' errors for Ubuntu builds.
-      - name: Run Actions Cleaner
-        if: matrix.os == 'ubuntu-latest'
-        uses: easimon/maximize-build-space@v8
-        with:
-          overprovision-lvm: 'true'
-          remove-android: 'true'
+    # Checkout MFEM in "mfem" subdirectory. Final path:
+    # /home/runner/work/mfem/mfem/mfem
+    # Note: Done now to access "install-hypre" and "install-metis" actions.
+    - name: checkout mfem
+      uses: actions/checkout@v3
+      with:
+        path: ${{ env.MFEM_TOP_DIR }}
+        # Fetch the complete history for codecov to access commits ID
+        fetch-depth: 0

-      # Checkout MFEM in "mfem" subdirectory. Final path:
-      # /home/runner/work/mfem/mfem/mfem
-      # Note: Done now to access "install-hypre" and "install-metis" actions.
-      - name: checkout mfem
-        uses: actions/checkout@v3
-        with:
-          path: ${{ env.MFEM_TOP_DIR }}
-          # Fetch the complete history for codecov to access commits ID
-          fetch-depth: 0
+    # Only get MPI if defined for the job.
+    # TODO: It would be nice to have only one step, e.g. with a dedicated
+    # action, but I (@adrienbernede) don't see how at the moment.
+    - name: get MPI (Linux)
+      if: matrix.mpi == 'par' && matrix.os == 'ubuntu-latest'
+      run: |
+        sudo apt-get install mpich libmpich-dev

-      # Only get MPI if defined for the job.
-      # TODO: It would be nice to have only one step, e.g. with a dedicated
-      # action, but I (@adrienbernede) don't see how at the moment.
-      - name: get MPI (Linux)
-        if: matrix.mpi == 'par' && matrix.os == 'ubuntu-latest'
-        run: |
-          sudo apt-get install mpich libmpich-dev
+    - name: get lcov (Linux)
+      if: matrix.codecov == 'YES' && matrix.os == 'ubuntu-latest'
+      run: |
+        sudo apt-get install lcov

-      - name: get lcov (Linux)
-        if: matrix.codecov == 'YES' && matrix.os == 'ubuntu-latest'
-        run: |
-          sudo apt-get install lcov
+    # Keep the following section in case we need it again in the future,
+    # see: https://github.com/mfem/mfem/pull/3385#discussion_r1058013032
+    # - name: Set up Homebrew
+    #   if: ( matrix.mpi == 'par' || matrix.codecov == 'YES' ) && matrix.os == 'macos-latest'
+    #   uses: Homebrew/actions/setup-homebrew@master

-      # Keep the following section in case we need it again in the future,
-      # see: https://github.com/mfem/mfem/pull/3385#discussion_r1058013032
-      # - name: Set up Homebrew
-      #   if: ( matrix.mpi == 'par' || matrix.codecov == 'YES' ) && matrix.os == 'macos-latest'
-      #   uses: Homebrew/actions/setup-homebrew@master
+    - name: get MPI (MacOS)
+      if: matrix.mpi == 'par' && matrix.os == 'macos-latest'
+      run: |
+        export HOMEBREW_NO_INSTALL_CLEANUP=1
+        brew install openmpi

-      - name: get MPI (MacOS)
-        if: matrix.mpi == 'par' && matrix.os == 'macos-latest'
-        run: |
-          export HOMEBREW_NO_INSTALL_CLEANUP=1
-          brew install openmpi
+    - name: get lcov (MacOS)
+      if: matrix.codecov == 'YES' && matrix.os == 'macos-latest'
+      run: |
+        export HOMEBREW_NO_INSTALL_CLEANUP=1
+        brew install lcov

-      - name: get lcov (MacOS)
-        if: matrix.codecov == 'YES' && matrix.os == 'macos-latest'
-        run: |
-          export HOMEBREW_NO_INSTALL_CLEANUP=1
-          brew install lcov
+    - name: get MPI (Windows)
+      if: matrix.mpi == 'par' && matrix.os == 'windows-latest'
+      uses: mpi4py/setup-mpi@v1.1.4

-      - name: get MPI (Windows)
-        if: matrix.mpi == 'par' && matrix.os == 'windows-latest'
-        uses: mpi4py/setup-mpi@v1.1.4
+    # Get Hypre through cache, or build it.
+    # Install will only run on cache miss.
+    - name: cache hypre
+      id: hypre-cache
+      if: matrix.mpi == 'par'
+      uses: actions/cache@v3
+      with:
+        path: ${{ env.HYPRE_TOP_DIR }}
+        key: ${{ runner.os }}-build-${{ env.HYPRE_TOP_DIR }}-${{ matrix.hypre-target }}-v2.2

-      # Get Hypre through cache, or build it.
-      # Install will only run on cache miss.
-      - name: cache hypre
-        id: hypre-cache
-        if: matrix.mpi == 'par'
-        uses: actions/cache@v3
-        with:
-          path: ${{ env.HYPRE_TOP_DIR }}
-          key: ${{ runner.os }}-build-${{ env.HYPRE_TOP_DIR }}-${{ matrix.hypre-target }}-v2.2
+    - name: get hypre
+      if: matrix.mpi == 'par' && steps.hypre-cache.outputs.cache-hit != 'true' && matrix.os != 'windows-latest'
+      uses: mfem/github-actions/build-hypre@v2.2
+      with:
+        archive: ${{ env.HYPRE_ARCHIVE }}
+        dir: ${{ env.HYPRE_TOP_DIR }}
+        target: ${{ matrix.hypre-target }}
+        build-system: make

-      - name: get hypre
-        if: matrix.mpi == 'par' && steps.hypre-cache.outputs.cache-hit != 'true' && matrix.os != 'windows-latest'
-        uses: mfem/github-actions/build-hypre@v2.4
-        with:
-          archive: ${{ env.HYPRE_ARCHIVE }}
-          dir: ${{ env.HYPRE_TOP_DIR }}
-          target: ${{ matrix.hypre-target }}
-          build-system: make
+    - name: get hypre (Windows)
+      if: matrix.mpi == 'par' && steps.hypre-cache.outputs.cache-hit != 'true' && matrix.os == 'windows-latest'
+      uses: mfem/github-actions/build-hypre@v2.2
+      with:
+        archive: ${{ env.HYPRE_ARCHIVE }}
+        dir: ${{ env.HYPRE_TOP_DIR }}
+        target: ${{ matrix.hypre-target }}
+        build-system: cmake

-      - name: get hypre (Windows)
-        if: matrix.mpi == 'par' && steps.hypre-cache.outputs.cache-hit != 'true' && matrix.os == 'windows-latest'
-        uses: mfem/github-actions/build-hypre@v2.4
-        with:
-          archive: ${{ env.HYPRE_ARCHIVE }}
-          dir: ${{ env.HYPRE_TOP_DIR }}
-          target: ${{ matrix.hypre-target }}
-          build-system: cmake
+    # Get Metis through cache, or build it.
+    # Install will only run on cache miss.
+    - name: cache metis
+      id: metis-cache
+      if: matrix.mpi == 'par' && matrix.os != 'windows-latest'
+      uses: actions/cache@v3
+      with:
+        path: ${{ env.METIS_TOP_DIR }}
+        key: ${{ runner.os }}-build-${{ env.METIS_TOP_DIR }}-v2.2

-      # Get Metis through cache, or build it.
-      # Install will only run on cache miss.
-      - name: cache metis
-        id: metis-cache
-        if: matrix.mpi == 'par' && matrix.os != 'windows-latest'
-        uses: actions/cache@v3
-        with:
-          path: ${{ env.METIS_TOP_DIR }}
-          key: ${{ runner.os }}-build-${{ env.METIS_TOP_DIR }}-v2.2
+    - name: install metis
+      if: matrix.mpi == 'par' && matrix.os != 'windows-latest' && steps.metis-cache.outputs.cache-hit != 'true'
+      uses: mfem/github-actions/build-metis@v2.2
+      with:
+        archive: ${{ env.METIS_ARCHIVE }}
+        dir: ${{ env.METIS_TOP_DIR }}

-      - name: install metis
-        if: matrix.mpi == 'par' && matrix.os != 'windows-latest' && steps.metis-cache.outputs.cache-hit != 'true'
-        uses: mfem/github-actions/build-metis@v2.4
-        with:
-          archive: ${{ env.METIS_ARCHIVE }}
-          dir: ${{ env.METIS_TOP_DIR }}
+    - name: cache vcpkg (Windows)
+      id: vcpkg-cache
+      if: matrix.os == 'windows-latest'
+      uses: actions/cache@v3
+      with:
+        path: vcpkg_cache
+        key: ${{ runner.os }}-${{ matrix.mpi }}-vcpkg-v1

-      - name: cache vcpkg (Windows)
-        id: vcpkg-cache
-        if: matrix.os == 'windows-latest'
-        uses: actions/cache@v3
-        with:
-          path: vcpkg_cache
-          key: ${{ runner.os }}-${{ matrix.mpi }}-vcpkg-v1
+    - name: prepare vcpkg binary cache location (Windows)
+      if: matrix.os == 'windows-latest' && steps.vcpkg-cache.outputs.cache-hit != 'true'
+      run: |
+        mkdir -p vcpkg_cache

-      - name: prepare vcpkg binary cache location (Windows)
-        if: matrix.os == 'windows-latest' && steps.vcpkg-cache.outputs.cache-hit != 'true'
-        run: |
-          mkdir -p vcpkg_cache
+    - name: install metis (Windows)
+      if: matrix.mpi == 'par' && matrix.os == 'windows-latest'
+      env:
+        VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}/vcpkg_cache
+      run: |
+        vcpkg install metis-mfem --triplet=x64-windows-static --overlay-ports=${{ env.MFEM_TOP_DIR }}/config/vcpkg/ports

-      - name: install metis (Windows)
-        if: matrix.mpi == 'par' && matrix.os == 'windows-latest'
-        env:
-          VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}/vcpkg_cache
-        run: |
-          vcpkg install metis-mfem --triplet=x64-windows-static --overlay-ports=${{ env.MFEM_TOP_DIR }}/config/vcpkg/ports
+    # MFEM build and test
+    - name: build
+      uses: mfem/github-actions/build-mfem@v2.3
+      env:
+        VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}/vcpkg_cache
+      with:
+        os: ${{ matrix.os }}
+        target: ${{ matrix.target }}
+        codecov: ${{ matrix.codecov }}
+        mpi: ${{ matrix.mpi }}
+        build-system: ${{ matrix.build-system }}
+        hypre-dir: ${{ env.HYPRE_TOP_DIR }}
+        metis-dir: ${{ env.METIS_TOP_DIR }}
+        mfem-dir: ${{ env.MFEM_TOP_DIR }}
+        config-options: ${{ matrix.config-opts }}
+        library-only: ${{ matrix.target == 'dbg' }}

-      # MFEM build and test
-      - name: build
-        uses: mfem/github-actions/build-mfem@v2.4
-        env:
-          VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}/vcpkg_cache
-        with:
-          os: ${{ matrix.os }}
-          target: ${{ matrix.target }}
-          codecov: ${{ matrix.codecov }}
-          mpi: ${{ matrix.mpi }}
-          build-system: ${{ matrix.build-system }}
-          hypre-dir: ${{ env.HYPRE_TOP_DIR }}
-          metis-dir: ${{ env.METIS_TOP_DIR }}
-          mfem-dir: ${{ env.MFEM_TOP_DIR }}
-          config-options: ${{ matrix.config-opts }}
-          library-only: ${{ matrix.target == 'dbg' && matrix.os != 'ubuntu-latest' }}
+    # Run checks (and only checks) on debug targets
+    - name: checks
+      if: matrix.build-system == 'make' && matrix.target == 'dbg'
+      run: |
+        cd ${{ env.MFEM_TOP_DIR }}  && make check

-      # Run checks (and only checks) on debug targets
-      - name: checks
-        if: matrix.build-system == 'make' && matrix.target == 'dbg'
-        run: |
-          cd ${{ env.MFEM_TOP_DIR }}  && make check
+    # Note: 'tests' include the unit tests
+    - name: tests
+      if: matrix.build-system == 'make' && matrix.target == 'opt'
+      run: |
+        cd ${{ env.MFEM_TOP_DIR }} && make test

-      # Note: 'tests' include the unit tests
-      - name: tests
-        if: matrix.build-system == 'make' && (matrix.target == 'opt' || matrix.os == 'ubuntu-latest')
-        run: |
-          cd ${{ env.MFEM_TOP_DIR }} && make test
+    - name: cmake checks
+      if: matrix.build-system == 'cmake' && matrix.target == 'dbg'
+      run: |
+        CTEST_CONFIG="Debug"
+        cd ${{ env.MFEM_TOP_DIR }} && cmake --build build --target check --config ${CTEST_CONFIG}
+      shell: bash

-      - name: cmake checks
-        if: matrix.build-system == 'cmake' && matrix.target == 'dbg'
-        run: |
-          CTEST_CONFIG="Debug"
-          cd ${{ env.MFEM_TOP_DIR }} && cmake --build build --target check --config ${CTEST_CONFIG}
-        shell: bash
+    - name: cmake unit tests (Ubuntu)
+      if: matrix.build-system == 'cmake' && matrix.target == 'opt' && matrix.os == 'ubuntu-latest'
+      run: |
+        CTEST_CONFIG="Release"
+        [[ ${{ matrix.target }} == 'dbg' ]] && CTEST_CONFIG="Debug"
+        cd ${{ env.MFEM_TOP_DIR }}/build/tests/unit && ctest --output-on-failure -C ${CTEST_CONFIG}
+      shell: bash

-      - name: cmake unit tests (Ubuntu)
-        if: matrix.build-system == 'cmake' && matrix.target == 'opt' && matrix.os == 'ubuntu-latest'
-        run: |
-          CTEST_CONFIG="Release"
-          [[ ${{ matrix.target }} == 'dbg' ]] && CTEST_CONFIG="Debug"
-          cd ${{ env.MFEM_TOP_DIR }}/build/tests/unit && ctest --output-on-failure -C ${CTEST_CONFIG}
-        shell: bash
+    - name: cmake tests
+      if: matrix.build-system == 'cmake' && matrix.target == 'opt' && matrix.os != 'ubuntu-latest'
+      run: |
+        CTEST_CONFIG="Release"
+        cd ${{ env.MFEM_TOP_DIR }}/build && ctest --output-on-failure -C ${CTEST_CONFIG}
+      shell: bash

-      - name: cmake tests
-        if: matrix.build-system == 'cmake' && matrix.target == 'opt' && matrix.os != 'ubuntu-latest'
-        run: |
-          CTEST_CONFIG="Release"
-          cd ${{ env.MFEM_TOP_DIR }}/build && \
-          ctest --output-on-failure -C ${CTEST_CONFIG} || \
-          ctest --rerun-failed --output-on-failure -C ${CTEST_CONFIG}
-        shell: bash
-
-      # Code coverage (process and upload reports)
-      - name: codecov
-        if: matrix.codecov == 'YES'
-        uses: mfem/github-actions/upload-coverage@v2.4
-        with:
-          name: ${{ matrix.os }}-${{ matrix.build-system }}-${{ matrix.target }}-${{ matrix.mpi }}-${{ matrix.hypre-target }}
-          project_dir: ${{ env.MFEM_TOP_DIR }}
-          directories: "fem general linalg mesh"
+    # Code coverage (process and upload reports)
+    - name: codecov
+      if: matrix.codecov == 'YES'
+      uses: mfem/github-actions/upload-coverage@v2.2
+      with:
+        name: ${{ matrix.os }}-${{ matrix.build-system }}-${{ matrix.target }}-${{ matrix.mpi }}-${{ matrix.hypre-target }}
+        project_dir: ${{ env.MFEM_TOP_DIR }}
+        directories: "fem general linalg mesh"
@@ -13,10 +13,10 @@ name: "Static Analysis"

 on:
  push:
-    branches: ["master", "next"]
+    branches: [ "master", "next"]
  pull_request:
    # The branches below must be a subset of the branches above
-    branches: ["master"]
+    branches: [ "master" ]

 jobs:
  analyze:
@@ -35,35 +35,36 @@ jobs:
        # Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v3
+    - name: Checkout repository
+      uses: actions/checkout@v3

-      # Initializes the CodeQL tools for scanning.
-      - name: Initialize CodeQL
-        uses: github/codeql-action/init@v2
-        with:
-          languages: ${{ matrix.language }}
-          # If you wish to specify custom queries, you can do so here or in a config file.
-          # By default, queries listed here will override any specified in a config file.
-          # Prefix the list here with "+" to use these queries and those in the config file.
+    # Initializes the CodeQL tools for scanning.
+    - name: Initialize CodeQL
+      uses: github/codeql-action/init@v2
+      with:
+        languages: ${{ matrix.language }}
+        # If you wish to specify custom queries, you can do so here or in a config file.
+        # By default, queries listed here will override any specified in a config file.
+        # Prefix the list here with "+" to use these queries and those in the config file.

-          # Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
-          # queries: security-extended,security-and-quality
+        # Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
+        # queries: security-extended,security-and-quality

-      # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
-      # If this step fails, then you should remove it and run the build manually (see below)
-      - name: Autobuild
-        uses: github/codeql-action/autobuild@v2

-      # ℹ️ Command-line programs to run using the OS shell.
-      # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
+    # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
+    # If this step fails, then you should remove it and run the build manually (see below)
+    - name: Autobuild
+      uses: github/codeql-action/autobuild@v2

-      #   If the Autobuild fails above, remove it and uncomment the following three lines.
-      #   modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
+    # ℹ️ Command-line programs to run using the OS shell.
+    # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

-      # - run: |
-      #   echo "Run, Build Application using script"
-      #   ./location_of_script_within_repo/buildscript.sh
+    #   If the Autobuild fails above, remove it and uncomment the following three lines.
+    #   modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.

-      - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@v2
+    # - run: |
+    #   echo "Run, Build Application using script"
+    #   ./location_of_script_within_repo/buildscript.sh
+
+    - name: Perform CodeQL Analysis
+      uses: github/codeql-action/analyze@v2
@@ -34,67 +34,67 @@ jobs:
    runs-on: ubuntu-latest

    steps:
-      - name: Cancel Previous Runs
-        uses: styfle/cancel-workflow-action@0.11.0
-        with:
-          access_token: ${{ github.token }}
+    - name: Cancel Previous Runs
+      uses: styfle/cancel-workflow-action@0.11.0
+      with:
+        access_token: ${{ github.token }}

-      - name: checkout MFEM
-        uses: actions/checkout@v3
-        with:
-          path: mfem
+    - name: checkout MFEM
+      uses: actions/checkout@v3
+      with:
+        path: mfem

-      - name: Get MPI (Linux)
-        run: |
-          sudo apt-get install mpich libmpich-dev
+    - name: Get MPI (Linux)
+      run: |
+        sudo apt-get install mpich libmpich-dev

-      - name: Cache Hypre Install
-        id: hypre-cache
-        uses: actions/cache@v3
-        with:
-          path: ${{ env.HYPRE_TOP_DIR }}
-          key: ${{ runner.os }}-build-${{ env.HYPRE_TOP_DIR }}-v2.2
+    - name: Cache Hypre Install
+      id: hypre-cache
+      uses: actions/cache@v3
+      with:
+        path: ${{ env.HYPRE_TOP_DIR }}
+        key: ${{ runner.os }}-build-${{ env.HYPRE_TOP_DIR }}-v2.2

-      - name: Get Hypre
-        if: steps.hypre-cache.outputs.cache-hit != 'true'
-        uses: mfem/github-actions/build-hypre@v2.4
-        with:
-          archive: ${{ env.HYPRE_ARCHIVE }}
-          dir: ${{ env.HYPRE_TOP_DIR }}
-          target: int32
+    - name: Get Hypre
+      if: steps.hypre-cache.outputs.cache-hit != 'true'
+      uses: mfem/github-actions/build-hypre@v2.2
+      with:
+        archive: ${{ env.HYPRE_ARCHIVE }}
+        dir: ${{ env.HYPRE_TOP_DIR }}
+        target: int32

-      - name: Cache Metis Install
-        id: metis-cache
-        uses: actions/cache@v3
-        with:
-          path: ${{ env.METIS_TOP_DIR }}
-          key: ${{ runner.os }}-build-${{ env.METIS_TOP_DIR }}-v2.2
+    - name: Cache Metis Install
+      id: metis-cache
+      uses: actions/cache@v3
+      with:
+        path: ${{ env.METIS_TOP_DIR }}
+        key: ${{ runner.os }}-build-${{ env.METIS_TOP_DIR }}-v2.2

-      - name: Install Metis
-        if: steps.metis-cache.outputs.cache-hit != 'true'
-        uses: mfem/github-actions/build-metis@v2.4
-        with:
-          archive: ${{ env.METIS_ARCHIVE }}
-          dir: ${{ env.METIS_TOP_DIR }}
+    - name: Install Metis
+      if: steps.metis-cache.outputs.cache-hit != 'true'
+      uses: mfem/github-actions/build-metis@v2.2
+      with:
+        archive: ${{ env.METIS_ARCHIVE }}
+        dir: ${{ env.METIS_TOP_DIR }}

-      # MFEM build and test
-      - name: build-mfem
-        uses: mfem/github-actions/build-mfem@v2.4
-        with:
-          os: ${{ runner.os }}
-          target: opt
-          codecov: NO
-          mpi: par
-          build-system: make
-          hypre-dir: ${{ env.HYPRE_TOP_DIR }}
-          metis-dir: ${{ env.METIS_TOP_DIR }}
-          mfem-dir: mfem
+    # MFEM build and test
+    - name: build-mfem
+      uses: mfem/github-actions/build-mfem@v2.2
+      with:
+        os: ${{ runner.os }}
+        target: opt
+        codecov: NO
+        mpi: par
+        build-system: make
+        hypre-dir: ${{ env.HYPRE_TOP_DIR }}
+        metis-dir: ${{ env.METIS_TOP_DIR }}
+        mfem-dir: mfem

-      - name: test (no clean)
-        run: |
-          cd mfem && make test-noclean
+    - name: test (no clean)
+      run: |
+        cd mfem && make test-noclean

-      - name: gitignore
-        run: |
-          cd mfem/tests/scripts
-          ./runtest gitignore
+    - name: gitignore
+      run: |
+        cd mfem/tests/scripts
+        ./runtest gitignore
@@ -1,70 +0,0 @@
-# Copyright (c) 2010-2023, Lawrence Livermore National Security, LLC. Produced
-# at the Lawrence Livermore National Laboratory. All Rights reserved. See files
-# LICENSE and NOTICE for details. LLNL-CODE-806117.
-#
-# This file is part of the MFEM library. For more information and source code
-# availability visit https://mfem.org.
-#
-# MFEM is free software; you can redistribute it and/or modify it under the
-# terms of the BSD-3 license. We welcome feedback and contributions, see file
-# CONTRIBUTING.md for details.
-
-name: "Sanitizer"
-
-permissions:
-  actions: write
-
-on:
-  push:
-    branches:
-      - master
-      - next
-  pull_request:
-  workflow_dispatch:
-
-jobs:
-  Serial:
-    runs-on: ubuntu-latest
-
-    steps:
-      - name: Cancel Previous Runs
-        uses: styfle/cancel-workflow-action@0.11.0
-        with:
-          access_token: ${{ github.token }}
-
-      - name: MFEM Checkout
-        uses: actions/checkout@v3
-        with:
-          path: mfem
-
-      - name: MFEM Build
-        uses: mfem/github-actions/build-mfem@v2.4
-        with:
-          os: ${{ runner.os }}
-          target: opt
-          mpi: seq
-          hypre-dir: unused-hypre-dir
-          metis-dir: unused-metis-dir
-          mfem-dir: mfem
-          build-system: make
-          library-only: false
-          config-options:
-            CXX="clang++-14"
-            CXXFLAGS="-g -O1 -std=c++11
-                      -fsanitize=address
-                      -fno-omit-frame-pointer
-                      -fsanitize-address-use-after-scope"
-
-      - name: MFEM Info
-        working-directory: mfem
-        run: make info
-
-      - name: MFEM Sanitize
-        working-directory: mfem
-        run:
-          ASAN_OPTIONS="detect_leaks=1,
-                        strict_init_order=1,
-                        strict_string_checks=1,
-                        check_initialization_order=1,
-                        detect_stack_use_after_return=1"
-          make test
@@ -33,49 +33,49 @@ jobs:
      (github.event_name == 'push' ||
       github.event.pull_request.head.repo.full_name != github.repository)
    steps:
-      - name: Cancel Previous Runs
-        uses: styfle/cancel-workflow-action@0.11.0
-        with:
-          access_token: ${{ github.token }}
+    - name: Cancel Previous Runs
+      uses: styfle/cancel-workflow-action@0.11.0
+      with:
+        access_token: ${{ github.token }}

-      - name: checkout mfem
-        uses: actions/checkout@v3
+    - name: checkout mfem
+      uses: actions/checkout@v3

-      - name: copyright check
-        id: copyright
-        run: |
-          ./config/githooks/pre-push --copyright
+    - name: copyright check
+      id: copyright
+      run: |
+        ./config/githooks/pre-push --copyright

-        continue-on-error: true
+      continue-on-error: true

-      - name: license check
-        id: license
-        run: |
-          ./config/githooks/pre-push --license
-        continue-on-error: true
+    - name: license check
+      id: license
+      run: |
+        ./config/githooks/pre-push --license
+      continue-on-error: true

-      - name: release check
-        id: release
-        run: |
-          ./config/githooks/pre-push --release
-        continue-on-error: true
+    - name: release check
+      id: release
+      run: |
+        ./config/githooks/pre-push --release
+      continue-on-error: true

-      - name: wrap-up
-        if: |
-          steps.copyright.outcome != 'success' ||
-          steps.license.outcome != 'success' ||
-          steps.release.outcome != 'success'
-        run: |
-          if [[ "${{ steps.copyright.outcome }}" != "success" ]]; then
-            echo "copyright check failed, unroll log for details"
-          fi
-          if [[ "${{ steps.license.outcome }}" != "success" ]]; then
-            echo "license check failed, unroll log for details"
-          fi
-          if [[ "${{ steps.release.outcome }}" != "success" ]]; then
-            echo "release check failed, unroll log for details"
-          fi
-          exit 1
+    - name: wrap-up
+      if: |
+        steps.copyright.outcome != 'success' ||
+        steps.license.outcome != 'success' ||
+        steps.release.outcome != 'success'
+      run: |
+        if [[ "${{ steps.copyright.outcome }}" != "success" ]]; then
+          echo "copyright check failed, unroll log for details"
+        fi
+        if [[ "${{ steps.license.outcome }}" != "success" ]]; then
+          echo "license check failed, unroll log for details"
+        fi
+        if [[ "${{ steps.release.outcome }}" != "success" ]]; then
+          echo "release check failed, unroll log for details"
+        fi
+        exit 1

  code-style:
    runs-on: ubuntu-latest
@@ -83,16 +83,16 @@ jobs:
      (github.event_name == 'push' ||
       github.event.pull_request.head.repo.full_name != github.repository)
    steps:
-      - name: checkout mfem
-        uses: actions/checkout@v3
+    - name: checkout mfem
+      uses: actions/checkout@v3

-      - name: get astyle
-        run: |
-          sudo apt-get install astyle
+    - name: get astyle
+      run: |
+        sudo apt-get install astyle

-      - name: style check
-        run: |
-          ./config/githooks/pre-push --style
+    - name: style check
+      run: |
+        ./config/githooks/pre-push --style

  documentation:
    runs-on: ubuntu-latest
@@ -100,22 +100,22 @@ jobs:
      (github.event_name == 'push' ||
       github.event.pull_request.head.repo.full_name != github.repository)
    steps:
-      - name: checkout mfem
-        uses: actions/checkout@v3
+    - name: checkout mfem
+      uses: actions/checkout@v3

-      - name: get doxygen and graphviz
-        run: |
-          sudo apt-get install doxygen graphviz
+    - name: get doxygen and graphviz
+      run: |
+        sudo apt-get install doxygen graphviz

-      - name: update doxygen config file
-        run: |
-          cd doc
-          doxygen -u CodeDocumentation.conf.in
+    - name: update doxygen config file
+      run: |
+        cd doc
+        doxygen -u CodeDocumentation.conf.in

-      - name: build documentation
-        run: |
-          cd tests/scripts
-          ./runtest documentation
+    - name: build documentation
+      run: |
+        cd tests/scripts
+        ./runtest documentation

  branch-history:
    if: |
@@ -125,16 +125,16 @@ jobs:
       github.event.pull_request.head.repo.full_name != github.repository)
    runs-on: ubuntu-latest
    steps:
-      - name: checkout mfem
-        uses: actions/checkout@v3
-        with:
-          fetch-depth: 0
+    - name: checkout mfem
+      uses: actions/checkout@v3
+      with:
+        fetch-depth: 0

-      - name: branch-history
-        run: |
-          # We override origin to make sure we point to the main repo.
-          # This is to have consistent test results on PRs from forks.
-          git remote remove origin
-          git remote add origin https://github.com/mfem/mfem.git
-          git checkout -b gh-actions-branch-history
-          ./config/githooks/pre-push --history
+    - name: branch-history
+      run: |
+        # We override origin to make sure we point to the main repo.
+        # This is to have consistent test results on PRs from forks.
+        git remote remove origin
+        git remote add origin https://github.com/mfem/mfem.git
+        git checkout -b gh-actions-branch-history
+        ./config/githooks/pre-push --history
@@ -29,7 +29,6 @@ CMakeFiles/
 config/_config.hpp
 config/config.mk
 config/sample-runs-build.log
-config/user.cmake
 config/user.mk
 doc/CodeDocumentation.conf
 doc/CodeDocumentation.html
@@ -113,12 +112,6 @@ examples/ex25p-*.*
 examples/ex28_*
 examples/ex28p_*
 examples/flux.*
-examples/dsol.*
-examples/cond.*
-examples/cond_j.*
-examples/cond_mesh.*
-examples/port_mesh.*
-examples/port_mode.*

 examples/amgx/ex1
 examples/amgx/ex1p
@@ -213,7 +206,6 @@ miniapps/meshing/twist
 miniapps/meshing/mesh-explorer
 miniapps/meshing/shaper
 miniapps/meshing/extruder
-miniapps/meshing/fit-node-position
 miniapps/meshing/trimmer
 miniapps/meshing/reflector
 miniapps/meshing/mesh-optimizer
@@ -222,7 +214,6 @@ miniapps/meshing/pmesh-fitting
 miniapps/meshing/minimal-surface
 miniapps/meshing/pminimal-surface
 miniapps/meshing/polar-nc
-miniapps/meshing/mesh-quality
 miniapps/meshing/mobius-strip.mesh
 miniapps/meshing/klein-bottle.mesh
 miniapps/meshing/toroid-*.mesh
@@ -266,16 +257,11 @@ miniapps/navier/*_output
 miniapps/nurbs/nurbs_ex1
 miniapps/nurbs/nurbs_ex1p
 miniapps/nurbs/nurbs_ex11p
-miniapps/nurbs/nurbs_printfunc
-miniapps/nurbs/nurbs_patch_ex1
-miniapps/nurbs/nurbs_curveint
 miniapps/nurbs/refined.mesh
 miniapps/nurbs/mesh.*
 miniapps/nurbs/sol.*
 miniapps/nurbs/mode_*
 miniapps/nurbs/Example1*
-miniapps/nurbs/sin-fit.mesh
-miniapps/nurbs/CurveInt

 miniapps/performance/ex1
 miniapps/performance/ex1p
@@ -298,14 +284,9 @@ miniapps/tools/display-basis
 miniapps/tools/load-dc
 miniapps/tools/convert-dc
 miniapps/tools/lor-transfer
-miniapps/tools/plor-transfer
 miniapps/tools/get-values
 miniapps/tools/check-tmop-metric
 miniapps/tools/tmop-metric-magnitude
-miniapps/tools/nodal-transfer
-miniapps/tools/ParaView
-miniapps/tools/gridfunc_*
-miniapps/tools/mesh_*

 miniapps/toys/automata
 miniapps/toys/life
@@ -335,28 +316,12 @@ miniapps/solvers/ParaView
 miniapps/solvers/mesh.*
 miniapps/solvers/sol.*

-miniapps/hdiv-linear-solver/darcy
-miniapps/hdiv-linear-solver/grad_div
-
 miniapps/parelag/MultilevelHcurlHdivSolver
 miniapps/parelag/*.mesh

 miniapps/multidomain/multidomain
 miniapps/hooke/hooke

-miniapps/dpg/diffusion
-miniapps/dpg/pdiffusion
-miniapps/dpg/convection-diffusion
-miniapps/dpg/pconvection-diffusion
-miniapps/dpg/acoustics
-miniapps/dpg/pacoustics
-miniapps/dpg/maxwell
-miniapps/dpg/pmaxwell
-miniapps/dpg/ParaView
-
-miniapps/spde/generate_random_field
-miniapps/spde/ParaView
-
 # Unit test binary and outputs
 tests/unit/output_meshes
 tests/unit/unit_tests
@@ -369,8 +334,6 @@ tests/unit/tmop_pa_tests_*
 tests/unit/ptmop_pa_tests_*
 tests/unit/ceed_tests
 tests/unit/debug_device_tests
-tests/unit/parallel_in_serial.mesh
-tests/unit/parallel_in_serial.gf

 # Benchmark binaries
 tests/benchmarks/bench_ceed
@@ -22,10 +22,12 @@
        date
        echo "Waiting to acquire lock on '$PWD/autotest.lock' ..."
        # try to get an exclusive lock on fd 9 (autotest.lock) repeating the try
-        # every 5 seconds; we may want to add a counter for the number of
+        # every 5 seconds; simply using no timeout, i.e. 'flock 9', causes the
+        # command to hang indefinitely sometimes, so we use the timeout & retry
+        # as a workaround; we may want to add a counter for the number of
        # retries to interrupt a potential infinite loop
-        while ! flock -n 9; do
-          sleep 5
+        while ! flock -w 5 9; do
+          true
        done
        echo "Acquired lock on '$PWD/autotest.lock'"
        date
@@ -55,10 +57,12 @@
        date
        echo "Waiting to acquire lock on '$PWD/autotest.lock' ..."
        # try to get an exclusive lock on fd 9 (autotest.lock) repeating the try
-        # every 5 seconds; we may want to add a counter for the number of
+        # every 5 seconds; simply using no timeout, i.e. 'flock 9', causes the
+        # command to hang indefinitely sometimes, so we use the timeout & retry
+        # as a workaround; we may want to add a counter for the number of
        # retries to interrupt a potential infinite loop
-        while ! flock -n 9; do
-          sleep 5
+        while ! flock -w 5 9; do
+          true
        done
        echo "Acquired lock on '$PWD/autotest.lock'"
        date
@@ -47,10 +47,12 @@ setup_baseline:
        date
        echo "Waiting to acquire lock on '$PWD/autotest.lock' ..."
        # try to get an exclusive lock on fd 9 (autotest.lock) repeating the try
-        # every 5 seconds; we may want to add a counter for the number of
+        # every 5 seconds; simply using no timeout, i.e. 'flock 9', causes the
+        # command to hang indefinitely sometimes, so we use the timeout & retry
+        # as a workaround; we may want to add a counter for the number of
        # retries to interrupt a potential infinite loop
-        while ! flock -n 9; do
-          sleep 5
+        while ! flock -w 5 9; do
+          true
        done
        echo "Acquired lock on '$PWD/autotest.lock'"
        date
@@ -35,11 +35,13 @@ setup:
      (
        date
        echo "Waiting to acquire lock on '$PWD/mfem-data.lock' ..."
-        # try to get an exclusive lock on fd 9 (mfem-data.lock) repeating the
-        # try every 5 seconds; we may want to add a counter for the number of
+        # try to get an exclusive lock on fd 9 (mfem-data.lock) repeating the try
+        # every 5 seconds; simply using no timeout, i.e. 'flock 9', causes the
+        # command to hang indefinitely sometimes, so we use the timeout & retry
+        # as a workaround; we may want to add a counter for the number of
        # retries to interrupt a potential infinite loop
-        while ! flock -n 9; do
-          sleep 5
+        while ! flock -w 5 9; do
+          true
        done
        echo "Acquired lock on '$PWD/mfem-data.lock'"
        date
@@ -67,10 +69,12 @@ setup:
        date
        echo "Waiting to acquire lock on '$PWD/autotest.lock' ..."
        # try to get an exclusive lock on fd 9 (autotest.lock) repeating the try
-        # every 5 seconds; we may want to add a counter for the number of
+        # every 5 seconds; simply using no timeout, i.e. 'flock 9', causes the
+        # command to hang indefinitely sometimes, so we use the timeout & retry
+        # as a workaround; we may want to add a counter for the number of
        # retries to interrupt a potential infinite loop
-        while ! flock -n 9; do
-          sleep 5
+        while ! flock -w 5 9; do
+          true
        done
        echo "Acquired lock on '$PWD/autotest.lock'"
        date
@@ -14,14 +14,14 @@ stages:
  - build_and_test
  - report

-opt_mpi_cuda_xl_16_1_1_12:
+opt_mpi_cuda_xl_16_1_1_8:
  variables:
-    SPEC: "%xl@16.1.1.12 +mpi +cuda cuda_arch=70"
+    SPEC: "%xl@16.1.1.8 +mpi +cuda cuda_arch=70"
  extends: .build_and_test_on_lassen

 opt_mpi_cuda_hypre_cuda_xl:
  variables:
-    SPEC: "%xl@16.1.1.12 +mpi +cuda cuda_arch=70 ^hypre+cuda~shared cuda_arch=70"
+    SPEC: "%xl@16.1.1.8 +mpi +cuda cuda_arch=70 ^hypre+cuda~shared cuda_arch=70"
  extends: .build_and_test_on_lassen

 # Jobs report
@@ -51,8 +51,6 @@ cleanup:
  script:
    - echo "BUILD_ROOT=${BUILD_ROOT}"
    - rm -rf "${BUILD_ROOT}" || true
-    - echo "CI_PROJECT_DIR=${CI_PROJECT_DIR}"
-    - make -C "${CI_PROJECT_DIR}" distclean

 report_baseline:
  extends: [.on_quartz]
@@ -68,10 +66,12 @@ report_baseline:
        date
        echo "Waiting to acquire lock on '$PWD/autotest.lock' ..."
        # try to get an exclusive lock on fd 9 (autotest.lock) repeating the try
-        # every 5 seconds; we may want to add a counter for the number of
+        # every 5 seconds; simply using no timeout, i.e. 'flock 9', causes the
+        # command to hang indefinitely sometimes, so we use the timeout & retry
+        # as a workaround; we may want to add a counter for the number of
        # retries to interrupt a potential infinite loop
-        while ! flock -n 9; do
-          sleep 5
+        while ! flock -w 5 9; do
+          true
        done
        echo "Acquired lock on '$PWD/autotest.lock'"
        date
@@ -82,14 +82,12 @@ report_baseline:
        rundir="${MACHINE_NAME}/$(date +%Y-%m-%d)-gitlab-${BASELINE_TEST}-${CI_COMMIT_REF_SLUG}"
        rundir=$(${CI_PROJECT_DIR}/.gitlab/scripts/safe_create_rundir ${rundir})
        cp ${CI_PROJECT_DIR}/${ARTIFACTS_DIR}/* ${rundir}
+        # We create an autotest-email.html file, because that's how we signal that there was a diff (temporary).
+        if [[ -f ${rundir}/${BASELINE_TEST}.err ]]; then
+          cp ${rundir}/${BASELINE_TEST}.err ${rundir}/autotest-email.html
+        fi
        printf "%s\n" "" "Pipeline URL:" "$CI_PIPELINE_URL" \
          >> ${rundir}/pipeline.txt
-        # We create an autotest-email.html file, because that's how we signal
-        # that there was an error / diff (temporary).
-        if [[ -f ${rundir}/${BASELINE_TEST}.err ]] || \
-           [[ -f ${rundir}/${BASELINE_TEST}-${SYS_TYPE}.diff ]]; then
-          cp ${rundir}/pipeline.txt ${rundir}/autotest-email.html
-        fi
        msg="GitLab CI log for ${BASELINE_TEST} on ${MACHINE_NAME} ($(date +%Y-%m-%d))"
        if [[ "$AUTOTEST_COMMIT" != "NO" ]]; then
          git pull && \
@@ -27,39 +27,39 @@ allocate_resource:
  timeout: 6h

 # GitLab jobs for the Quartz machine at LLNL
-debug_ser_gcc_10:
+debug_ser_gcc_6_1_0:
  variables:
-    SPEC: "%gcc@10.3.1 +debug~mpi"
+    SPEC: "%gcc@6.1.0 +debug~mpi"
  extends: .build_and_test_on_quartz

-debug_par_gcc_10:
+debug_par_gcc_6_1_0:
  variables:
-    SPEC: "%gcc@10.3.1 +debug+mpi"
+    SPEC: "%gcc@6.1.0 +debug+mpi"
  extends: .build_and_test_on_quartz

-opt_ser_gcc_10:
+opt_ser_gcc_6_1_0:
  variables:
-    SPEC: "%gcc@10.3.1 ~mpi"
+    SPEC: "%gcc@6.1.0 ~mpi"
  extends: .build_and_test_on_quartz

-opt_par_gcc_10:
+opt_par_gcc_6_1_0:
  variables:
-    SPEC: "%gcc@10.3.1"
+    SPEC: "%gcc@6.1.0"
  extends: .build_and_test_on_quartz

-opt_par_gcc_10_sundials:
+opt_par_gcc_6_1_0_sundials:
  variables:
-    SPEC: "%gcc@10.3.1 +sundials"
+    SPEC: "%gcc@6.1.0 +sundials"
  extends: .build_and_test_on_quartz

-opt_par_gcc_10_petsc:
+opt_par_gcc_6_1_0_petsc:
  variables:
-    SPEC: "%gcc@10.3.1 +petsc ^petsc+mumps~superlu-dist"
+    SPEC: "%gcc@6.1.0 +petsc ^petsc+mumps~superlu-dist"
  extends: .build_and_test_on_quartz

-opt_par_gcc_10_pumi:
+opt_par_gcc_6_1_0_pumi:
  variables:
-    SPEC: "%gcc@10.3.1 +pumi"
+    SPEC: "%gcc@6.1.0 +pumi"
  extends: .build_and_test_on_quartz

 # Release
@@ -42,34 +42,47 @@ fi
 # post
 mkdir ${artifacts_path}

-status=0
-if [[ -f ${BASELINE_TEST}.out ]]; then
-  cp ${BASELINE_TEST}.out ${artifacts_path}
-fi
-if [[ -s ${glob_err} ]]; then
-  echo "ERROR during ${BASELINE_TEST} execution"
-  echo "Here is the ${glob_err} file content"
+if [[ -s ${glob_err} ]]
+then
+  echo "ERROR during ${BASELINE_TEST} execution";
+  echo "Here is the ${glob_err} file content";
  cat ${glob_err}
  cp ${glob_err} ${artifacts_path}/${glob_err}
-  status=1
-fi
-if [[ -f ${base_patch} ]]; then
+  exit 1;
+elif [[ ! -f ${base_patch} && ! -f ${base_out} ]]
+then
+  echo "Something went WRONG in ${BASELINE_TEST}:";
+  echo "Either ${base_patch} or ${base_out} should exists";
+  exit 1;
+elif [[ -f ${base_patch} ]]
+then
  echo "${BASELINE_TEST}: Differences found, patch generated"
  cp ${base_patch} ${artifacts_path}/${base_patch}
-elif [[ -f ${base_out} ]]; then
+elif [[ -f ${base_out} ]]
+then
  echo "${BASELINE_TEST}: Differences found, replacement file generated"
  cp ${base_out} ${artifacts_path}/${base_out}
 fi
+
+if [[ -f ${BASELINE_TEST}.out ]]; then
+  cp ${BASELINE_TEST}.out ${artifacts_path}
+fi
+
 # base_diff won't even exist if there is no difference.
-if [[ -f ${base_diff} ]]; then
+if [[ -f ${base_diff} ]]
+then
  echo "${BASELINE_TEST}: Relevant differences (filtered diff) ..."
  cat ${base_diff}
  cp ${base_diff} ${artifacts_path}/${base_diff}
-  status=1
+  # We create a .err file, because that's how we signal that there was a diff.
+  cp ${base_diff} ${artifacts_path}/gitlab-${BASELINE_TEST}-${MACHINE_NAME}.err
 fi
-if [[ $status -eq 0 ]]; then
+
+if [[ ! -s ${base_diff} ]]
+then
  echo "${BASELINE_TEST}: PASSED"
+  true
 else
  echo "${BASELINE_TEST}: FAILED"
+  false
 fi
-exit $status
@@ -8,148 +8,25 @@
                               https://mfem.org


-Version 4.6.1 (development)
+Version 4.5.3 (development)
 ===========================

-Discretization improvements
---------------------------
- Introduced support for higher order non conformal Nedelec elements on
-  simplices in ParMesh.
+- Added new methods in the Mesh class to set and get attributes on NURBS patches
+  and patch boundaries.

-Miscellaneous
-------------
- The ReadCubit Genesis mesh importer has been rewritten to improve readability.
+New and updated examples and miniapps
+-------------------------------------
+- Added a miniapp pmesh-fitting in miniapps/meshing for interface and boundary fitting to implicit domains defined using level-set functions.

-
-Version 4.6, released on September 27, 2023
-===========================================
-
- MFEM is now available in Homebrew and can be installed on a Mac with just
-  "brew install mfem". See https://formulae.brew.sh/formula/mfem.
-
-Meshing improvements
--------------------
- Added asymptotically-balanced TMOP compound metrics 90, 94, 328, 338. A new
-  tool, tmop-metric-magnitude, can be used to track how metrics change under
-  geometric perturbations. See miniapps/tools.
-
- Several NURBS meshing improvements:
-  * Support for free connectivity of NURBS patches allowing for more complex
-    patch configurations such as C-meshes.
-  * New methods to set and get attributes on NURBS patches and patch boundaries.
-  * The edge to knot map for NURBS meshes can be determined automatically. It is
-    no longer needed to specify this in the NURBS mesh.
-  * Added curve interpolation method for NURBS.
-  * Added new small miniapp for printing of shape functions of a KnotVector
-  * See miniapps/nurbs for example meshes and miniapps.
+- Moved the distance solver methods from miniapps/shifted to miniapps/common. 

 Discretization improvements
 ---------------------------
- SubMesh and ParSubMesh have been extended to support the transfer of
-  Nedelec and Raviart-Thomas finite element spaces.
-
- Added support for partial assembly on NURBS patches, and NURBS-patch sparse
-  matrix assembly. Patch matrix assembly includes the option to use reduced
-  approximate integration rules, computed by the newly implemented non-negative
-  least-squares (NNLS) solver.
-
- Support for parallel transfer of H1 fields using the low-order refined (LOR)
-  transfer operators in L2ProjectionGridTransfer
-
- Added KDTree class for 2D/3D set of points, which is then utilized in the new
-  KDTreeNodalProjection class to project a function defined on an arbitrary set
-  of points onto an MFEM grid function. This functionality is demonstrated in
-  the nodal-transfer miniapp. The current implementation is serial only. Further
-  extensions can include search in arbitrary dimensional spaces.
-
- Added support for p-refined meshes in GSLIB-FindPoints.
-
- Device kernels can now access device-specific DOF and quadrature limits using
-  the DofQuadLimits structure, allowing increased limits when executing on CPU.
-  The limits for the runtime selected device can be accessed in host code using
-  DeviceDofQuadLimits::Get(). The global constants MAX_D1D and MAX_Q1D are no
-  longer available.
-
 - Face restriction operators for Nedelec and Raviart-Thomas finite element
  spaces are now supported through the ConformingFaceRestriction class.

 - VectorFEBoundaryFluxLFIntegrator is now supported on device/GPU.

-Linear and nonlinear solvers
----------------------------
- Updated the MUMPS interface to support multiple right-hand sides, block
-  low-rank compression, builds using 64-bit integers, and other improvements.
-
- Added an interface to the MKL Pardiso sparse direct solver developed by Intel.
-  The interface provides a serial (OpenMP shared memory) version of Pardiso for
-  use with SparseMatrix. This complements the existing parallel (MPI distributed
-  memory) version already available through the CPardiso MFEM integration.
-
- Added HIP support to the PETSc and SUNDIALS interfaces.
-
-New and updated examples and miniapps
-------------------------------------
- Added a new H(div) solver miniapp demonstrating the use of a matrix-free
-  saddle-point solver methodology, suitable for high-order discretizations and
-  for GPU acceleration. Examples illustrating the solution of Darcy and grad-div
-  problems are included. See miniapps/hdiv-linear-solver.
-
- Added new Discontinuous Petrov-Galerkin (DPG) miniapp which includes serial
-  and parallel examples for diffusion, convection-diffusion, acoustics and
-  Maxwell equations. The miniapp includes new classes such as (Par)DPGWeakForm,
-  (Par)ComplexDPGWeakForm and (Complex)BlockStaticCondensation. Three new
-  integrators are added in support of DPG systems: TraceIntegrator,
-  NormalTraceIntegrator and TangentTraceIntegrator. See miniapps/dpg.
-
- Added a new miniapp that implements the SPDE method for generating Gaussian
-  random fields of Matern covariance. The resulting random field can be used,
-  e.g., to model material uncertainties. See miniapps/spde.
-
- Added a new parallel LOR transfer miniapp, plor-transfer, which mirrors the
-  functionality of the serial LOR transfer miniapp. See miniapps/tools.
-
- New serial miniapp, nodal-transfer, demonstrating the use of KDTree to map a
-  parallel grid function to a different parallel partitioning of the same mesh.
-
- Added 3 additional TMOP miniapps in miniapps/meshing:
-  * Mesh-Quality evaluates quality using size, skewness, and aspect-ratio
-    computed from the Jacobian of the transformation.
-  * Mesh-Fitting can be used for interface and boundary fitting to implicit
-    domains defined using level-set functions.
-  * Fit-Node-Position fits selected mesh nodes to specified positions, while
-    maintaining overall mesh quality.
-
- Added 4 new example codes:
-  * Example 34/34p solves a simple magnetostatic problem where source terms and
-    boundary conditions are transferred with SubMesh objects.
-  * Example 35p implements H1, H(curl) and H(div) variants of a damped harmonic
-    oscillator with field transfer using SubMesh objects.
-  * Example 36/36p demonstrates the solution of the obstacle problem with a new
-    finite element method (proximal Galerkin).
-  * Example 37/37p demonstrates topology optimization with MFEM.
-
- Added a random refinement option to the mesh-explorer miniapp to assist users
-  in experimenting with nonconforming meshes.
-
- Moved the distance solver methods from miniapps/shifted to miniapps/common.
-
-Miscellaneous
-------------
- Improved lambda body debugging with the addition of mfem::forall functions.
-  These functions can take the place of the MFEM_FORALL macros, which have been
-  preserved for backwards compatibility.
-
- Added an address sanitizer GitHub action for a serial build/test on Ubuntu,
-  based on Clang/LLVM (https://clang.llvm.org/docs/AddressSanitizer.html).
-
- Reorganized files for bilinear form, linear form, and nonlinear form integrators
-  in the fem/integ/ subdirectory.
-
- FiniteElementSpace::GetFE has been updated to abort instead of returning NULL for
-  an empty partition.
-
- Various other simplifications, extensions, and bugfixes in the code.
-

 Version 4.5.2, released on March 23, 2023
 =========================================
@@ -57,7 +57,7 @@ project(mfem NONE)
 # Current version of MFEM, see also `makefile`.
 #   mfem_VERSION = (string)
 #   MFEM_VERSION = (int)   [automatically derived from mfem_VERSION]
-set(${PROJECT_NAME}_VERSION 4.6.1)
+set(${PROJECT_NAME}_VERSION 4.5.3)

 # Prohibit in-source build
 if (${PROJECT_SOURCE_DIR} STREQUAL ${PROJECT_BINARY_DIR})
@@ -82,7 +82,7 @@ if (MFEM_USE_CONDUIT OR
   #  * find_package(PETSc REQUIRED)
   set(XSDK_ENABLE_C ON)
 endif()
-if (MFEM_USE_STRUMPACK OR MFEM_USE_MUMPS)
+if (MFEM_USE_STRUMPACK)
  # Just needed to find the MPI_Fortran libraries to link with
  set(XSDK_ENABLE_Fortran ON)
 endif()
@@ -138,7 +138,7 @@ if (MFEM_USE_CUDA)
      set(CUDA_FLAGS "-ccbin=${CMAKE_CXX_COMPILER} ${CUDA_FLAGS}")
      set(CMAKE_CUDA_HOST_LINK_LAUNCHER ${CMAKE_CXX_COMPILER})
   endif()
-   set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} ${CUDA_FLAGS}")
+   set(CMAKE_CUDA_FLAGS ${CMAKE_CUDA_FLAGS} ${CUDA_FLAGS})
   set(CUSPARSE_FOUND TRUE)
   set(CUSPARSE_LIBRARIES "cusparse")
   set(CUBLAS_FOUND TRUE)
@@ -317,9 +317,6 @@ if (MFEM_USE_SUNDIALS)
  if (MFEM_USE_CUDA)
    list(APPEND SUNDIALS_COMPONENTS NVector_Cuda)
  endif()
-  if (MFEM_USE_HIP)
-    list(APPEND SUNDIALS_COMPONENTS NVector_Hip)
-  endif()
  find_package(SUNDIALS REQUIRED ${SUNDIALS_COMPONENTS})
 endif()

@@ -336,7 +333,6 @@ endif()
 if (MFEM_USE_MUMPS)
  if (MFEM_USE_MPI)
    find_package(MUMPS REQUIRED mumps_common pord)
-    set(MFEM_MUMPS_VERSION ${MUMPS_VERSION})
  else()
    message(FATAL_ERROR " *** MUMPS requires that MPI be enabled.")
  endif()
@@ -470,18 +466,12 @@ if (MFEM_USE_ADIOS2)
   find_package(ADIOS2 REQUIRED)
 endif()

-# MKL CPardiso
 if (MFEM_USE_MKL_CPARDISO)
  if (MFEM_USE_MPI)
   find_package(MKL_CPARDISO REQUIRED MKL_SEQUENTIAL MKL_LP64 MKL_MPI_WRAPPER)
  endif()
 endif()

-# MKL Pardiso
-if (MFEM_USE_MKL_PARDISO)
-   find_package(MKL_PARDISO REQUIRED MKL_SEQUENTIAL MKL_LP64)
-endif()
-
 # PARELAG
 if (MFEM_USE_PARELAG)
   find_package(PARELAG REQUIRED)
@@ -531,8 +521,8 @@ find_package(Threads REQUIRED)
 set(MFEM_TPLS OPENMP HYPRE LAPACK BLAS SuperLUDist STRUMPACK METIS SuiteSparse
    SUNDIALS PETSC SLEPC MUMPS AXOM FMS CONDUIT Ginkgo GNUTLS GSLIB
    NETCDF MPFR PUMI HIOP POSIXCLOCKS MFEMBacktrace ZLIB OCCA CEED RAJA UMPIRE
-    ADIOS2 CUBLAS CUSPARSE MKL_CPARDISO MKL_PARDISO AMGX CALIPER CODIPACK
-    BENCHMARK PARELAG MPI_CXX HIP HIPSPARSE MOONOLITH BLITZ ALGOIM ENZYME)
+    ADIOS2 CUBLAS CUSPARSE MKL_CPARDISO AMGX CALIPER CODIPACK BENCHMARK PARELAG
+    MPI_CXX HIP HIPSPARSE MOONOLITH BLITZ ALGOIM ENZYME)

 # Add all *_FOUND libraries in the variable TPL_LIBRARIES.
 set(TPL_LIBRARIES "")
@@ -121,7 +121,6 @@ The MFEM source code has the following structure:
  ├── fem
  │   ├── ceed
  │   ├── fe
-  │   ├── integ
  │   ├── lor
  │   ├── moonolith
  │   ├── qinterp
@@ -135,10 +134,8 @@ The MFEM source code has the following structure:
  │   ├── adjoint
  │   ├── autodiff
  │   ├── common
-  │   ├── dpg
  │   ├── electromagnetics
  │   ├── gslib
-  │   ├── hdiv-linear-solver
  │   ├── hooke
  │   ├── meshing
  │   ├── mtop
@@ -149,7 +146,6 @@ The MFEM source code has the following structure:
  │   ├── performance
  │   ├── shifted
  │   ├── solvers
-  │   ├── spde
  │   ├── tools
  │   └── toys
  └── tests
@@ -213,7 +209,7 @@ device/host memory manager.
 - The main device-relevant classes and sources are:
  + [`Device`](https://docs.mfem.org/html/device_8hpp.html)
  + [`MemoryManager`](https://docs.mfem.org/html/mem_manager_8hpp.html)
-  + the [`mfem::forall`](https://docs.mfem.org/html/forall_8hpp.html) function
+  + the [`MFEM_FORALL`](https://docs.mfem.org/html/forall_8hpp.html) macro
  + the [`cuda.hpp`](https://docs.mfem.org/html/cuda_8hpp.html) and [`occa.hpp`](https://docs.mfem.org/html/occa_8hpp.html) files

 #### Utilities, building and documentation
@@ -628,13 +628,9 @@ The specific libraries and their options are:
  both MPI and hypre.
  If MFEM_USE_CUDA is enabled, we expect that SUNDIALS is built with support
  for CUDA.
-  If MFEM_USE_HIP is enabled, we expect that SUNDIALS is built with support
-  for HIP.
-  URL: http://computing.llnl.gov/projects/sundials/sundials-software
+  URL: http://computation.llnl.gov/projects/sundials/sundials-software
  Options: SUNDIALS_OPT, SUNDIALS_LIB.
-  Versions: SUNDIALS >= 5.0.0,
-            SUNDIALS >= 5.4.0 for CUDA support, and
-            SUNDIALS >= 5.7.0 for HIP support.
+  Versions: SUNDIALS >= 5.0.0, SUNDIALS >= 5.4.0 for CUDA support.

 - SuiteSparse (optional), used when MFEM_USE_SUITESPARSE = YES.
  URL: http://faculty.cse.tamu.edu/davis/suitesparse.html
@@ -699,15 +695,12 @@ The specific libraries and their options are:
  PETSc has been cloned on the same level as mfem and hypre:
    ./configure --download-fblaslapack=yes --download-scalapack=yes \
                --download-mumps=yes --download-suitesparse=yes \
-                --with-hypre-dir=../hypre/src/hypre \
+                --with-hypre-dir=../hypre-2.10.0b/src/hypre \
                --with-shared-libraries=0
-  When building PETSc with HIP, one may need to add a flag like -std=c2x to
-  CFLAGS to allow proper parsing of the hipsparse header under C.
  URL: https://www.mcs.anl.gov/petsc
  Options: PETSC_OPT, PETSC_LIB.
-  Versions: PETSc >= 3.8.0   (PETSc build without CUDA/HIP)
+  Versions: PETSc >= 3.8.0   (PETSc build without CUDA)
            PETSc >= 3.15.0  (PETSc built with CUDA)
-            PETSc >= 3.19.0  (PETSc built with HIP, older versions may work too)

 - SLEPc (optional), used when MFEM_USE_SLEPC = YES. SLEPc depends on PETSc and
  uses some of the PETSc options when compiled.
@@ -55,8 +55,6 @@ set(MFEM_USE_SIMD @MFEM_USE_SIMD@)
 set(MFEM_USE_ADIOS2 @MFEM_USE_ADIOS2@)
 set(MFEM_USE_MOONOLITH @MFEM_USE_MOONOLITH@)
 set(MFEM_USE_CODIPACK @MFEM_USE_CODIPACK@)
-set(MFEM_USE_MKL_CPARDISO @MFEM_USE_MKL_CPARDISO@)
-set(MFEM_USE_MKL_PARDISO @MFEM_USE_MKL_PARDISO@)
 set(MFEM_USE_ADFORWARD @MFEM_USE_ADFORWARD@)
 set(MFEM_USE_CALIPER @MFEM_USE_CALIPER@)
 set(MFEM_USE_ALGOIM @MFEM_USE_ALGOIM@)
@@ -80,102 +80,97 @@
 // Internal MFEM option: enable group/batch allocation for some small objects.
 #cmakedefine MFEM_USE_MEMALLOC

-// Which library functions to use in class StopWatch for measuring time.
-// For a list of the available options, see INSTALL.
-// If not defined, an option is selected automatically.
-#cmakedefine MFEM_TIMER_TYPE @MFEM_TIMER_TYPE@
-
-// Enable MFEM functionality based on the SUNDIALS libraries.
-#cmakedefine MFEM_USE_SUNDIALS
-
 // Enable MFEM functionality based on the SuiteSparse library.
 #cmakedefine MFEM_USE_SUITESPARSE

 // Enable MFEM functionality based on the SuperLU_DIST library.
 #cmakedefine MFEM_USE_SUPERLU
-#cmakedefine MFEM_USE_SUPERLU5

 // Enable MFEM functionality based on the MUMPS library.
 #cmakedefine MFEM_USE_MUMPS
-#cmakedefine MFEM_MUMPS_VERSION @MFEM_MUMPS_VERSION@

 // Enable MFEM functionality based on the STRUMPACK library.
 #cmakedefine MFEM_USE_STRUMPACK

-// Enable functionality based on the Ginkgo library.
+// Enable functionality based on the Ginkgo library
 #cmakedefine MFEM_USE_GINKGO

-// Enable MFEM functionality based on the AmgX library.
+// Enable MFEM functionality based on the AmgX library
 #cmakedefine MFEM_USE_AMGX

-// Enable secure socket streams based on the GNUTLS library.
+// Enable MFEM functionality based on the GnuTLS library
 #cmakedefine MFEM_USE_GNUTLS

-// Enable Sidre support.
-#cmakedefine MFEM_USE_SIDRE
-
-// Enable the use of SIMD in the high performance templated classes.
-#cmakedefine MFEM_USE_SIMD
-
-// Enable FMS support.
-#cmakedefine MFEM_USE_FMS
-
-// Enable Conduit support.
-#cmakedefine MFEM_USE_CONDUIT
-
-// Enable functionality based on the NetCDF library (reading CUBIT files).
-#cmakedefine MFEM_USE_NETCDF
-
-// Enable functionality based on the PETSc library.
-#cmakedefine MFEM_USE_PETSC
-
-// Enable functionality based on the SLEPc library.
-#cmakedefine MFEM_USE_SLEPC
-
-// Enable functionality based on the MPFR library.
-#cmakedefine MFEM_USE_MPFR
-
-// Enable MFEM functionality based on the PUMI library.
-#cmakedefine MFEM_USE_PUMI
-
-// Enable Moonolith-based general interpolation between finite element spaces.
-#cmakedefine MFEM_USE_MOONOLITH
-
-// Enable MFEM functionality based on the HIOP library.
-#cmakedefine MFEM_USE_HIOP
-
-// Enable MFEM functionality based on the GSLIB library.
+// Enable MFEM functionality based on the GSLIB library
 #cmakedefine MFEM_USE_GSLIB

-// Build the NVIDIA GPU/CUDA-enabled version of the MFEM library.
+// Enable MFEM functionality based on the NetCDF library
+#cmakedefine MFEM_USE_NETCDF
+
+// Enable MFEM functionality based on the PETSc library
+#cmakedefine MFEM_USE_PETSC
+
+// Enable MFEM functionality based on the SLEPc library
+#cmakedefine MFEM_USE_SLEPC
+
+// Enable MFEM functionality based on the Sidre library
+#cmakedefine MFEM_USE_SIDRE
+
+// Enable the use of SIMD in the high performance templated classes
+#cmakedefine MFEM_USE_SIMD
+
+// Enable MFEM functionality based on the FMS library
+#cmakedefine MFEM_USE_FMS
+
+// Enable MFEM functionality based on Conduit
+#cmakedefine MFEM_USE_CONDUIT
+
+// Enable MFEM functionality based on the PUMI library
+#cmakedefine MFEM_USE_PUMI
+
+// Enable MFEM functionality based on the Moonolith library
+#cmakedefine MFEM_USE_MOONOLITH
+
+// Enable MFEM functionality based on the HiOp library
+#cmakedefine MFEM_USE_HIOP
+
+// Build the GPU/CUDA-enabled version of the MFEM library.
 // Requires a CUDA compiler (nvcc).
 #cmakedefine MFEM_USE_CUDA

-// Build the AMD GPU/HIP-enabled version of the MFEM library.
+// Build the HIP-enabled version of the MFEM library.
 // Requires a HIP compiler (hipcc).
 #cmakedefine MFEM_USE_HIP

-// Enable functionality based on the RAJA library.
+// Enable MFEM functionality based on the RAJA library
 #cmakedefine MFEM_USE_RAJA

-// Enable functionality based on the OCCA library.
+// Enable MFEM functionality based on the OCCA library
 #cmakedefine MFEM_USE_OCCA

-// Enable functionality based on the libCEED library.
+// Enable MFEM functionality based on the libCEED library
 #cmakedefine MFEM_USE_CEED

-// Enable functionality based on the Caliper library.
-#cmakedefine MFEM_USE_CALIPER
-
-// Enable functionality based on the Algoim library.
-#cmakedefine MFEM_USE_ALGOIM
-
-// Enable functionality based on the Umpire library.
+// Enable MFEM functionality based on the Umpire library
 #cmakedefine MFEM_USE_UMPIRE

-// Enable IO functionality based on the ADIOS2 library.
+// Enable MFEM functionality based on the ADIOS2 library
 #cmakedefine MFEM_USE_ADIOS2

+// Enable MFEM functionality based on the Caliper library
+#cmakedefine MFEM_USE_CALIPER
+
+// Enable MFEM functionality based on the Algoim library
+#cmakedefine MFEM_USE_ALGOIM
+
+// Which library functions to use in class StopWatch for measuring time.
+// For a list of the available options, see INSTALL.
+// If not defined, an option is selected automatically.
+#define MFEM_TIMER_TYPE @MFEM_TIMER_TYPE@
+
+// Enable MFEM functionality based on the SUNDIALS libraries.
+#cmakedefine MFEM_USE_SUNDIALS
+
 // Version of HYPRE used for building MFEM.
 #cmakedefine MFEM_HYPRE_VERSION @MFEM_HYPRE_VERSION@

@@ -186,16 +181,13 @@
 // Enable interface to the MKL CPardiso library.
 #cmakedefine MFEM_USE_MKL_CPARDISO

-// Enable interface to the MKL Pardiso library.
-#cmakedefine MFEM_USE_MKL_PARDISO
-
-// Use forward mode for automatic differentiation.
+// Use forward mode for automatic differentiation
 #cmakedefine MFEM_USE_ADFORWARD

-// Enable the use of the CoDiPack library for AD.
+// Enable the use of the CoDiPack library for AD
 #cmakedefine MFEM_USE_CODIPACK

-// Enable functionality based on the Google Benchmark library.
+// Enable MFEM functionality based on the Google Benchmark library.
 #cmakedefine MFEM_USE_BENCHMARK

 // Enable Enzyme for AD
@@ -1,27 +0,0 @@
-# Copyright (c) 2010-2023, Lawrence Livermore National Security, LLC. Produced
-# at the Lawrence Livermore National Laboratory. All Rights reserved. See files
-# LICENSE and NOTICE for details. LLNL-CODE-806117.
-#
-# This file is part of the MFEM library. For more information and source code
-# availability visit https://mfem.org.
-#
-# MFEM is free software; you can redistribute it and/or modify it under the
-# terms of the BSD-3 license. We welcome feedback and contributions, see file
-# CONTRIBUTING.md for details.
-
-# Defines the following variables:
-#   - MKL_PARDISO_FOUND
-#   - MKL_PARDISO_LIBRARIES
-#   - MKL_PARDISO_INCLUDE_DIRS
-
-if(NOT MKL_LIBRARY_DIR)
-  message(WARNING "Using default MKL library path. Double check the variable MKL_LIBRARY_DIR")
-  set(MKL_LIBRARY_DIR "lib/intel64")
-endif()
-
-include(MfemCmakeUtilities)
-mfem_find_package(MKL_PARDISO MKL_PARDISO
-    MKL_PARDISO_DIR "include" mkl_pardiso.h ${MKL_LIBRARY_DIR} mkl_core
-  "Paths to headers required by MKL Pardiso." "Libraries required by MKL PARDISO."
-  ADD_COMPONENT MKL_LP64 "include" "" ${MKL_LIBRARY_DIR} mkl_intel_lp64
-  ADD_COMPONENT MKL_SEQUENTIAL "include" "" ${MKL_LIBRARY_DIR} mkl_sequential)
@@ -11,9 +11,8 @@

 # Sets the following variables:
 #   - MUMPS_FOUND
-#   - MUMPS_LIBRARIES
 #   - MUMPS_INCLUDE_DIRS
-#   - MUMPS_VERSION
+#   - MUMPS_LIBRARIES

 include(MfemCmakeUtilities)
 mfem_find_package(MUMPS MUMPS MUMPS_DIR
@@ -22,18 +21,3 @@ mfem_find_package(MUMPS MUMPS MUMPS_DIR
  "Libraries required by MUMPS."
  ADD_COMPONENT mumps_common "include" dmumps_c.h "lib" mumps_common
  ADD_COMPONENT pord "include" dmumps_c.h "lib" pord)
-
-if (MUMPS_FOUND AND (NOT MUMPS_VERSION))
-  try_run(MUMPS_VERSION_RUN_RESULT MUMPS_VERSION_COMPILE_RESULT
-          ${CMAKE_CURRENT_BINARY_DIR}/config
-          ${CMAKE_CURRENT_SOURCE_DIR}/config/get_mumps_version.cpp
-          CMAKE_FLAGS -DINCLUDE_DIRECTORIES:STRING=${MUMPS_INCLUDE_DIRS}
-          RUN_OUTPUT_VARIABLE MUMPS_VERSION_OUTPUT)
-  if ((MUMPS_VERSION_RUN_RESULT EQUAL 0) AND MUMPS_VERSION_OUTPUT)
-    string(STRIP "${MUMPS_VERSION_OUTPUT}" MUMPS_VERSION)
-    set(MUMPS_VERSION ${MUMPS_VERSION} CACHE STRING "MUMPS version." FORCE)
-    message(STATUS "Found MUMPS version ${MUMPS_VERSION}")
-  else()
-    message(FATAL_ERROR "Unable to determine MUMPS version.")
-  endif()
-endif()
@@ -22,8 +22,8 @@ mfem_find_package(SUNDIALS SUNDIALS SUNDIALS_DIR
    "include" nvector/nvector_serial.h "lib" sundials_nvecserial
  ADD_COMPONENT NVector_Cuda
    "include" nvector/nvector_cuda.h "lib" sundials_nveccuda
-  ADD_COMPONENT NVector_Hip
-    "include" nvector/nvector_hip.h "lib" sundials_nvechip
+  ADD_COMPONENT NVector_ParHyp
+    "include" nvector/nvector_parhyp.h "lib" sundials_nvecparhyp
  ADD_COMPONENT NVector_Parallel
    "include" nvector/nvector_parallel.h "lib" sundials_nvecparallel
  ADD_COMPONENT NVector_MPIPlusX
@@ -30,10 +30,10 @@
 #define MFEM_VERSION_MINOR (((MFEM_VERSION)/100)%100)
 #define MFEM_VERSION_PATCH ((MFEM_VERSION)%100)

-// The absolute path of the MFEM source prefix.
+// The absolute path of the MFEM source prefix
 // #define MFEM_SOURCE_DIR "@MFEM_SOURCE_DIR@"

-// The absolute path of the MFEM installation prefix.
+// The absolute path of the MFEM installation prefix
 // #define MFEM_INSTALL_DIR "@MFEM_INSTALL_DIR@"

 // Description of the git commit used to build MFEM.
@@ -91,7 +91,7 @@
 // Enable MFEM functionality based on the SuiteSparse library.
 // #define MFEM_USE_SUITESPARSE

-// Enable MFEM functionality based on the SuperLU_DIST library.
+// Enable MFEM functionality based on the SuperLU library.
 // #define MFEM_USE_SUPERLU
 // #define MFEM_USE_SUPERLU5

@@ -102,40 +102,40 @@
 // Enable MFEM functionality based on the STRUMPACK library.
 // #define MFEM_USE_STRUMPACK

-// Enable MFEM features based on the Ginkgo library.
+// Enable MFEM features based on the Ginkgo library
 // #define MFEM_USE_GINKGO

 // Enable MFEM functionality based on the AmgX library.
 // #define MFEM_USE_AMGX

-// Enable secure socket streams based on the GNUTLS library.
+// Enable secure socket streams based on the GNUTLS library
 // #define MFEM_USE_GNUTLS

-// Enable Sidre support.
+// Enable Sidre support
 // #define MFEM_USE_SIDRE

-// Enable the use of SIMD in the high performance templated classes.
+// Enable the use of SIMD in the high performance templated classes
 // #define MFEM_USE_SIMD

-// Enable FMS support.
+// Enable FMS support
 // #define MFEM_USE_FMS

-// Enable Conduit support.
+// Enable Conduit support
 // #define MFEM_USE_CONDUIT

-// Enable functionality based on the NetCDF library (reading CUBIT files).
+// Enable functionality based on the NetCDF library (reading CUBIT files)
 // #define MFEM_USE_NETCDF

-// Enable functionality based on the PETSc library.
+// Enable functionality based on the PETSc library
 // #define MFEM_USE_PETSC

-// Enable functionality based on the SLEPc library.
+// Enable functionality based on the SLEPc library
 // #define MFEM_USE_SLEPC

 // Enable functionality based on the MPFR library.
 // #define MFEM_USE_MPFR

-// Enable MFEM functionality based on the PUMI library.
+// Enable MFEM functionality based on the PUMI library
 // #define MFEM_USE_PUMI

 // Enable Moonolith-based general interpolation between finite element spaces.
@@ -144,7 +144,7 @@
 // Enable MFEM functionality based on the HIOP library.
 // #define MFEM_USE_HIOP

-// Enable MFEM functionality based on the GSLIB library.
+// Enable MFEM functionality based on the GSLIB library
 // #define MFEM_USE_GSLIB

 // Build the NVIDIA GPU/CUDA-enabled version of the MFEM library.
@@ -186,13 +186,10 @@
 // Enable interface to the MKL CPardiso library.
 // #define MFEM_USE_MKL_CPARDISO

-// Enable interface to the MKL Pardiso library.
-// #define MFEM_USE_MKL_PARDISO
-
-// Use forward mode for automatic differentiation.
+// Use forward mode for automatic differentiation
 // #define MFEM_USE_ADFORWARD

-// Enable the use of the CoDiPack library for AD.
+// Enable the use of the CoDiPack library for AD
 // #define MFEM_USE_CODIPACK

 // Enable functionality based on the Google Benchmark library.
@@ -57,7 +57,6 @@ MFEM_USE_UMPIRE        = @MFEM_USE_UMPIRE@
 MFEM_USE_SIMD          = @MFEM_USE_SIMD@
 MFEM_USE_ADIOS2        = @MFEM_USE_ADIOS2@
 MFEM_USE_MKL_CPARDISO  = @MFEM_USE_MKL_CPARDISO@
-MFEM_USE_MKL_PARDISO   = @MFEM_USE_MKL_PARDISO@
 MFEM_USE_MOONOLITH     = @MFEM_USE_MOONOLITH@
 MFEM_USE_ADFORWARD     = @MFEM_USE_ADFORWARD@
 MFEM_USE_CODIPACK      = @MFEM_USE_CODIPACK@
@@ -60,7 +60,6 @@ option(MFEM_USE_ADIOS2 "Enable ADIOS2" OFF)
 option(MFEM_USE_CALIPER "Enable Caliper support" OFF)
 option(MFEM_USE_ALGOIM "Enable Algoim support" OFF)
 option(MFEM_USE_MKL_CPARDISO "Enable MKL CPardiso" OFF)
-option(MFEM_USE_MKL_PARDISO "Enable MKL Pardiso" OFF)
 option(MFEM_USE_ADFORWARD "Enable forward mode for AD" OFF)
 option(MFEM_USE_CODIPACK "Enable automatic differentiation (AD) using CoDiPack" OFF)
 option(MFEM_USE_BENCHMARK "Enable Google Benchmark" OFF)
@@ -135,18 +134,16 @@ set(ParMETIS_DIR "${MFEM_DIR}/../parmetis-4.0.3" CACHE PATH
 set(ParMETIS_REQUIRED_PACKAGES "METIS" CACHE STRING
    "Additional packages required by ParMETIS.")

-set(SuperLUDist_DIR "${MFEM_DIR}/../SuperLU_DIST_8.1.2" CACHE PATH
+set(SuperLUDist_DIR "${MFEM_DIR}/../SuperLU_DIST_6.3.1" CACHE PATH
    "Path to the SuperLU_DIST library.")
 # SuperLU_DIST may also depend on "OpenMP", depending on how it was compiled.
-set(SuperLUDist_REQUIRED_PACKAGES "MPI" "ParMETIS" "METIS"
-    "LAPACK" "BLAS" CACHE STRING
+set(SuperLUDist_REQUIRED_PACKAGES "MPI" "BLAS" "ParMETIS" CACHE STRING
    "Additional packages required by SuperLU_DIST.")

-set(MUMPS_DIR "${MFEM_DIR}/../MUMPS_5.5.0" CACHE PATH
+set(MUMPS_DIR "${MFEM_DIR}/../MUMPS_5.2.0" CACHE PATH
    "Path to the MUMPS library.")
-# MUMPS may also depend on "OpenMP", depending on how it was compiled.
-set(MUMPS_REQUIRED_PACKAGES "MPI" "MPI_Fortran" "ParMETIS" "METIS"
-    "ScaLAPACK" "LAPACK" "BLAS" CACHE STRING
+# Packages required by MUMPS, depending on how it was compiled.
+set(MUMPS_REQUIRED_PACKAGES "MPI" "BLAS" "METIS" "ScaLAPACK" CACHE STRING
    "Additional packages required by MUMPS.")
 # If the MPI package does not find all required Fortran libraries:
 # set(MUMPS_REQUIRED_LIBRARIES "gfortran" "mpi_mpifh" CACHE STRING
@@ -229,8 +226,6 @@ set(MKL_CPARDISO_DIR "" CACHE STRING "MKL installation path.")
 set(MKL_MPI_WRAPPER_LIB "mkl_blacs_mpich_lp64" CACHE STRING "MKL MPI wrapper library")
 set(MKL_LIBRARY_DIR "" CACHE STRING "Custom library subdirectory")

-set(MKL_PARDISO_DIR "" CACHE STRING "MKL installation path.")
-
 set(OCCA_DIR "${MFEM_DIR}/../occa" CACHE PATH "Path to OCCA")
 set(RAJA_DIR "${MFEM_DIR}/../raja" CACHE PATH "Path to RAJA")
 set(CEED_DIR "${MFEM_DIR}/../libCEED" CACHE PATH "Path to libCEED")
@@ -160,7 +160,6 @@ MFEM_USE_UMPIRE        = NO
 MFEM_USE_SIMD          = NO
 MFEM_USE_ADIOS2        = NO
 MFEM_USE_MKL_CPARDISO  = NO
-MFEM_USE_MKL_PARDISO   = NO
 MFEM_USE_MOONOLITH     = NO
 MFEM_USE_ADFORWARD     = NO
 MFEM_USE_CODIPACK      = NO
@@ -267,9 +266,6 @@ endif
 ifeq ($(MFEM_USE_CUDA),YES)
   SUNDIALS_LIB += -lsundials_nveccuda
 endif
-ifeq ($(MFEM_USE_HIP),YES)
-   SUNDIALS_LIB += -lsundials_nvechip
-endif
 # If SUNDIALS was built with KLU:
 # MFEM_USE_SUITESPARSE = YES

@@ -288,10 +284,10 @@ ifeq ($(MFEM_USE_SUPERLU5),YES)
   SUPERLU_LIB = $(XLINKER)-rpath,$(SUPERLU_DIR)/lib -L$(SUPERLU_DIR)/lib\
      -lsuperlu_dist_5.1.0
 else
-   SUPERLU_DIR = @MFEM_DIR@/../SuperLU_DIST_8.1.2
+   SUPERLU_DIR = @MFEM_DIR@/../SuperLU_DIST_6.3.1
   SUPERLU_OPT = -I$(SUPERLU_DIR)/include
   SUPERLU_LIB = $(XLINKER)-rpath,$(SUPERLU_DIR)/lib64 -L$(SUPERLU_DIR)/lib64\
-      -lsuperlu_dist $(LAPACK_LIB)
+      -lsuperlu_dist -lblas
 endif

 # SCOTCH library configuration (required by STRUMPACK <= v2.1.0, optional in
@@ -315,7 +311,7 @@ MPI_FORTRAN_LIB = -lmpifort
 # MPI_FORTRAN_LIB += -lgfortran

 # MUMPS library configuration
-MUMPS_DIR = @MFEM_DIR@/../MUMPS_5.5.0
+MUMPS_DIR = @MFEM_DIR@/../MUMPS_5.2.0
 MUMPS_OPT = -I$(MUMPS_DIR)/include
 MUMPS_LIB = $(XLINKER)-rpath,$(MUMPS_DIR)/lib -L$(MUMPS_DIR)/lib -ldmumps\
 -lmumps_common -lpord $(SCALAPACK_LIB) $(LAPACK_LIB) $(MPI_FORTRAN_LIB)
@@ -331,30 +327,16 @@ STRUMPACK_OPT = -I$(STRUMPACK_DIR)/include $(SCOTCH_OPT)
 STRUMPACK_LIB = -L$(STRUMPACK_DIR)/lib -lstrumpack $(MPI_FORTRAN_LIB)\
 $(SCOTCH_LIB) $(SCALAPACK_LIB)

-# Ginkgo library configuration
+# Ginkgo library configuration (currently not needed)
 GINKGO_DIR = @MFEM_DIR@/../ginkgo/install
-GINKGO_SEARCH_DIR = $(subst @MFEM_DIR@,$(MFEM_DIR),$(GINKGO_DIR))
 GINKGO_BUILD_TYPE=Release
 ifeq ($(MFEM_USE_GINKGO),YES)
   BASE_FLAGS = -std=c++14
 endif
 GINKGO_OPT = -isystem $(GINKGO_DIR)/include
-GINKGO_LIB_DIR = $(sort $(dir $(wildcard\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.a\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.so\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.dylib\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.dll)))
-GINKGO_LINK_LIB_DIR = $(GINKGO_DIR)$(subst $(GINKGO_SEARCH_DIR),,$(GINKGO_LIB_DIR))
-ALL_GINKGO_LIBS_DEBUG = $(notdir $(basename $(wildcard\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*d.a\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*d.so\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*d.dylib\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*d.dll)))
-ALL_GINKGO_LIBS = $(notdir $(basename $(wildcard\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.a\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.so\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.dylib\
- $(GINKGO_SEARCH_DIR)/lib*/libginkgo*.dll)))
+GINKGO_LIB_DIR = $(sort $(dir $(wildcard $(GINKGO_DIR)/lib*/libginkgo*.a  $(GINKGO_DIR)/lib*/libginkgo*.so $(GINKGO_DIR)/lib*/libginkgo*.dylib $(GINKGO_DIR)/lib*/libginkgo*.dll)))
+ALL_GINKGO_LIBS_DEBUG = $(notdir $(basename $(wildcard $(GINKGO_DIR)/lib*/libginkgo*d.a  $(GINKGO_DIR)/lib*/libginkgo*d.so $(GINKGO_DIR)/lib*/libginkgo*d.dylib $(GINKGO_DIR)/lib*/libginkgo*d.dll)))
+ALL_GINKGO_LIBS = $(notdir $(basename $(wildcard $(GINKGO_DIR)/lib*/libginkgo*.a  $(GINKGO_DIR)/lib*/libginkgo*.so $(GINKGO_DIR)/lib*/libginkgo*.dylib $(GINKGO_DIR)/lib*/libginkgo*.dll)))
 ALL_GINKGO_LIBS_RELEASE = $(filter-out $(ALL_GINKGO_LIBS_DEBUG),$(ALL_GINKGO_LIBS))
 GINKGO_LINK = $(subst libginkgo,-lginkgo,$(ALL_GINKGO_LIBS_RELEASE))
 ifeq ($(GINKGO_BUILD_TYPE),Debug)
@@ -363,8 +345,7 @@ ifeq ($(GINKGO_BUILD_TYPE),Debug)
  endif
 else
 endif
-GINKGO_LIB = $(XLINKER)-rpath,$(GINKGO_LINK_LIB_DIR) -L$(GINKGO_LINK_LIB_DIR)\
- $(GINKGO_LINK)
+GINKGO_LIB = $(XLINKER)-rpath,$(GINKGO_LIB_DIR) -L$(GINKGO_LIB_DIR) $(GINKGO_LINK)

 # AmgX library configuration
 AMGX_DIR = @MFEM_DIR@/../amgx
@@ -503,6 +484,7 @@ ifdef GOTCHA_DIR
   CALIPER_LIB += $(XLINKER)-rpath,$(GOTCHA_DIR)/lib64 $(XLINKER)-rpath,$(GOTCHA_DIR)/lib -L$(GOTCHA_DIR)/lib64 -L$(GOTCHA_DIR)/lib -lgotcha
 endif

+
 # BLITZ library configuration
 BLITZ_DIR = @MFEM_DIR@/../blitz
 BLITZ_OPT = -I$(BLITZ_DIR)/include
@@ -557,14 +539,6 @@ MKL_CPARDISO_LIB = $(XLINKER)-rpath,$(MKL_CPARDISO_DIR)/$(MKL_LIBRARY_SUBDIR)\
   -L$(MKL_CPARDISO_DIR)/$(MKL_LIBRARY_SUBDIR) -l$(MKL_MPI_WRAPPER)\
   -lmkl_intel_lp64 -lmkl_sequential -lmkl_core

-# MKL Pardiso library configuration
-MKL_PARDISO_DIR ?=
-MKL_LIBRARY_SUBDIR ?= lib
-MKL_PARDISO_OPT = -I$(MKL_PARDISO_DIR)/include
-MKL_PARDISO_LIB = $(XLINKER)-rpath,$(MKL_PARDISO_DIR)/$(MKL_LIBRARY_SUBDIR)\
-   -L$(MKL_PARDISO_DIR)/$(MKL_LIBRARY_SUBDIR)\
-   -lmkl_intel_lp64 -lmkl_sequential -lmkl_core
-
 # PARELAG library configuration
 PARELAG_DIR = @MFEM_DIR@/../parelag
 PARELAG_OPT = -I$(PARELAG_DIR)/src -I$(PARELAG_DIR)/build/src
@@ -19,7 +19,9 @@ RUN apt-get update && \
    apt-get install -y libcurl4-openssl-dev libssl-dev

 ENV PATH=$PATH:/opt/mfem-view/bin
+ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/mfem-view/lib:/opt/mfem-view/lib64
 ENV DEBIAN_FRONTEND=noninteractive

 # The user will see the view on shell into the container
 WORKDIR /opt/mfem-view
+ENTRYPOINT ["/bin/bash"]
@@ -34,14 +34,14 @@ RUN cd /opt/mfem-env && \
    . /opt/spack/share/spack/setup-env.sh && \
    spack env activate . && \
    spack develop --path /code mfem@master+examples+miniapps && \
-    spack add mfem@master+examples+miniapps && \
-    spack install
+    spack add mfem@master+examples+miniapps # && \
+#    spack install

 # ensure mfem always on various paths
-RUN cd /opt/mfem-env && \
-    spack env activate --sh -d . >> /etc/profile.d/z10_spack_environment.sh
+#RUN cd /opt/mfem-env && \
+#    spack env activate --sh -d . >> /etc/profile.d/z10_spack_environment.sh

 # Present the software install when we shell in
 # The view is at /opt/mfem-env/.spack-env/view
-WORKDIR /opt/software
-ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l", "-c"]
+#WORKDIR /opt/software
+#ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l", "-c"]
@@ -7,31 +7,21 @@ You can use this image for a demo of using mfem! 🎉️
 Updated containers are built and deployed on merges to the main branch and releases.
 If you want to request a build on demand, you can [manually run the workflow](https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow) thanks to the workflow dispatch event.

-## Usage
+### Usage

-We provide two containers, which you can either build or use directly from
-[GitHub packages](https://github.com/orgs/mfem/packages?repo_name=mfem).
-
- - `ghcr.io/mfem/mfem-ubuntu-base`: a "build from scratch" for mfem
- - `ghcr.io/mfem/mfem-ubuntu`: a quick build that uses the base container
-
-In the above, "ghcr.io" means "GitHub Container Registry" and
+Here is how to build the container. Note that we build so it belongs to the same
+namespace as the repository here. "ghcr.io" means "GitHub Container Registry" and
 is the [GitHub packages](https://github.com/features/packages) registry that supports
-Docker images and other OCI artifacts. 
-
-### Ubuntu
-
-> Use or build this container for a multi-stage, slimmer base to develop on top of mfem
-
-Note that this container is provided on GitHub packages [here](https://github.com/mfem/mfem/pkgs/container/mfem-ubuntu)
-so you don't need to build it. However, if you want to, you can do the following:
+ Docker images and other OCI artifacts. From the root of the repository:

 ```bash
 $ docker build -f config/docker/Dockerfile -t ghcr.io/mfem/mfem-ubuntu .
+$ docker build -f config/docker/Dockerfile.base -t ghcr.io/mfem/mfem-ubuntu-base .
 ```

-Note that this will pull the base image. If you want to rebuild it, see [ubuntu base](#ubuntu-base)
-below. Once you have built (or prefer to pull) you can shell into the container as follows:
+### Shell Ubuntu
+
+To shell into the container:

 ```bash
 $ docker run -it ghcr.io/mfem/mfem-ubuntu
@@ -47,13 +37,39 @@ bin  etc  include  lib  libexec  sbin  share  var
 - Examples are in share/mfem/examples
 - Examples are in share/mfem/miniapps

-Using this container, if you want to develop a tool that _uses_ mfem, you can find the libraries / includes in:
+You can read more about interaction with these examples and miniapps below.
+
+### Shell Ubuntu Base
+
+To shell into the container:

 ```bash
-$ ls include/ | grep mfem
-mfem
-mfem-performance.hpp
-mfem.hpp
+$ docker run -it ghcr.io/mfem/mfem-ubuntu-base bash
+```
+
+Off the bat, you can see mfem libraries are in your path so you can jump into development:
+
+```bash
+env | grep mfem
+```
+```bash
+PKG_CONFIG_PATH=/opt/mfem-env/.spack-env/view/lib/pkgconfig:/opt/mfem-env/.spack-env/view/share/pkgconfig:/opt/mfem-env/.spack-env/view/lib64/pkgconfig
+PWD=/opt/mfem-env
+MANPATH=/opt/mfem-env/.spack-env/view/share/man:/opt/mfem-env/.spack-env/view/man:
+CMAKE_PREFIX_PATH=/opt/mfem-env/.spack-env/view
+SPACK_ENV=/opt/mfem-env
+ACLOCAL_PATH=/opt/mfem-env/.spack-env/view/share/aclocal
+LD_LIBRARY_PATH=/opt/mfem-env/.spack-env/view/lib:/opt/mfem-env/.spack-env/view/lib64
+PATH=/opt/mfem-env/.spack-env/view/bin:/opt/view/bin:/opt/spack/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+```
+
+#### Examples and MiniApps
+
+If you want to develop a tool that _uses_ mfem, you can find the built libraries in:
+
+```
+$ ls /opt/mfem-env/.spack-env/view/
+bin  etc  include  lib  libexec  sbin  share  var
 ```

 And yes, this is the working directory when you shell into the container!
@@ -63,16 +79,6 @@ You can find the examples here:
 ```bash
 cd share/mfem/examples
 ```
-
-Try quickly setting the `LD_LIBRARY_PATH` so we can see the shared libraries
-we need:
-
-```bash
-export LD_LIBRARY_PATH=/opt/mfem-view/lib:$LD_LIBRARY_PATH
-```
-
-And then run:
-
 ```bash
 $ ./ex0
 Options used:
@@ -91,6 +97,7 @@ Number of unknowns: 101
 Average reduction factor = 0.140201
 ```

+Try running a few, and look at the associated .cpp file for the source code!
 You can also explore the "mini apps," also in share/mfem, but under miniapps.

 ```bash
@@ -123,87 +130,18 @@ Rule:
 Applying rule...done.
 ```

-Have fun! As a reminder, this container is ideal for developing your own
-applications that might use mfem, or having a nice environment to test out
-examples.
+Have fun!


-### Ubuntu Base
-
-> Use this build for a development environment with spack and mfem
-
-This container is also [provided on GitHub packages](https://github.com/mfem/mfem/pkgs/container/mfem-ubuntu-base), 
-however you can build it locally too:
+#### Your own App
+If you want to develop with your own code base
+(and mfem as is in the container) you can bind to somewhere else in the container (e.g., src)

 ```bash
-$ docker build -f config/docker/Dockerfile.base -t ghcr.io/mfem/mfem-ubuntu-base .
-```
-
-To shell into the container:
-
-```bash
-$ docker run -it ghcr.io/mfem/mfem-ubuntu-base bash
-```
-
-Change directory to the mfem environment, setup spack, and activate the environment:
-
-```bash
-source /opt/spack/share/spack/setup-env.sh
-cd /opt/mfem-env/
-spack env activate .
-```
-
-Note that this environment is installing to the view at `/opt/view`. Since the environment
-knows to install mfem from `/code` this means that you could make changes in the container (or bind
-`/code` to your container) and then update spack:
-
-```bash
-# Note that concretization takes a hot minute!
-$ spack install
-```
-
-And if you want to load mfem:
-
-```bash
-$ spack load mfem
-$ env | grep mfem
-```
-
-In this development container, you can find the examples and miniapps alongside
-mfem under `/code`:
-
-```bash
-cd /code/examples
-```
-```bash
-$ ./ex0
-```
-```console
-Options used:
-   --mesh ../data/star.mesh
-   --order 1
-Number of unknowns: 101
-   Iteration :   0  (B r, r) = 0.184259
-   Iteration :   1  (B r, r) = 0.102754
-   Iteration :   2  (B r, r) = 0.00558141
-   Iteration :   3  (B r, r) = 1.5247e-05
-   Iteration :   4  (B r, r) = 1.13807e-07
-   Iteration :   5  (B r, r) = 6.27231e-09
-   Iteration :   6  (B r, r) = 3.76268e-11
-   Iteration :   7  (B r, r) = 6.07423e-13
-   Iteration :   8  (B r, r) = 4.10615e-15
-Average reduction factor = 0.140201
-```
-
-This container is likely ideal for someone that wants to develop mfem itself.
-For other use cases, we recommend using the slimmer image. As an example,
-if you want to develop with your own code base (and mfem as is in the container) 
-you can bind to somewhere else in the container (e.g., src)
-
-```bash
-$ docker run -it ghcr.io/mfem/mfem-ubuntu-base -v $PWD:/code bash
+$ docker run -it ghcr.io/mfem/mfem-ubuntu-base -v $PWD:/src bash
 ```

 In the above, we can pretend your project is in the present working directory (PWD) and we are
 binding to source. You can then use the mfem in the container for development, and if you
 want to distribute your library or app in a container, you can use the mfem container as the base.
+
@@ -1,37 +0,0 @@
-MFEM mesh v1.0
-
-#
-# MFEM Geometry Types (see mesh/geom.hpp):
-#
-# POINT       = 0
-# SEGMENT     = 1
-# TRIANGLE    = 2
-# SQUARE      = 3
-# TETRAHEDRON = 4
-# CUBE        = 5
-# PRISM       = 6
-#
-
-dimension
-1
-
-elements
-4
-1 1 0 1
-1 1 1 2
-1 1 2 3
-1 1 3 4
-
-boundary
-2
-1 0 0
-2 0 4
-
-vertices
-5
-2
-0 0
-0.25 0.25
-0.50 0.50
-0.75 0.75
-1 1
@@ -1,37 +0,0 @@
-MFEM mesh v1.0
-
-#
-# MFEM Geometry Types (see mesh/geom.hpp):
-#
-# POINT       = 0
-# SEGMENT     = 1
-# TRIANGLE    = 2
-# SQUARE      = 3
-# TETRAHEDRON = 4
-# CUBE        = 5
-# PRISM       = 6
-#
-
-dimension
-1
-
-elements
-4
-1 1 0 1
-1 1 1 2
-1 1 2 3
-1 1 3 4
-
-boundary
-2
-1 0 0
-2 0 4
-
-vertices
-5
-3
-0 0 0
-0.25 0.25 0.25
-0.50 0.50 0.50
-0.75 0.75 0.75
-1 1 1
@@ -0,0 +1,109 @@
+MFEM NURBS mesh v1.0
+
+#
+# MFEM Geometry Types (see mesh/geom.hpp):
+#
+# SEGMENT     = 1
+# SQUARE      = 3
+# CUBE        = 5
+#
+
+dimension
+2
+
+elements
+5
+1 3 4 5 6 7
+1 3 0 1 5 4
+1 3 1 2 6 5
+1 3 3 7 6 2
+1 3 0 4 7 3
+
+boundary
+4
+1 1 0 1
+1 1 2 3
+1 1 1 2
+1 1 3 0
+
+edges
+12
+0 0 1
+0 4 5
+0 7 6
+0 3 2
+1 1 2
+1 5 6
+1 4 7
+1 0 3
+2 0 4
+2 1 5
+2 2 6
+2 3 7
+
+vertices
+8
+
+knotvectors
+3
+2 3 0 0 0 1 1 1
+2 3 0 0 0 1 1 1
+2 3 0 0 0 1 1 1
+
+weights
+1
+1
+1
+1
+1
+1
+1
+1
+0.70710678118655
+1
+1
+0.70710678118655
+0.70710678118655
+1
+1
+0.70710678118655
+1
+1
+1
+1
+1
+0.85355339059327
+0.85355339059327
+0.85355339059327
+0.85355339059327
+
+FiniteElementSpace
+FiniteElementCollection: NURBS2
+VDim: 2
+Ordering: 1
+
+-0.70710678118 -0.70710678118
+0.70710678118 -0.70710678118
+0.70710678118 0.70710678118
+-0.70710678118 0.70710678118
+-0.35355339059 -0.35355339059
+0.35355339059 -0.35355339059
+0.35355339059 0.35355339059
+-0.35355339059 0.35355339059
+0 -1.41421356236
+0 -0.35355339059
+0 0.35355339059
+0 1.41421356236
+1.41421356236 0
+0.35355339059 0
+-0.35355339059 0
+-1.41421356236 0
+-0.530330085885 -0.530330085885
+0.530330085885 -0.530330085885
+0.530330085885 0.530330085885
+-0.530330085885 0.530330085885
+0 0
+0 -0.883883476475
+0.883883476475 0
+0 0.883883476475
+-0.883883476475 0
@@ -1,48 +0,0 @@
-MFEM mesh v1.0
-#
-# MFEM Geometry Types (see mesh/geom.hpp):
-#
-# POINT       = 0
-# SEGMENT     = 1
-# TRIANGLE    = 2
-# SQUARE      = 3
-# TETRAHEDRON = 4
-# CUBE        = 5
-# PRISM       = 6
-#
-
-dimension
-2
-
-elements
-6
-1 3 0 1 4 3
-1 3 2 3 6 5
-1 2 3 4 8
-1 2 4 7 8
-1 2 7 6 8
-1 2 6 3 8
-
-boundary
-8
-1 1 0 1
-2 1 1 4
-3 1 4 7
-4 1 7 6
-5 1 6 5
-6 1 5 2
-7 1 2 3
-8 1 3 0
-
-vertices
-9
-2
-0.5 0
-1 0
-0 0.5
-0.5 0.5
-1 0.5
-0 1
-0.5 1
-1 1
-0.75 0.75
@@ -1,44 +0,0 @@
-MFEM mesh v1.0
-#
-# MFEM Geometry Types (see mesh/geom.hpp):
-#
-# POINT       = 0
-# SEGMENT     = 1
-# TRIANGLE    = 2
-# SQUARE      = 3
-# TETRAHEDRON = 4
-# CUBE        = 5
-# PRISM       = 6
-#
-
-dimension
-2
-
-elements
-3
-1 3 0 1 4 3
-1 3 2 3 6 5
-1 3 3 4 7 6
-
-boundary
-8
-1 1 0 1
-2 1 1 4
-3 1 4 7
-4 1 7 6
-5 1 6 5
-6 1 5 2
-7 1 2 3
-8 1 3 0
-
-vertices
-8
-2
-0.5 0
-1 0
-0 0.5
-0.5 0.5
-1 0.5
-0 1
-0.5 1
-1 1
@@ -1,322 +0,0 @@
-MFEM mesh v1.0
-
-#
-# MFEM Geometry Types (see mesh/geom.hpp):
-#
-# POINT       = 0
-# SEGMENT     = 1
-# TRIANGLE    = 2
-# SQUARE      = 3
-# TETRAHEDRON = 4
-# CUBE        = 5
-# PRISM       = 6
-# PYRAMID     = 7
-#
-
-dimension
-2
-
-elements
-26
-1 2 1 18 0
-1 3 1 3 19 18
-2 3 3 6 20 19
-1 3 6 9 21 20
-2 3 9 12 22 21
-1 3 12 15 23 22
-2 2 23 15 24
-1 2 1 4 3
-2 3 4 7 6 3
-1 3 7 10 9 6
-2 3 10 13 12 9
-1 3 13 16 15 12
-2 3 16 25 24 15
-1 3 2 5 4 1
-1 3 5 8 7 4
-1 3 8 11 10 7
-1 3 11 14 13 10
-1 3 14 17 16 13
-1 2 25 16 17
-1 3 18 19 27 26
-2 3 19 20 28 27
-1 3 20 21 29 28
-2 3 21 22 30 29
-1 3 22 23 31 30
-2 3 23 24 32 31
-1 3 24 25 33 32
-
-boundary
-18
-1 1 28 27
-2 1 30 29
-3 1 32 31
-4 1 0 1
-4 1 1 2
-4 1 2 5
-4 1 5 8
-4 1 8 11
-4 1 11 14
-4 1 14 17
-4 1 17 25
-4 1 25 33
-4 1 33 32
-4 1 31 30
-4 1 29 28
-4 1 27 26
-4 1 26 18
-4 1 18 0
-
-vertices
-34
-
-nodes
-FiniteElementSpace
-FiniteElementCollection: H1_2D_P3
-VDim: 2
-Ordering: 1
-
-0 0
-0.53125 0
-1 0
-0.53125 0.09375
-0.5625 0.09375
-1 0.09375
-0.53125 0.21875
-0.6875 0.21875
-1 0.1875
-0.53125 0.25
-0.71875 0.25
-1 0.25
-0.53125 0.375
-0.84375 0.375
-1 0.34375
-0.53125 0.40625
-0.875 0.40625
-1 0.40625
-0 0.53125
-0.09375 0.53125
-0.21875 0.53125
-0.25 0.53125
-0.375 0.53125
-0.40625 0.53125
-0.53125 0.53125
-1 0.53125
-0 1
-0.09375 1
-0.21875 1
-0.25 1
-0.375 1
-0.40625 1
-0.53125 1
-1 1
-
-0.33175106835972 0.094168845750364
-0.094168845750364 0.33175106835972
-5.1759634627347e-17 0.14683388869532
-6.5255471622478e-17 0.38441611130468
-0.14683388869532 6.0713766400335e-17
-0.38441611130468 8.2458945395444e-19
-0.53125 0.025911862710939
-0.53125 0.067838137289061
-0.34721731046049 0.13433915461926
-0.13433915461926 0.34721731046049
-0.025911862710939 0.53125
-0.067838137289061 0.53125
-0.53125 0.12829915028125
-0.53125 0.18420084971875
-0.39979807890035 0.24774225329947
-0.24774225329947 0.39979807890035
-0.12829915028125 0.53125
-0.18420084971875 0.53125
-0.53125 0.22738728757031
-0.53125 0.24136271242969
-0.41294327101031 0.27609302796952
-0.27609302796952 0.41294327101031
-0.22738728757031 0.53125
-0.24136271242969 0.53125
-0.53125 0.28454915028125
-0.53125 0.34045084971875
-0.46552403945017 0.38949612664974
-0.38949612664974 0.46552403945017
-0.28454915028125 0.53125
-0.34045084971875 0.53125
-0.53125 0.38363728757031
-0.53125 0.39761271242969
-0.47866923156014 0.41784690131979
-0.41784690131979 0.47866923156014
-0.38363728757031 0.53125
-0.39761271242969 0.53125
-0.53125 0.44079915028125
-0.53125 0.49670084971875
-0.44079915028125 0.53125
-0.49670084971875 0.53125
-0.53988728757031 0.025911862710939
-0.55386271242969 0.067838137289061
-0.53988728757031 0.09375
-0.55386271242969 0.09375
-0.59704915028125 0.12829915028125
-0.65295084971875 0.18420084971875
-0.57443643785157 0.21875
-0.64431356214843 0.21875
-0.69613728757031 0.22738728757031
-0.71011271242969 0.24136271242969
-0.58307372542188 0.25
-0.66692627457812 0.25
-0.75329915028125 0.28454915028125
-0.80920084971875 0.34045084971875
-0.61762287570313 0.375
-0.75737712429687 0.375
-0.85238728757031 0.38363728757031
-0.86636271242969 0.39761271242969
-0.62626016327344 0.40625
-0.77998983672656 0.40625
-0.90954915028125 0.44079915028125
-0.96545084971875 0.49670084971875
-0.6608093135547 0.53125
-0.8704406864453 0.53125
-1 0.025911862710939
-1 0.067838137289061
-0.68342202598438 0.09375
-0.87907797401562 0.09375
-0.6608093135547 0
-0.8704406864453 0
-1 0.11966186271094
-1 0.16158813728906
-0.77387287570313 0.21011271242969
-0.91362712429687 0.19613728757031
-1 0.20477457514063
-1 0.23272542485937
-0.79648558813282 0.25
-0.92226441186718 0.25
-1 0.27591186271094
-1 0.31783813728906
-0.88693643785157 0.36636271242969
-0.95681356214843 0.35238728757031
-1 0.36102457514063
-1 0.38897542485937
-0.90954915028125 0.40625
-0.96545084971875 0.40625
-1 0.44079915028125
-1 0.49670084971875
-0.09375 0.6608093135547
-0.09375 0.8704406864453
-0.025911862710939 1
-0.067838137289061 1
-0 0.6608093135547
-0 0.8704406864453
-0.21875 0.6608093135547
-0.21875 0.8704406864453
-0.12829915028125 1
-0.18420084971875 1
-0.25 0.6608093135547
-0.25 0.8704406864453
-0.22738728757031 1
-0.24136271242969 1
-0.375 0.6608093135547
-0.375 0.8704406864453
-0.28454915028125 1
-0.34045084971875 1
-0.40625 0.6608093135547
-0.40625 0.8704406864453
-0.38363728757031 1
-0.39761271242969 1
-0.53125 0.6608093135547
-0.53125 0.8704406864453
-0.44079915028125 1
-0.49670084971875 1
-1 0.6608093135547
-1 0.8704406864453
-0.6608093135547 1
-0.8704406864453 1
-0.14782497614169 0.14782497614169
-0.3364183509774 0.10629113008478
-0.3439701728879 0.12590539815918
-0.10629113008478 0.3364183509774
-0.12590539815918 0.3439701728879
-0.36175027742635 0.16568300020856
-0.38526511193449 0.21639840771017
-0.16568300020856 0.36175027742635
-0.21639840771017 0.38526511193449
-0.40343132064181 0.2555782146968
-0.40931002926885 0.2682570665722
-0.2555782146968 0.40343132064181
-0.2682570665722 0.40931002926885
-0.42747623797617 0.30743687355882
-0.45099107248431 0.35815228106044
-0.30743687355882 0.42747623797617
-0.35815228106044 0.45099107248431
-0.46915728119164 0.39733208804706
-0.47503598981867 0.41001093992246
-0.39733208804706 0.46915728119164
-0.41001093992246 0.47503598981867
-0.47232443490112 0.47232443490112
-0.54166666666667 0.0625
-0.57886271242969 0.12829915028125
-0.61931356214843 0.18420084971875
-0.54943643785157 0.12829915028125
-0.56488728757031 0.18420084971875
-0.65056356214843 0.22738728757031
-0.66067627457812 0.24136271242969
-0.57682372542188 0.22738728757031
-0.58068643785157 0.24136271242969
-0.69192627457812 0.28454915028125
-0.73237712429687 0.34045084971875
-0.59262287570313 0.28454915028125
-0.60807372542188 0.34045084971875
-0.76362712429687 0.38363728757031
-0.77373983672656 0.39761271242969
-0.62001016327344 0.38363728757031
-0.62387287570313 0.39761271242969
-0.80498983672656 0.44079915028125
-0.8454406864453 0.49670084971875
-0.6358093135547 0.44079915028125
-0.65126016327344 0.49670084971875
-0.87282797401562 0.025911862710939
-0.8766906864453 0.067838137289061
-0.6670593135547 0.025911862710939
-0.67717202598438 0.067838137289061
-0.88862712429687 0.12204915028125
-0.90407797401562 0.16783813728906
-0.70842202598438 0.12591186271094
-0.74887287570313 0.17795084971875
-0.91601441186718 0.21102457514063
-0.91987712429687 0.23511271242969
-0.78012287570313 0.22113728757031
-0.79023558813282 0.23897542485937
-0.93181356214843 0.27829915028125
-0.94726441186718 0.32408813728906
-0.82148558813282 0.28216186271094
-0.86193643785157 0.33420084971875
-0.95920084971875 0.36727457514063
-0.96306356214843 0.39136271242969
-0.89318643785157 0.37738728757031
-0.90329915028125 0.39522542485937
-0.95833333333333 0.44791666666667
-0.025911862710939 0.6608093135547
-0.067838137289061 0.6608093135547
-0.025911862710939 0.8704406864453
-0.067838137289061 0.8704406864453
-0.12829915028125 0.6608093135547
-0.18420084971875 0.6608093135547
-0.12829915028125 0.8704406864453
-0.18420084971875 0.8704406864453
-0.22738728757031 0.6608093135547
-0.24136271242969 0.6608093135547
-0.22738728757031 0.8704406864453
-0.24136271242969 0.8704406864453
-0.28454915028125 0.6608093135547
-0.34045084971875 0.6608093135547
-0.28454915028125 0.8704406864453
-0.34045084971875 0.8704406864453
-0.38363728757031 0.6608093135547
-0.39761271242969 0.6608093135547
-0.38363728757031 0.8704406864453
-0.39761271242969 0.8704406864453
-0.44079915028125 0.6608093135547
-0.49670084971875 0.6608093135547
-0.44079915028125 0.8704406864453
-0.49670084971875 0.8704406864453
-0.6608093135547 0.6608093135547
-0.8704406864453 0.6608093135547
-0.6608093135547 0.8704406864453
-0.8704406864453 0.8704406864453
@@ -38,7 +38,7 @@ PROJECT_NAME           = "MFEM"
 # could be handy for archiving the generated documentation or if some version
 # control system is used.

-PROJECT_NUMBER         = v4.6.1
+PROJECT_NUMBER         = v4.5.3

 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a
@@ -795,7 +795,6 @@ INPUT                  = @MFEM_SOURCE_DIR@/doc/CodeDocumentation.dox \
                         @MFEM_SOURCE_DIR@/miniapps/common \
                         @MFEM_SOURCE_DIR@/miniapps/electromagnetics \
                         @MFEM_SOURCE_DIR@/miniapps/gslib \
-                         @MFEM_SOURCE_DIR@/miniapps/hdiv-linear-solver \
                         @MFEM_SOURCE_DIR@/miniapps/hooke \
                         @MFEM_SOURCE_DIR@/miniapps/hooke/kernels \
                         @MFEM_SOURCE_DIR@/miniapps/hooke/materials \
@@ -811,10 +810,7 @@ INPUT                  = @MFEM_SOURCE_DIR@/doc/CodeDocumentation.dox \
                         @MFEM_SOURCE_DIR@/miniapps/shifted \
                         @MFEM_SOURCE_DIR@/miniapps/solvers \
                         @MFEM_SOURCE_DIR@/miniapps/tools \
-                         @MFEM_SOURCE_DIR@/miniapps/toys \
-                         @MFEM_SOURCE_DIR@/miniapps/spde \
-                         @MFEM_SOURCE_DIR@/miniapps/dpg \
-                         @MFEM_SOURCE_DIR@/miniapps/dpg/util
+                         @MFEM_SOURCE_DIR@/miniapps/toys

 # This tag can be used to specify the character encoding of the source files
 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses
@@ -39,7 +39,7 @@ namespace mfem {
 * - Device
 * - Memory
 * - MemoryManager
- * - mfem::forall functions in forall.hpp
+ * - MFEM_FORALL macro in forall.hpp
 *
 * <H3>Example codes</H3>
 * - <a class="el" href="ex0_8cpp_source.html">Example 0</a>: simplest example, nodal H1 FEM for the Laplace problem
@@ -105,13 +105,6 @@ namespace mfem {
 * - <a class="el" href="ex32p_8cpp_source.html">Example 32p</a>: parallel anisotropic Maxwell eigensolver
 * - <a class="el" href="ex33_8cpp_source.html">Example 33</a>: nodal H1 FEM for the fractional Laplacian problem
 * - <a class="el" href="ex33p_8cpp_source.html">Example 33p</a>: parallel nodal H1 FEM for the fractional Laplacian problem
- * - <a class="el" href="ex34_8cpp_source.html">Example 34</a>: multi-domain magnetostatics
- * - <a class="el" href="ex34p_8cpp_source.html">Example 34p</a>: parallel multi-domain magnetostatics
- * - <a class="el" href="ex35p_8cpp_source.html">Example 35p</a>: parallel multi-domain damped harmonic oscillators
- * - <a class="el" href="ex36_8cpp_source.html">Example 36</a>: Proximal Galerkin FEM for the obstacle problem
- * - <a class="el" href="ex36p_8cpp_source.html">Example 36p</a>: parallel Proximal Galerkin FEM for the obstacle problem
- * - <a class="el" href="ex37_8cpp_source.html">Example 37</a>: Topology optimization
- * - <a class="el" href="ex37p_8cpp_source.html">Example 37p</a>: parallel topology optimization
 *
 * <H4>AmgX Examples</H4>
 * - Variants of Examples
@@ -193,7 +186,6 @@ namespace mfem {
 * - <a class="el" href="extruder_8cpp_source.html">Extruder</a>: extrude a low-dimensional mesh into a higher dimension
 * - <a class="el" href="mesh-explorer_8cpp_source.html">Mesh Explorer</a>: visualize and manipulate meshes
 * - <a class="el" href="mesh-optimizer_8cpp_source.html">Mesh Optimizer</a>: optimize high-order meshes, <a class="el" href="mesh-optimizer_8cpp_source.html">serial</a> and <a class="el" href="pmesh-optimizer_8cpp_source.html">parallel</a> versions
- * - <a class="el" href="mesh-quality_8cpp_source.html">Mesh Quality</a>: visualize and check mesh quality
 * - <a class="el" href="trimmer_8cpp_source.html">Trimmer</a>: trim elements from existing meshes
 * - <a class="el" href="display-basis_8cpp_source.html">Display Basis</a>: visualize finite element basis functions
 * - <a class="el" href="get-values_8cpp_source.html">Get Values</a>: extract field values via DataCollection classes
@@ -206,15 +198,12 @@ namespace mfem {
 * - <a class="el" href="distance_8cpp_source.html">Distance</a>: finite element distance function solver
 * - <a class="el" href="diffusion_8cpp_source.html">Shifted Diffusion</a>: shifted boundary diffusion solver
 * - <a class="el" href="extrapolate_8cpp_source.html">Extrapolation</a>: PDE-based extrapolation of finite element functions
- * - <a class="el" href="block-solvers_8cpp_source.html">Block Solvers</a>: comparison of saddle point system solvers
+ * - <a class="el" href="distance_8cpp_source.html">Block Solvers</a>: comparison of saddle point system solvers
 * - <a class="el" href="parheat_8cpp_source.html">Optimization gradients</a>: Gradients of PDE-constrained function
 * - <a class="el" href="par__example_8cpp_source.html">Parallel AD</a>: Parallel p-Laplacian example
 * - <a class="el" href="seq__example_8cpp_source.html">Serial AD</a>: Serial p-Laplacian example
 * - <a class="el" href="miniapps_2performance_2ex1_8cpp_source.html">HPC Example 1</a>: high-performance nodal H1 FEM for the Laplace problem
 * - <a class="el" href="miniapps_2performance_2ex1p_8cpp_source.html">HPC Example 1p</a>: high-performance parallel nodal H1 FEM for the Laplace problem
- * - <a class="el" href="generate__random__field_8cpp_source.html">SPDE Solvers</a>: SPDE solver random field generation
- * - <a class="el" href="pdiffusion_8cpp_source.html">DPG Diffusion example</a>: DPG formulation for the diffusion problem
- * - <a class="el" href="pmaxwell_8cpp_source.html">DPG Maxwell example</a>: DPG formulation for the indefinite Maxwell problem
 *
 * See also the <a class="el" href="https://mfem.org/examples/">examples documentation</a> online.
 */
@@ -40,9 +40,6 @@ list(APPEND ALL_EXE_SRCS
  ex30.cpp
  ex31.cpp
  ex33.cpp
-  ex34.cpp
-  ex36.cpp
-  ex37.cpp
  )

 if (MFEM_USE_MPI)
@@ -80,10 +77,6 @@ if (MFEM_USE_MPI)
    ex31p.cpp
    ex32p.cpp
    ex33p.cpp
-    ex34p.cpp
-    ex35p.cpp
-    ex36p.cpp
-    ex37p.cpp
    )
 endif()

@@ -109,8 +102,6 @@ if (MFEM_ENABLE_TESTING)
      list(APPEND THIS_TEST_OPTIONS "-e" "1")
    elseif(${TEST_NAME} MATCHES "ex27p*")
      list(APPEND THIS_TEST_OPTIONS "-dg")
-    elseif(${TEST_NAME} MATCHES "ex37p*")
-      list(APPEND THIS_TEST_OPTIONS "-mi" "3")
    endif()

    if (NOT (${TEST_NAME} MATCHES ".*p$"))
@@ -128,10 +119,9 @@ if (MFEM_ENABLE_TESTING)
  # Add CUDA/HIP tests.
  set(DEVICE_EXAMPLES
      # serial examples with device support:
-      ex1 ex3 ex4 ex5 ex6 ex9 ex22 ex24 ex25 ex26 ex34
+      ex1 ex3 ex4 ex5 ex6 ex9 ex22 ex24 ex25 ex26
      # parallel examples with device support:
-      ex1p ex2p ex3p ex4p ex5p ex6p ex7p ex9p ex13p ex22p ex24p ex25p ex26p
-      ex34p ex35p)
+      ex1p ex2p ex3p ex4p ex5p ex6p ex7p ex9p ex13p ex22p ex24p ex25p ex26p)
  set(MFEM_TEST_DEVICE)
  if (MFEM_USE_CUDA)
    set(MFEM_TEST_DEVICE "cuda")
@@ -171,15 +161,6 @@ if (MFEM_ENABLE_TESTING)
      $<TARGET_FILE:ex11p> "-no-vis" "--superlu"
      ${MPIEXEC_POSTFLAGS})
  endif()
-
-  # If MUMPS is enabled, add a test run that uses it.
-  if (MFEM_USE_MUMPS)
-    add_test(NAME ex25p_mumps_np=${MFEM_MPI_NP}
-      COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} ${MFEM_MPI_NP}
-      ${MPIEXEC_PREFLAGS}
-      $<TARGET_FILE:ex25p> "-no-vis" "--mumps-solver"
-      ${MPIEXEC_POSTFLAGS})
-  endif()
 endif()

 # Include the examples/amgx directory if AmgX is enabled
@@ -0,0 +1,141 @@
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include "IPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+
+double fRhs(const Vector &pt);
+double obstacle(const Vector &pt);
+double dmanufacturedFun(const Vector &pt);
+
+int main(int argc, char *argv[])
+{
+  int FEorder = 1; // order of the finite elements
+  int linSolver = 0;
+  int maxIPMiters = 30;
+  bool iAmRoot = true;
+  int ref_levels = 3; 
+  OptionsParser args(argc, argv);
+  args.AddOption(&FEorder, "-o", "--order",\
+		  "Order of the finite elements.");
+  args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+       "IP-Newton linear system solution strategy.");
+  args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+		  "Maximum number of IPM iterations");
+  args.AddOption(&ref_levels, "-r", "--mesh_refinement", \
+		  "Mesh Refinement");
+  
+  args.Parse();
+  if(!args.Good())
+  {
+    args.PrintUsage(cout);
+    return 1;
+  }
+  else
+  {
+    if( iAmRoot )
+    {
+      args.PrintOptions(cout);
+    }
+  }
+
+  const char *meshFile = "../../data/inline-quad.mesh";
+  Mesh *mesh = new Mesh(meshFile, 1, 1);
+  int dim = mesh->Dimension(); // geometric dimension of the domain
+  for (int l = 0; l < ref_levels; l++)
+  {
+      mesh->UniformRefinement();
+  }
+
+  FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+  FiniteElementSpace      *Vh  = new FiniteElementSpace(mesh, fec);
+  Array<int> ess_tdof_list;
+  if (mesh->bdr_attributes.Size())
+  {
+     Array<int> ess_bdr(mesh->bdr_attributes.Max());
+     ess_bdr = 1;
+     Vh->GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
+  }
+  
+  double DC_val = 0.0;
+  int dimD = Vh->GetTrueVSize();
+  Vector x0(dimD); x0 = DC_val;
+  Vector xf(dimD); xf = 0.0;
+
+  ObstacleProblem problem(Vh, x0, &fRhs, &obstacle, ess_tdof_list);
+  
+
+  InteriorPointSolver optimizer(&problem); 
+  optimizer.SetTol(1.e-7);
+  optimizer.SetLinearSolver(linSolver);
+  optimizer.SetMaxIter(maxIPMiters);
+  optimizer.Mult(x0, xf);
+  
+  double Einitial = problem.E(x0);
+  double Efinal = problem.E(xf);
+  cout << "Energy objective at initial point = " << Einitial << endl;
+  cout << "Energy objective at optimizer = " << Efinal << endl;
+
+  GridFunction d_gf(Vh);
+
+  d_gf = xf;
+  
+  FunctionCoefficient dm_fc(dmanufacturedFun); // pseudo-manufactured solution
+  GridFunction dm_gf(Vh);
+  dm_gf.ProjectCoefficient(dm_fc);
+
+  
+  ParaViewDataCollection paraview_dc("BarrierProblemSolution", mesh);
+  paraview_dc.SetPrefixPath("ParaView");
+  paraview_dc.SetLevelsOfDetail(FEorder);
+  paraview_dc.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc.SetHighOrderOutput(true);
+  paraview_dc.SetCycle(0);
+  paraview_dc.SetTime(0.0);
+  paraview_dc.RegisterField("d(x) (numerical)", &d_gf);
+  paraview_dc.RegisterField("d(x) (pseudo-manufactured)", &dm_gf);
+  paraview_dc.Save();
+  
+
+  delete Vh;
+  delete fec;
+  delete mesh;
+  return 0;
+}
+
+
+
+double dmanufacturedFun(const Vector &pt)
+{
+  double alpha = 16.5;
+  return sin(M_PI * pt(1)) * (sin(M_PI * pt(0)) - alpha * pow(pt(0) * (1. - pt(0)), 2));
+}
+
+
+// f(x) forcing term... which enters the objective energy functional
+// E(d) = 0.5 d^T K d - f^T d, where f is a discrete vector representation
+// of f(x). f(x) is such that in the absence of bound-constraints then
+// the solution of the optimization problem satisfies the PDE
+// -div(grad(d)) + d = f + homogeneous Neumann conditions on the unit interval,
+// for d(x) = cos(2 \pi x) + a0 + a3 (x^3 - 1.5 x^2), a2 = 0.2, a3 = -2
+
+double fRhs(const Vector &pt)
+{
+  double alpha = 16.5;
+  double fx;
+  fx = pow(M_PI, 2) * sin(M_PI * pt(0));
+  fx += alpha * (2. * pow(pt(0), 2) + 2. * pow(1.-pt(0), 2) - 8. * pt(0) * (1.-pt(0)));
+  fx += pow(M_PI, 2) * sin(M_PI * pt(0)) * dmanufacturedFun(pt);
+  fx *= sin(M_PI * pt(1));
+  return fx;
+}
+
+double obstacle(const Vector &pt)
+{
+  return 0.0;
+}
@@ -0,0 +1,156 @@
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include "IPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+
+double fRhs(const Vector &pt);
+double obstacle(const Vector &pt);
+double dmanufacturedFun(const Vector &pt);
+
+int main(int argc, char *argv[])
+{
+  int FEorder = 1; // order of the finite elements
+  int linSolver = 0;
+  int maxIPMiters = 30;
+  bool iAmRoot = true;
+  int ref_levels = 3; 
+  OptionsParser args(argc, argv);
+  args.AddOption(&FEorder, "-o", "--order",\
+		  "Order of the finite elements.");
+  args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+       "IP-Newton linear system solution strategy.");
+  args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+		  "Maximum number of IPM iterations");
+  args.AddOption(&ref_levels, "-r", "--mesh_refinement", \
+		  "Mesh Refinement");
+  
+  args.Parse();
+  if(!args.Good())
+  {
+    args.PrintUsage(cout);
+    return 1;
+  }
+  else
+  {
+    if( iAmRoot )
+    {
+      args.PrintOptions(cout);
+    }
+  }
+
+  const char *meshFile = "../../data/inline-quad.mesh";
+  Mesh *mesh = new Mesh(meshFile, 1, 1);
+  int dim = mesh->Dimension(); // geometric dimension of the domain
+  for (int l = 0; l < ref_levels; l++)
+  {
+      mesh->UniformRefinement();
+  }
+
+  FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+  FiniteElementSpace      *Vh  = new FiniteElementSpace(mesh, fec);
+  Array<int> ess_tdof_list;
+  if (mesh->bdr_attributes.Size())
+  {
+     Array<int> ess_bdr(mesh->bdr_attributes.Max());
+     ess_bdr = 1;
+     Vh->GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
+  }
+  
+  double DC_val = 0.06;
+  Vector x0DC(Vh->GetTrueVSize()); x0DC = DC_val;
+  int dimD = Vh->GetTrueVSize() - ess_tdof_list.Size();
+  Vector x0(dimD); x0 = 0.0;
+  Vector xf(dimD); xf = 0.0;
+
+  ObstacleProblemVariant problem(Vh, x0DC, &fRhs, &obstacle, ess_tdof_list);
+
+  InteriorPointSolver optimizer(&problem); 
+  optimizer.SetTol(1.e-7);
+  optimizer.SetLinearSolver(linSolver);
+  optimizer.SetMaxIter(maxIPMiters);
+  optimizer.Mult(x0, xf);
+  
+  double Einitial = problem.E(x0);
+  double Efinal = problem.E(xf);
+  cout << "Energy objective at initial point = " << Einitial << endl;
+  cout << "Energy objective at optimizer = " << Efinal << endl;
+
+  Array<int> noness_tdof_list;
+  noness_tdof_list.SetSize(dimD);
+  int i = 0;
+  for(int j = 0; j < Vh->GetTrueVSize(); j++)
+  {
+     if(ess_tdof_list.Find(j) == -1)
+     {
+        noness_tdof_list[i] = j;
+        i += 1;
+     }
+  }
+
+
+  GridFunction d_gf(Vh);
+  d_gf.Set(1.0, x0DC);
+  d_gf.SetSubVector(noness_tdof_list, xf);
+
+
+
+  FunctionCoefficient dm_fc(dmanufacturedFun); // pseudo-manufactured solution
+  GridFunction dm_gf(Vh);
+  dm_gf.ProjectCoefficient(dm_fc);
+
+  
+  ParaViewDataCollection paraview_dc("BarrierProblemSolution", mesh);
+  paraview_dc.SetPrefixPath("ParaView");
+  paraview_dc.SetLevelsOfDetail(FEorder);
+  paraview_dc.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc.SetHighOrderOutput(true);
+  paraview_dc.SetCycle(0);
+  paraview_dc.SetTime(0.0);
+  paraview_dc.RegisterField("d(x) (numerical)", &d_gf);
+  paraview_dc.RegisterField("d(x) (pseudo-manufactured)", &dm_gf);
+  paraview_dc.Save();
+ 
+
+  delete Vh;
+  delete fec;
+  delete mesh;
+  return 0;
+}
+
+
+
+double dmanufacturedFun(const Vector &pt)
+{
+  double alpha = 16.5;
+  return sin(M_PI * pt(1)) * (sin(M_PI * pt(0)) - alpha * pow(pt(0) * (1. - pt(0)), 2));
+}
+
+
+// f(x) forcing term... which enters the objective energy functional
+// E(d) = 0.5 d^T K d - f^T d, where f is a discrete vector representation
+// of f(x). f(x) is such that in the absence of bound-constraints then
+// the solution of the optimization problem satisfies the PDE
+// -div(grad(d)) + d = f + homogeneous Neumann conditions on the unit interval,
+// for d(x) = cos(2 \pi x) + a0 + a3 (x^3 - 1.5 x^2), a2 = 0.2, a3 = -2
+
+double fRhs(const Vector &pt)
+{
+  double alpha = 16.5;
+  double fx;
+  fx = pow(M_PI, 2) * sin(M_PI * pt(0));
+  fx += alpha * (2. * pow(pt(0), 2) + 2. * pow(1.-pt(0), 2) - 8. * pt(0) * (1.-pt(0)));
+  fx += pow(M_PI, 2) * sin(M_PI * pt(0)) * dmanufacturedFun(pt);
+  fx *= sin(M_PI * pt(1));
+  return fx;
+}
+
+double obstacle(const Vector &pt)
+{
+  return 0.0;
+}
@@ -0,0 +1,827 @@
+#include "mfem.hpp"
+#include "IPsolver.hpp"
+#include "Problems.hpp"
+#include <fstream>
+#include <iostream>
+#include <cstdlib>
+
+using namespace std;
+using namespace mfem;
+
+
+
+InteriorPointSolver::InteriorPointSolver(GeneralOptProblem * Problem) : optProblem(Problem), block_offsetsumlz(5), block_offsetsuml(4), block_offsetsx(3),
+Huu(nullptr), Hum(nullptr), Hmu(nullptr), Hmm(nullptr), Wmm(nullptr), D(nullptr), Ju(nullptr), Jm(nullptr), JuT(nullptr), JmT(nullptr), Huucl(nullptr), HLuu(nullptr), saveLogBarrierIterates(false)
+{
+  rel_tol  = 1.e-2;
+  max_iter = 20;
+  mu_k     = 1.0;
+
+  sMax     = 1.e2;
+  kSig     = 1.e10;   // control deviation from primal Hessian
+  tauMin   = 0.8;     // control rate at which iterates can approach the boundary
+  eta      = 1.e-4;   // backtracking constant
+  thetaMin = 1.e-4;   // allowed violation of the equality constraints
+
+  // constants in line-step A-5.4
+  delta    = 1.0;
+  sTheta   = 1.1;
+  sPhi     = 2.3;
+
+  // control the rate at which the penalty parameter is decreased
+  kMu     = 0.2;
+  thetaMu = 1.5;
+
+  // TO DO -- include the filter
+
+  thetaMax = 1.e6; // maximum constraint violation
+  // data for the second order correction
+  kSoc     = 0.99;
+
+  // equation (18)
+  gTheta = 1.e-5;
+  gPhi   = 1.e-5;
+
+  kEps   = 1.e1;
+
+  dimU = optProblem->GetDimU();
+  dimM = optProblem->GetDimM();
+  dimC = optProblem->GetDimC();
+  ckSoc.SetSize(dimC);
+  
+  block_offsetsumlz[0] = 0;
+  block_offsetsumlz[1] = dimU; // u
+  block_offsetsumlz[2] = dimM; // m
+  block_offsetsumlz[3] = dimC; // lambda
+  block_offsetsumlz[4] = dimM; // zl
+  block_offsetsumlz.PartialSum();
+
+  for(int i = 0; i < block_offsetsuml.Size(); i++)  { block_offsetsuml[i] = block_offsetsumlz[i]; }
+  for(int i = 0; i < block_offsetsx.Size(); i++)    { block_offsetsx[i]   = block_offsetsuml[i] ; }
+
+  // lower-bound for the inequality constraint m >= ml
+  ml = optProblem->Getml();
+  
+  lk.SetSize(dimC);  lk  = 0.0;
+  zlk.SetSize(dimM); zlk = 0.0;
+
+  linSolver = 0;
+  MyRank = 0;
+  iAmRoot = MyRank == 0 ? true : false;
+}
+
+double InteriorPointSolver::MaxStepSize(Vector &x, Vector &xl, Vector &xhat, double tau)
+{
+  double alphaMaxloc = 1.0;
+  double alphaTmp;
+  for(int i = 0; i < x.Size(); i++)
+  {
+    if( xhat(i) < 0. )
+    {
+      alphaTmp = -1. * tau * (x(i) - xl(i)) / xhat(i);
+      alphaMaxloc = min(alphaMaxloc, alphaTmp);
+    } 
+  }
+
+  // alphaMaxloc is the local maximum step size which is
+  // distinct on each MPI process. Need to compute
+  // the global maximum step size 
+  double alphaMaxglb;
+  alphaMaxglb = alphaMaxloc;
+  return alphaMaxglb;
+}
+
+double InteriorPointSolver::MaxStepSize(Vector &x, Vector &xhat, double tau)
+{
+  Vector zero(x.Size()); zero = 0.0;
+  return MaxStepSize(x, zero, xhat, tau);
+}
+
+
+void InteriorPointSolver::Mult(const Vector &x0, Vector &xf)
+{
+  BlockVector x0block(block_offsetsx); x0block = 0.0;
+  x0block.GetBlock(0).Set(1.0, x0);
+  // To do: give options for user specificiation of initialization m0
+  x0block.GetBlock(1) = 100.;
+  x0block.GetBlock(1).Add(1.0, ml);
+  BlockVector xfblock(block_offsetsx); xfblock = 0.0;
+  Mult(x0block, xfblock);
+  xf.Set(1.0, xfblock.GetBlock(0));
+}
+
+void InteriorPointSolver::Mult(const BlockVector &x0, BlockVector &xf)
+{
+  converged = false;
+  
+  BlockVector xk(block_offsetsx), xhat(block_offsetsx); xk = 0; xhat = 0.0;
+  BlockVector Xk(block_offsetsumlz), Xhat(block_offsetsumlz); Xk = 0.0; Xhat = 0.0;
+  BlockVector Xhatuml(block_offsetsuml); Xhatuml = 0.0;
+  Vector zlhat(dimM); zlhat = 0.0;
+
+  xk.GetBlock(0).Set(1.0, x0.GetBlock(0));
+  xk.GetBlock(1).Set(1.0, x0.GetBlock(1));
+  // running estimate of the final values of the Lagrange multipliers
+  lk  = 0.0;
+  zlk = 0.0;
+
+  for(int i = 0; i < dimM; i++)
+  {
+    zlk(i) = 1.e1 * mu_k / (xk(i+dimU) - ml(i));
+  }
+
+  Xk.GetBlock(0).Set(1.0, xk.GetBlock(0));
+  Xk.GetBlock(1).Set(1.0, xk.GetBlock(1));
+  Xk.GetBlock(2).Set(1.0, lk);
+  Xk.GetBlock(3).Set(1.0, zlk);
+
+  /* set theta0 = theta(x0)
+   *     thetaMin
+   *     thetaMax
+   * when theta(xk) < thetaMin and the switching condition holds
+   * then we ask for the Armijo sufficient decrease of the barrier
+   * objective to be satisfied, in order to accept the trial step length alphakl
+   * 
+   * thetaMax controls how the filter is initialized for each log-barrier subproblem
+   * F0 = {(th, phi) s.t. th > thetaMax}
+   * that is the filter does not allow for iterates where the constraint violation
+   * is larger than that of thetaMax
+   */
+  double theta0 = theta(xk);
+  thetaMin = 1.e-4 * max(1.0, theta0);
+  thetaMax = 1.e8  * thetaMin;
+
+  double Eeval, maxBarrierSolves, Eevalmu0;
+  bool printOptimalityError; // control optimality error print to console for log-barrier subproblems
+  
+  maxBarrierSolves = 10;
+
+  for(jOpt = 0; jOpt < max_iter; jOpt++)
+  {
+    mfem::out << "interior-point solve step " << jOpt << endl;
+    // A-2. Check convergence of overall optimization problem
+    printOptimalityError = false;
+    Eevalmu0 = E(xk, lk, zlk, printOptimalityError);
+    if(Eevalmu0 < rel_tol)
+    {
+      converged = true;
+      mfem::out << "solved optimization problem :)\n";
+      break;
+    }
+    
+    if(jOpt > 0) { maxBarrierSolves = 1; }
+    
+    for(int i = 0; i < maxBarrierSolves; i++)
+    {
+      // A-3. Check convergence of the barrier subproblem
+      printOptimalityError = true;
+      Eeval = E(xk, lk, zlk, mu_k, printOptimalityError);
+      if(Eeval < kEps * mu_k)
+      {
+        mfem::out << "solved barrier subproblem, for mu = " << mu_k << endl;
+        // A-3.1. Recompute the barrier parameter
+        mu_k  = max(rel_tol / 10., min(kMu * mu_k, pow(mu_k, thetaMu)));
+        // A-3.2. Re-initialize the filter
+        F1.DeleteAll();
+        F2.DeleteAll();
+      }
+      else
+      {
+        break;
+      }
+    }
+    
+    // A-4. Compute the search direction
+    // solve for (uhat, mhat, lhat)
+    mfem::out << "\n** A-4. IP-Newton solve **\n";
+    zlhat = 0.0; Xhatuml = 0.0;
+    // why do we have Xhatuml ....???
+    // TO DO: remove Xhatuml in favor of passing Xhat
+    IPNewtonSolve(xk, lk, zlk, zlhat, Xhatuml, mu_k, false); 
+
+
+    // assign data stack, X = (u, m, l, zl)
+    Xk = 0.0;
+    Xk.GetBlock(0).Set(1.0, xk.GetBlock(0));
+    Xk.GetBlock(1).Set(1.0, xk.GetBlock(1));
+    Xk.GetBlock(2).Set(1.0, lk);
+    Xk.GetBlock(3).Set(1.0, zlk);
+
+    // assign data stack, Xhat = (uhat, mhat, lhat, zlhat)
+    Xhat = 0.0;
+    for(int i = 0; i < 3; i++)
+    {
+      Xhat.GetBlock(i).Set(1.0, Xhatuml.GetBlock(i));
+    }
+    Xhat.GetBlock(3).Set(1.0, zlhat);
+
+
+    // A-5. Backtracking line search.
+    mfem::out << "\n** A-5. Linesearch **\n";
+    mfem::out << "mu = " << mu_k << endl;
+    
+    lineSearch(Xk, Xhat, mu_k);
+    
+    if(lineSearchSuccess)
+    {
+      if(!switchCondition || !sufficientDecrease)
+      {
+        F1.Append( (1. - gTheta) * thx0);
+        F2.Append( phx0 - gPhi * thx0);
+      }
+      // ----- A-6: Accept the trial point
+      // print info regarding zl...
+      xk.GetBlock(0).Add(alpha, Xhat.GetBlock(0));
+      xk.GetBlock(1).Add(alpha, Xhat.GetBlock(1));
+      lk.Add(alpha,   Xhat.GetBlock(2));
+      zlk.Add(alphaz, Xhat.GetBlock(3));
+      projectZ(xk, zlk, mu_k);
+    }
+    else
+    {
+      mfem::out << "lineSearch not successful :(\n";
+      mfem::out << "attempting feasibility restoration with theta = " << thx0 << endl;
+      mfem::out << "no feasibility restoration implemented, exiting now \n";
+      break;
+    }
+    //
+    if(jOpt + 1 == max_iter) 
+    { 
+      mfem::out << "maximum optimization iterations :(\n";
+    }
+  }
+  // done with optimization routine, just reassign data to xf reference so
+  // that the application code has access to the optimal point
+  xf = 0.0;
+  xf.GetBlock(0).Set(1.0, xk.GetBlock(0));
+  xf.GetBlock(1).Set(1.0, xk.GetBlock(1));
+}
+
+void InteriorPointSolver::FormIPNewtonMat(BlockVector & x, Vector & l, Vector &zl, BlockOperator &Ak)
+{
+  // WARNING: Huu, Hum, Hmu, Hmm should all be Hessian terms of the Lagrangian, currently we 
+  //          them by Hessian terms of the objective function and neglect the Hessian of l^T c
+
+  Huu = optProblem->Duuf(x); Hum = optProblem->Dumf(x);
+  Hmu = optProblem->Dmuf(x); Hmm = optProblem->Dmmf(x);
+
+  Vector DiagLogBar(dimM); DiagLogBar = 0.0;
+  for(int ii = 0; ii < dimM; ii++)
+  {
+    DiagLogBar(ii) = zl(ii) / (x(ii+dimU) - ml(ii));
+  }
+  if(saveLogBarrierIterates)
+  {
+    std::ofstream diagStream;
+    char diagString[100];
+    snprintf(diagString, 100, "logBarrierHessiandata/D%d.dat", jOpt);
+    diagStream.open(diagString, ios::out | ios::trunc);
+    for(int ii = 0; ii < dimM; ii++)
+    {
+      diagStream << setprecision(30) << DiagLogBar(ii) << endl;
+    }
+    diagStream.close();
+  } 
+
+
+  D = new SparseMatrix(DiagLogBar);
+  
+  if(Hmm != nullptr)
+  {
+    Wmm = new SparseMatrix(*Hmm);
+    Wmm->Add(1.0, *D);
+  }
+  else
+  {
+    Wmm = D;
+  }
+
+  Ju = optProblem->Duc(x); JuT = Transpose(*Ju);
+  Jm = optProblem->Dmc(x); JmT = Transpose(*Jm);
+ 
+  Huucl = optProblem->lDuuc(x, l);
+  if(Huucl != nullptr)
+  {
+    HLuu = Add(*Huucl, *Huu);
+    Ak.SetBlock(0, 0, HLuu);
+  } 
+  else
+  {
+    Ak.SetBlock(0, 0, Huu);
+  }
+
+  //         IP-Newton system matrix
+  //    Ak = [[H_(u,u)  H_(u,m)   J_u^T]
+  //          [H_(m,u)  W_(m,m)   J_m^T]
+  //          [ J_u      J_m       0  ]]
+
+  Ak.SetBlock(0, 2, JuT);
+  Ak.SetBlock(1, 1, Wmm); Ak.SetBlock(1, 2, JmT);
+  Ak.SetBlock(2, 0,  Ju); Ak.SetBlock(2, 1,  Jm);
+
+  if(Hum != nullptr) { Ak.SetBlock(0, 1, Hum); Ak.SetBlock(1, 0, Hmu); }
+}
+
+
+// perturbed KKT system solve
+// determine the search direction
+void InteriorPointSolver::IPNewtonSolve(BlockVector &x, Vector &l, Vector &zl, Vector &zlhat, BlockVector &Xhat, double mu, bool socSolve)
+{
+  // solve A x = b, where A is the IP-Newton matrix
+  BlockOperator A(block_offsetsuml, block_offsetsuml); BlockVector b(block_offsetsuml); b = 0.0;
+  FormIPNewtonMat(x, l, zl, A);
+
+  //       [grad_u phi + Ju^T l]
+  // b = - [grad_m phi + Jm^T l]
+  //       [          c        ]
+  BlockVector gradphi(block_offsetsx); gradphi = 0.0;
+  BlockVector JTl(block_offsetsx); JTl = 0.0;
+  Dxphi(x, mu, gradphi);
+  
+  (A.GetBlock(0,2)).Mult(l, JTl.GetBlock(0));
+  (A.GetBlock(1,2)).Mult(l, JTl.GetBlock(1));
+
+  for(int ii = 0; ii < 2; ii++)
+  {
+    b.GetBlock(ii).Set(1.0, gradphi.GetBlock(ii));
+    b.GetBlock(ii).Add(1.0, JTl.GetBlock(ii));
+  }
+  if(!socSolve) 
+  {
+    optProblem->c(x, b.GetBlock(2));
+  }
+  else
+  {
+    b.GetBlock(2).Set(1.0, ckSoc);
+  }
+  b *= -1.0; 
+  Xhat = 0.0;
+
+
+  #ifdef MFEM_USE_SUITESPARSE
+    // Direct solve for IP-Newton saddle-point system
+    // A = [ [ Huu   0    Ju^T]
+    //       [  0    D     -I ]
+    //       [ Ju   -I      0 ]]
+    if(linSolver == 0)
+    {
+      BlockMatrix ABlockMatrix(block_offsetsuml, block_offsetsuml);
+      for(int ii = 0; ii < 3; ii++)
+      {
+        for(int jj = 0; jj < 3; jj++)
+        {
+          if(!A.IsZeroBlock(ii, jj))
+          {
+            ABlockMatrix.SetBlock(ii, jj, dynamic_cast<SparseMatrix *>(&(A.GetBlock(ii, jj))));
+          }
+        }
+      }
+      /* direct solve of the 3x3 IP-Newton linear system */
+      UMFPackSolver ASolver;
+      SparseMatrix *ASparse = ABlockMatrix.CreateMonolithic();
+      ASolver.SetOperator(*ASparse);
+      ASolver.Mult(b, Xhat);
+
+      Vector residual(Xhat.Size());
+      ASparse->Mult(Xhat, residual);
+      residual.Add(-1.0, b);
+      delete ASparse;
+    }
+    else if(linSolver == 1)
+    {
+      // Direct solve for 0,0 Schur complement of IP-Newton system, Huu + Ju^T Wmm Ju,
+      // where Wmm = D for contact problems
+      SparseMatrix * Huuloc = dynamic_cast<SparseMatrix *>(&(A.GetBlock(0, 0)));
+      SparseMatrix * Wmmloc = dynamic_cast<SparseMatrix *>(&(A.GetBlock(1, 1)));
+      SparseMatrix * Juloc  = dynamic_cast<SparseMatrix *>(&(A.GetBlock(2, 0)));
+      SparseMatrix * JuTloc = dynamic_cast<SparseMatrix *>(&(A.GetBlock(0, 2)));
+      Vector DVec(dimM); DVec = 0.0;
+      Vector one(dimM); one = 1.0;
+      D->Mult(one, DVec);
+      SparseMatrix *JuTDJu = Mult_AtDA(*Juloc, DVec);     // Ju^T D Ju
+      SparseMatrix *Areduced = Add(*Huuloc, *JuTDJu);  // Huu + Ju^T D Ju
+
+      
+      /* prepare the reduced rhs */
+      // breduced = bu + Ju^T (bm + Wmm bl)
+      Vector breduced(dimU); breduced = 0.0;
+      Vector tempVec(dimM); tempVec = 0.0;
+      Wmmloc->Mult(b.GetBlock(2), tempVec);
+      tempVec.Add(1.0, b.GetBlock(1));
+      JuTloc->Mult(tempVec, breduced);
+      breduced.Add(1.0, b.GetBlock(0));
+
+      // solve the reduced linear system
+      UMFPackSolver AreducedSolver;
+      AreducedSolver.SetOperator(*Areduced);
+      AreducedSolver.Mult(breduced, Xhat.GetBlock(0));
+
+      // now propagate solved uhat to obtain mhat and lhat
+      // xm = Ju xu - bl
+      Juloc->Mult(Xhat.GetBlock(0), Xhat.GetBlock(1));
+      Xhat.GetBlock(1).Add(-1.0, b.GetBlock(2));
+
+      // xl = Wmm xm - bm
+      Wmmloc->Mult(Xhat.GetBlock(1), Xhat.GetBlock(2));
+      Xhat.GetBlock(2).Add(-1.0, b.GetBlock(1));
+
+      delete JuTDJu;
+      delete Areduced;
+    }
+  #else
+    MFEM_VERIFY(linSolver > 1, "linSolver = 0, 1 require MFEM_USE_SUITESPARSE=YES");
+  #endif
+  if(linSolver > 1)
+  { 
+    // Iterative solve for 0,0 Schur complement of IP-Newton system, Huu + Ju^T Wmm Ju,
+    // where Wmm = D for contact problems
+    // here the iterative solver is a Jacobi-preconditioned CG-solve
+    SparseMatrix * Huuloc = dynamic_cast<SparseMatrix *>(&(A.GetBlock(0, 0)));
+    SparseMatrix * Wmmloc = dynamic_cast<SparseMatrix *>(&(A.GetBlock(1, 1)));
+    SparseMatrix * Juloc  = dynamic_cast<SparseMatrix *>(&(A.GetBlock(2, 0)));
+    SparseMatrix * JuTloc = dynamic_cast<SparseMatrix *>(&(A.GetBlock(0, 2)));
+    Vector DVec(dimM); DVec = 0.0;
+    Vector one(dimM); one = 1.0;
+    D->Mult(one, DVec);
+    SparseMatrix *JuTDJu   = Mult_AtDA(*Juloc, DVec);     // Ju^T D Ju
+    SparseMatrix *Areduced = Add(*Huuloc, *JuTDJu);  // Huu + Ju^T D Ju
+
+    /* prepare the reduced rhs */
+    // breduced = bu + Ju^T (bm + Wmm bl)
+    Vector breduced(dimU); breduced = 0.0;
+    Vector tempVec(dimM); tempVec = 0.0;
+    Wmmloc->Mult(b.GetBlock(2), tempVec);
+    tempVec.Add(1.0, b.GetBlock(1));
+    JuTloc->Mult(tempVec, breduced);
+    breduced.Add(1.0, b.GetBlock(0));
+    
+    if (linSolver == 2)
+    {
+      /* Jacobi preconditioned conjugate-gradient solve */
+      DSmoother AreducedPrec((SparseMatrix &)(*Areduced));
+      CGSolver AreducedSolver;
+      AreducedSolver.SetOperator(*Areduced);
+      AreducedSolver.SetAbsTol(1.e-12);
+      AreducedSolver.SetRelTol(1.e-8);
+      AreducedSolver.SetMaxIter(500);
+      AreducedSolver.SetPreconditioner(AreducedPrec);
+      AreducedSolver.SetPrintLevel(1);
+      AreducedSolver.Mult(breduced, Xhat.GetBlock(0));
+    }
+    else
+    {
+      /* Gauss-Seidel preconditioned GMRES solve */
+      GSSmoother AreducedPrec((SparseMatrix &)(*Areduced));
+      GMRESSolver AreducedSolver;
+      AreducedSolver.SetOperator(*Areduced);
+      AreducedSolver.SetAbsTol(1.e-12);
+      AreducedSolver.SetRelTol(1.e-8);
+      AreducedSolver.SetMaxIter(500);
+      AreducedSolver.SetPreconditioner(AreducedPrec);
+      AreducedSolver.SetPrintLevel(1);
+      AreducedSolver.Mult(breduced, Xhat.GetBlock(0));
+    }
+    
+    // now propagate solved uhat to obtain mhat and lhat
+    // xm = Ju xu - bl
+    Juloc->Mult(Xhat.GetBlock(0), Xhat.GetBlock(1));
+    Xhat.GetBlock(1).Add(-1.0, b.GetBlock(2));
+
+    // xl = Wmm xm - bm
+    Wmmloc->Mult(Xhat.GetBlock(1), Xhat.GetBlock(2));
+    Xhat.GetBlock(2).Add(-1.0, b.GetBlock(1));
+
+    delete JuTDJu;
+    delete Areduced;
+  }
+    
+  
+  /* backsolve to determine zlhat */
+  for(int ii = 0; ii < dimM; ii++)
+  {
+    zlhat(ii) = -1.*(zl(ii) + (zl(ii) * Xhat(ii + dimU) - mu) / (x(ii + dimU) - ml(ii)) );
+  }
+
+  // free memory
+  if(Hmm != nullptr)
+  {
+    delete Wmm;
+  }
+  if( Huucl != nullptr)
+  {
+    delete HLuu; HLuu = nullptr;
+  }
+  delete D;
+  delete JuT;
+  delete JmT;
+}
+
+// here Xhat, X will be BlockVectors w.r.t. the 4 partitioning X = (u, m, l, zl)
+
+void InteriorPointSolver::lineSearch(BlockVector& X0, BlockVector& Xhat, double mu)
+{
+  double tau  = max(tauMin, 1.0 - mu);
+  Vector u0   = X0.GetBlock(0);
+  Vector m0   = X0.GetBlock(1);
+  Vector l0   = X0.GetBlock(2);
+  Vector z0   = X0.GetBlock(3);
+  Vector uhat = Xhat.GetBlock(0);
+  Vector mhat = Xhat.GetBlock(1);
+  Vector lhat = Xhat.GetBlock(2);
+  Vector zhat = Xhat.GetBlock(3);
+  double alphaMax  = MaxStepSize(m0, ml, mhat, tau);
+  double alphaMaxz = MaxStepSize(z0, zhat, tau);
+  alphaz = alphaMaxz;
+
+
+  BlockVector x0(block_offsetsx); x0 = 0.0;
+  x0.GetBlock(0).Set(1.0, u0);
+  x0.GetBlock(1).Set(1.0, m0);
+  
+  BlockVector xhat(block_offsetsx); xhat = 0.0;
+  xhat.GetBlock(0).Set(1.0, uhat);
+  xhat.GetBlock(1).Set(1.0, mhat);
+  
+  BlockVector xtrial(block_offsetsx); xtrial = 0.0;
+  BlockVector Dxphi0(block_offsetsx); Dxphi0 = 0.0;
+  int maxBacktrack = 20;
+  alpha = alphaMax;
+
+
+  Vector ck0(dimC); ck0 = 0.0;
+  Vector zhatsoc(dimM); zhatsoc = 0.0;
+  BlockVector Xhatumlsoc(block_offsetsuml); Xhatumlsoc = 0.0;
+  BlockVector xhatsoc(block_offsetsx); xhatsoc = 0.0;
+  Vector uhatsoc(dimU); uhatsoc = 0.0;
+  Vector mhatsoc(dimM); mhatsoc = 0.0;
+
+  Dxphi(x0, mu, Dxphi0);
+  Dxphi0_xhat = InnerProduct(Dxphi0, xhat);
+  descentDirection = Dxphi0_xhat < 0. ? true : false;
+  if(descentDirection)
+  {
+    mfem::out << "is a descent direction for the log-barrier objective\n";
+  }
+  else
+  {
+    mfem::out << "is not a descent direction for the log-barrier objective\n";
+  }
+  mfem::out << "Dxphi^T xhat / (|| Dxphi||_2 * || xhat ||_2) = " << Dxphi0_xhat / (xhat.Norml2() * Dxphi0.Norml2()) << endl;
+  thx0 = theta(x0);
+  phx0 = phi(x0, mu);
+
+  lineSearchSuccess = false;
+  for(int i = 0; i < maxBacktrack; i++)
+  {
+    mfem::out << "\n--------- alpha = " << alpha << " ---------\n";
+
+    // ----- A-5.2. Compute trial point: xtrial = x0 + alpha_i xhat
+    xtrial.Set(1.0, x0);
+    xtrial.Add(alpha, xhat);
+
+    // ------ A-5.3. if not in filter region go to A.5.4 otherwise go to A-5.5.
+    thxtrial = theta(xtrial);
+    phxtrial = phi(xtrial, mu);
+
+    filterCheck(thxtrial, phxtrial);    
+    if(!inFilterRegion)
+    {
+      mfem::out << "not in filter region :)\n";
+      // ------ A.5.4: Check sufficient decrease
+      if(!descentDirection)
+      {
+        switchCondition = false;
+      }
+      else
+      {
+        switchCondition = (alpha * pow(abs(Dxphi0_xhat), sPhi) > delta * pow(thx0, sTheta)) ? true : false;
+      }
+      mfem::out << "theta(x0) = "     << thx0     << ", thetaMin = "                  << thetaMin             << endl;
+      mfem::out << "theta(xtrial) = " << thxtrial << ", (1-gTheta) *theta(x0) = "     << (1. - gTheta) * thx0 << endl;
+      mfem::out << "phi(xtrial) = "   << phxtrial << ", phi(x0) - gPhi *theta(x0) = " << phx0 - gPhi * thx0   << endl;
+      
+      // Case I      
+      if(thx0 <= thetaMin && switchCondition)
+      {
+        sufficientDecrease = phxtrial <= phx0 + eta * alpha * Dxphi0_xhat ? true : false;
+        if(sufficientDecrease)
+        {
+          mfem::out << "Accepted step length -- sufficient decrease in log-barrier objective.\n"; 
+          // accept the trial step
+          lineSearchSuccess = true;
+          break;
+        }
+      }
+      else
+      {
+        if(thxtrial <= (1. - gTheta) * thx0 || phxtrial <= phx0 - gPhi * thx0)
+        {
+          mfem::out << "Accepted step length -- decrease in either constraint violation or log-barrier objective.\n"; 
+          // accept the trial step
+          lineSearchSuccess = true;
+          break;
+        }
+      }
+      // A-5.5: Initialize the second-order correction
+      if((!(thx0 < thxtrial)) && i == 0)
+      {
+        mfem::out << "second order correction\n";
+        optProblem->c(xtrial, ckSoc);
+        optProblem->c(x0, ck0);
+        ckSoc.Add(alphaMax, ck0);
+        // A-5.6 Compute the second-order correction.
+        IPNewtonSolve(x0, l0, z0, zhatsoc, Xhatumlsoc, mu, true);
+        mhatsoc.Set(1.0, Xhatumlsoc.GetBlock(1));
+        // alphasoc = MaxStepSize(m0, ml, mhatsoc, tau);
+        //WARNING: not complete but currently solver isn't entering this region
+      }
+    }
+    else
+    {
+      mfem::out << "in filter region\n";
+    }
+
+    // include more if needed
+    alpha *= 0.5;
+
+  } 
+}
+
+
+void InteriorPointSolver::projectZ(const Vector &x, Vector &z, double mu)
+{
+  double zi;
+  double mudivmml;
+  for(int i = 0; i < dimM; i++)
+  {
+    zi = z(i);
+    mudivmml = mu / (x(i + dimU) - ml(i));
+    z(i) = max(min(zi, kSig * mudivmml), mudivmml / kSig);
+  }
+}
+
+void InteriorPointSolver::filterCheck(double th, double ph)
+{
+  inFilterRegion = false;
+  if(th > thetaMax)
+  {
+    inFilterRegion = true;
+  }
+  else
+  {
+    for(int i = 0; i < F1.Size(); i++)
+    {
+      if(th >= F1[i] && ph >= F2[i])
+      {
+        inFilterRegion = true;
+        break;
+      }
+    }
+  }
+}
+
+double InteriorPointSolver::E(const BlockVector &x, const Vector &l, const Vector &zl, double mu, bool print)
+{
+  double E1, E2, E3;
+  double sc, sd;
+  BlockVector gradL(block_offsetsx); gradL = 0.0; // stationarity grad L = grad f + J^T l - z
+  Vector cx(dimC); cx = 0.0;     // feasibility c = c(x)
+  Vector comp(dimM); comp = 0.0; // complementarity M Z - mu 1
+
+  DxL(x, l, zl, gradL);
+  E1 = gradL.Normlinf();
+
+  optProblem->c(x, cx);
+  E2 = cx.Normlinf();
+
+  for(int ii = 0; ii < dimM; ii++) 
+  { 
+    comp(ii) = x(dimU + ii) * zl(ii) - mu;
+  }
+  E3 = comp.Normlinf();
+
+  double ll1, zl1;
+  zl1 = zl.Norml1() / double(dimC + dimM);
+  ll1 =  l.Norml1();
+  sc = max(sMax, zl1 / (double(dimM)) ) / sMax;
+  sd = max(sMax, (ll1 + zl1) / (double(dimC + dimM))) / sMax;
+  if(print)
+  {
+    mfem::out << "evaluating optimality error for mu = " << mu << endl;
+    mfem::out << "stationarity measure = "    << E1 / sd << endl;
+    mfem::out << "feasibility measure  = "    << E2      << endl;
+    mfem::out << "complimentarity measure = " << E3 / sc << endl;
+  }
+  return max(max(E1 / sd, E2), E3 / sc);
+}
+
+double InteriorPointSolver::E(const BlockVector &x, const Vector &l, const Vector &zl, bool print)
+{
+  return E(x, l, zl, 0.0, print);
+}
+
+double InteriorPointSolver::theta(const BlockVector &x)
+{
+  Vector cx(dimC); cx = 0.0;
+  optProblem->c(x, cx);
+  return cx.Norml2();
+}
+
+// log-barrier objective
+double InteriorPointSolver::phi(const BlockVector &x, double mu)
+{
+  double fx = optProblem->CalcObjective(x); 
+  double logBarrierLoc = 0.0;
+  for(int i = 0; i < dimM; i++) 
+  { 
+    logBarrierLoc += log(x(dimU+i)-ml(i));
+  }
+  double logBarrierGlb = 0.0;
+  logBarrierGlb = logBarrierLoc;
+  return fx - mu * logBarrierGlb;
+}
+
+
+
+
+// gradient of log-barrier objective with respect to x = (u, m)
+void InteriorPointSolver::Dxphi(const BlockVector &x, double mu, BlockVector &y)
+{
+  optProblem->CalcObjectiveGrad(x, y);
+  for(int i = 0; i < dimM; i++) 
+  { 
+    y(dimU + i) -= mu / (x(dimU + i));
+  } 
+}
+
+
+// Lagrangian function evaluation
+// L(x, l, zl) = f(x) + l^T c(x) - zl^T m
+double InteriorPointSolver::L(const BlockVector &x, const Vector &l, const Vector &zl)
+{
+  double fx = optProblem->CalcObjective(x);
+  Vector cx(dimC); optProblem->c(x, cx);
+  return (fx + InnerProduct(cx, l) - InnerProduct(x.GetBlock(1), zl));
+}
+
+void InteriorPointSolver::DxL(const BlockVector &x, const Vector &l, const Vector &zl, BlockVector &y)
+{
+  // evaluate the gradient of the objective with respect to the primal variables x = (u, m)
+  BlockVector gradxf(block_offsetsx); gradxf = 0.0;
+  optProblem->CalcObjectiveGrad(x, gradxf);
+  
+  SparseMatrix *Jacu, *Jacm, *JacuT, *JacmT;
+  Jacu = optProblem->Duc(x); Jacm = optProblem->Dmc(x);
+  JacuT = Transpose(*Jacu);
+  JacmT = Transpose(*Jacm);
+  JacuT->Mult(l, y.GetBlock(0));
+  JacmT->Mult(l, y.GetBlock(1));
+  delete JacuT; 
+  delete JacmT;
+  y.Add(1.0, gradxf);
+  (y.GetBlock(1)).Add(-1.0, zl);
+}
+
+
+bool InteriorPointSolver::GetConverged() const
+{
+  return converged;
+}
+
+void InteriorPointSolver::SetTol(double Tol)
+{
+  rel_tol = Tol;
+}
+
+void InteriorPointSolver::SetMaxIter(int max_it)
+{
+  max_iter = max_it;
+}
+
+void InteriorPointSolver::SetBarrierParameter(double mu_0)
+{
+  mu_k = mu_0;
+}
+
+void InteriorPointSolver::SaveLogBarrierHessianIterates(bool save)
+{
+  MFEM_ASSERT(MyRank == 0 || save == false, "currently can only save logbarrier hessian in serial codes");
+  saveLogBarrierIterates = save;
+}
+
+void InteriorPointSolver::SetLinearSolver(int LinSolver)
+{
+  linSolver = LinSolver;
+}
+
+
+
+InteriorPointSolver::~InteriorPointSolver() 
+{
+  F1.DeleteAll();
+  F2.DeleteAll();
+  block_offsetsx.DeleteAll();
+  block_offsetsumlz.DeleteAll();
+  block_offsetsuml.DeleteAll();
+  ml.SetSize(0);
+}
@@ -0,0 +1,80 @@
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+#ifndef IPSOLVER 
+#define IPSOLVER
+
+class InteriorPointSolver
+{
+protected:
+    GeneralOptProblem* optProblem;
+    double rel_tol;
+    int  max_iter;
+    double mu_k; // \mu_k
+    Vector lk, zlk;
+
+    double sMax, kSig, tauMin, eta, thetaMin, delta, sTheta, sPhi, kMu, thetaMu;
+    double thetaMax, kSoc, gTheta, gPhi, kEps;
+	
+    // filter
+    Array<double> F1, F2;
+	
+    // quantities computed in lineSearch
+    double alpha, alphaz;
+    double thx0, thxtrial;
+    double phx0, phxtrial;
+    bool descentDirection, switchCondition, sufficientDecrease, lineSearchSuccess, inFilterRegion;
+    double Dxphi0_xhat;
+
+    int dimU, dimM, dimC;
+    Array<int> block_offsetsumlz, block_offsetsuml, block_offsetsx;
+    Vector ml;
+
+    Vector ckSoc;
+    SparseMatrix * Huu, * Hum, * Hmu, * Hmm, * Wmm, *D, * Ju, * Jm, * JuT, * JmT;
+    SparseMatrix * Huucl, *HLuu;    
+    int jOpt;
+    bool converged;
+    
+    int MyRank;
+    bool iAmRoot;
+
+    bool saveLogBarrierIterates;
+
+    int linSolver;
+
+public:
+    InteriorPointSolver(GeneralOptProblem*);
+    void Mult(const BlockVector& , BlockVector&); // used when the user wants to be aware of bound-constrained variable m >= ml
+    void Mult(const Vector&, Vector &); // useful when the user doesn't need to know about bound-constrained variable   m >= ml, e.g., when m is a slack variable
+    double MaxStepSize(Vector& , Vector& , Vector& , double);
+    double MaxStepSize(Vector& , Vector& , double);
+    void FormIPNewtonMat(BlockVector& , Vector& , Vector& , BlockOperator &);
+    void IPNewtonSolve(BlockVector& , Vector& , Vector& , Vector&, BlockVector& , double, bool);
+    void lineSearch(BlockVector& , BlockVector& , double);
+    void projectZ(const Vector & , Vector &, double);
+    void filterCheck(double, double);
+    double E(const BlockVector &, const Vector &, const Vector &, double, bool);
+    double E(const BlockVector &, const Vector &, const Vector &, bool);
+    bool GetConverged() const;
+    // TO DO: include Hessian of Lagrangian
+    double theta(const BlockVector &);
+    double phi(const BlockVector &, double);
+    void Dxphi(const BlockVector &, double, BlockVector &);
+    double L(const BlockVector &, const Vector &, const Vector &);
+    void DxL(const BlockVector &, const Vector &, const Vector &, BlockVector &);
+    void SetTol(double);
+    void SetMaxIter(int);
+    void SetBarrierParameter(double);    
+    void SaveLogBarrierHessianIterates(bool);
+    void SetLinearSolver(int);
+    virtual ~InteriorPointSolver();
+};
+
+#endif
@@ -0,0 +1,125 @@
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include "IPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+
+double dmanufacturedFun(const Vector &);
+double fRhs(const Vector &);
+double obstacle(const Vector &);
+
+int main(int argc, char *argv[])
+{
+  int FEorder = 1; // order of the finite elements
+  int linSolver = 0;
+  int maxIPMiters = 30;
+  bool iAmRoot = true;
+  
+  OptionsParser args(argc, argv);
+  args.AddOption(&FEorder, "-o", "--order",\
+		  "Order of the finite elements.");
+  args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+       "IP-Newton linear system solution strategy.");
+  args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+		  "Maximum number of IPM iterations");
+  
+  args.Parse();
+  if(!args.Good())
+  {
+    args.PrintUsage(cout);
+    return 1;
+  }
+  else
+  {
+    if( iAmRoot )
+    {
+      args.PrintOptions(cout);
+    }
+  }
+
+  const char *meshFile = "../../data/inline-quad.mesh";
+  Mesh *mesh = new Mesh(meshFile, 1, 1);
+  int dim = mesh->Dimension(); // geometric dimension of the domain
+  {
+     int ref_levels = 3;
+     for (int l = 0; l < ref_levels; l++)
+     {
+        mesh->UniformRefinement();
+     }
+  }
+
+  FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+  FiniteElementSpace      *Vh  = new FiniteElementSpace(mesh, fec);
+  ObstacleProblem problem(Vh, &fRhs, &obstacle);
+  
+  int dimD = problem.GetDimD();
+  Vector x0(dimD); x0 = 0.0;
+  Vector xf(dimD); xf = 0.0;
+
+  InteriorPointSolver optimizer(&problem); 
+  optimizer.SetTol(1.e-7);
+  optimizer.SetLinearSolver(linSolver);
+  optimizer.SetMaxIter(maxIPMiters);
+  optimizer.Mult(x0, xf);
+
+  double Einitial = problem->E(x0);
+  double Efinal = problem->E(xf);
+  cout << "Energy objective at initial point = " << Einitial << endl;
+  cout << "Energy objective at QP optimizer = " << Efinal << endl;
+
+
+  GridFunction d_gf(Vh);
+
+  d_gf = xf;
+
+  FunctionCoefficient dm_fc(dmanufacturedFun); // pseudo-manufactured solution
+  GridFunction dm_gf(Vh);
+  dm_gf.ProjectCoefficient(dm_fc);
+  
+  ParaViewDataCollection paraview_dc("BarrierProblemSolution", mesh);
+  paraview_dc.SetPrefixPath("ParaView");
+  paraview_dc.SetLevelsOfDetail(FEorder);
+  paraview_dc.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc.SetHighOrderOutput(true);
+  paraview_dc.SetCycle(0);
+  paraview_dc.SetTime(0.0);
+  paraview_dc.RegisterField("d(x) (numerical)", &d_gf);
+  paraview_dc.RegisterField("d(x) (pseudo-manufactured)", &dm_gf);
+  paraview_dc.Save();
+  
+  
+  delete Vh;
+  delete fec;
+  delete mesh;
+  return 0;
+}
+
+
+double dmanufacturedFun(const Vector &x)
+{
+  return cos(2*M_PI*x(0)) + 0.2 - 2.0*(pow(x(0),3) - 1.5*pow(x(0),2));
+}
+
+
+// f(x) forcing term... which enters the objective energy functional
+// E(d) = 0.5 d^T K d - f^T d, where f is a discrete vector representation
+// of f(x). f(x) is such that in the absence of bound-constraints then
+// the solution of the optimization problem satisfies the PDE
+// -div(grad(d)) + d = f + homogeneous Neumann conditions on the unit interval,
+// for d(x) = cos(2 \pi x) + a0 + a3 (x^3 - 1.5 x^2), a2 = 0.2, a3 = -2
+double fRhs(const Vector &x)
+{
+  double fx = 0.;
+  fx = 0.2 - 2.0 * (pow(x(0),3)- 1.5*pow(x(0),2.) - 6 * x(0) + 3.) + (1. + pow(2.*M_PI,2))*cos(2.*M_PI*x(0));
+  return fx;
+}
+
+double obstacle(const Vector &x)
+{
+  return 0.0;
+}
@@ -0,0 +1,834 @@
+#include "mfem.hpp"
+#include "ParIPsolver.hpp"
+#include "ParProblems.hpp"
+#include <fstream>
+#include <iostream>
+#include <cstdlib>
+
+using namespace std;
+using namespace mfem;
+
+
+ParInteriorPointSolver::ParInteriorPointSolver(ParGeneralOptProblem * problem_) 
+                     : problem(problem_), 
+                       block_offsetsumlz(5), block_offsetsuml(4), block_offsetsx(3),
+                       Huu(nullptr), Hum(nullptr), Hmu(nullptr), 
+                       Hmm(nullptr), Wmm(nullptr), D(nullptr), 
+                       Ju(nullptr), Jm(nullptr), JuT(nullptr), JmT(nullptr), 
+                       saveLogBarrierIterates(false)
+{
+   OptTol  = 1.e-2;
+   max_iter = 20;
+   mu_k     = 1.0;
+
+   sMax     = 1.e2;
+   kSig     = 1.e10;   // control deviation from primal Hessian
+   tauMin   = 0.8;     // control rate at which iterates can approach the boundary
+   eta      = 1.e-4;   // backtracking constant
+   thetaMin = 1.e-4;   // allowed violation of the equality constraints
+
+   // constants in line-step A-5.4
+   delta    = 1.0;
+   sTheta   = 1.1;
+   sPhi     = 2.3;
+
+   // control the rate at which the penalty parameter is decreased
+   kMu     = 0.2;
+   thetaMu = 1.5;
+
+   thetaMax = 1.e6; // maximum constraint violation
+   // data for the second order correction
+   kSoc     = 0.99;
+
+   // equation (18)
+   gTheta = 1.e-5;
+   gPhi   = 1.e-5;
+
+   kEps   = 1.e1;
+
+   dimU = problem->GetDimU();
+   dimM = problem->GetDimM();
+   dimC = problem->GetDimC();
+   MPI_Allreduce(&dimU, &dimUglb, 1, MPI_INT, MPI_SUM, MPI_COMM_WORLD);
+   MPI_Allreduce(&dimM, &dimMglb, 1, MPI_INT, MPI_SUM, MPI_COMM_WORLD);
+   MPI_Allreduce(&dimC, &dimCglb, 1, MPI_INT, MPI_SUM, MPI_COMM_WORLD);
+   ckSoc.SetSize(dimC);
+  
+   block_offsetsumlz[0] = 0;
+   block_offsetsumlz[1] = dimU; // u
+   block_offsetsumlz[2] = dimM; // m
+   block_offsetsumlz[3] = dimC; // lambda
+   block_offsetsumlz[4] = dimM; // zl
+   block_offsetsumlz.PartialSum();
+
+   for(int i = 0; i < block_offsetsuml.Size(); i++)  
+   { 
+      block_offsetsuml[i] = block_offsetsumlz[i]; 
+   }
+   for(int i = 0; i < block_offsetsx.Size(); i++)    
+   { 
+      block_offsetsx[i] = block_offsetsuml[i] ; 
+   }
+
+  
+   ml = problem->Getml();
+  
+   lk.SetSize(dimC);  lk  = 0.0;
+   zlk.SetSize(dimM); zlk = 0.0;
+
+   linSolver = 0;
+   linSolveTol = 1.e-8;
+   MyRank = Mpi::WorldRank();
+   iAmRoot = MyRank == 0 ? true : false;
+}
+
+double ParInteriorPointSolver::MaxStepSize(Vector &x, Vector &xl, Vector &xhat, double tau)
+{
+   double alphaMaxloc = 1.0;
+   double alphaTmp;
+   for(int i = 0; i < x.Size(); i++)
+   {   
+      if( xhat(i) < 0. )
+      {
+         alphaTmp = -1. * tau * (x(i) - xl(i)) / xhat(i);
+         alphaMaxloc = min(alphaMaxloc, alphaTmp);
+      } 
+   }
+
+   // alphaMaxloc is the local maximum step size which is
+   // distinct on each MPI process. Need to compute
+   // the global maximum step size 
+   double alphaMaxglb;
+   MPI_Allreduce(&alphaMaxloc, &alphaMaxglb, 1, MPI_DOUBLE, MPI_MIN, MPI_COMM_WORLD);
+   return alphaMaxglb;
+}
+
+double ParInteriorPointSolver::MaxStepSize(Vector &x, Vector &xhat, double tau)
+{
+   Vector zero(x.Size()); zero = 0.0;
+   return MaxStepSize(x, zero, xhat, tau);
+}
+
+
+void ParInteriorPointSolver::Mult(const Vector &x0, Vector &xf)
+{
+   BlockVector x0block(block_offsetsx); x0block = 0.0;
+   x0block.GetBlock(0).Set(1.0, x0);
+   x0block.GetBlock(1) = 100.;
+   x0block.GetBlock(1).Add(1.0, ml);
+   BlockVector xfblock(block_offsetsx); xfblock = 0.0;
+   Mult(x0block, xfblock);
+   xf.Set(1.0, xfblock.GetBlock(0));
+}
+
+
+void ParInteriorPointSolver::Mult(const BlockVector &x0, BlockVector &xf)
+{
+   converged = false;
+   
+   BlockVector xk(block_offsetsx), xhat(block_offsetsx); xk = 0; xhat = 0.0;
+   BlockVector Xk(block_offsetsumlz), Xhat(block_offsetsumlz); Xk = 0.0; Xhat = 0.0;
+   BlockVector Xhatuml(block_offsetsuml); Xhatuml = 0.0;
+   Vector zlhat(dimM); zlhat = 0.0;
+
+   xk.GetBlock(0).Set(1.0, x0.GetBlock(0));
+   xk.GetBlock(1).Set(1.0, x0.GetBlock(1));
+   // running estimate of the final values of the Lagrange multipliers
+   lk  = 0.0;
+   zlk = 0.0;
+
+   for(int i = 0; i < dimM; i++)
+   {
+      zlk(i) = 1.e1 * mu_k / (xk(i+dimU) - ml(i));
+   }
+
+   Xk.GetBlock(0).Set(1.0, xk.GetBlock(0));
+   Xk.GetBlock(1).Set(1.0, xk.GetBlock(1));
+   Xk.GetBlock(2).Set(1.0, lk);
+   Xk.GetBlock(3).Set(1.0, zlk);
+
+  /* set theta0 = theta(x0)
+   *     thetaMin
+   *     thetaMax
+   * when theta(xk) < thetaMin and the switching condition holds
+   * then we ask for the Armijo sufficient decrease of the barrier
+   * objective to be satisfied, in order to accept the trial step length alphakl
+   * 
+   * thetaMax controls how the filter is initialized for each log-barrier subproblem
+   * F0 = {(th, phi) s.t. th > thetaMax}
+   * that is the filter does not allow for iterates where the constraint violation
+   * is larger than that of thetaMax
+   */
+   double theta0 = theta(xk);
+   thetaMin = 1.e-4 * max(1.0, theta0);
+   thetaMax = 1.e8  * thetaMin; // 1.e4 * max(1.0, theta0)
+
+   double Eeval, maxBarrierSolves, Eevalmu0;
+   bool printOptimalityError; // control optimality error print to console for log-barrier subproblems
+   
+   maxBarrierSolves = 10;
+
+   for(jOpt = 0; jOpt < max_iter; jOpt++)
+   {
+      if(iAmRoot)
+      {
+         cout << "interior-point solve step " << jOpt << endl;
+      }
+      // A-2. Check convergence of overall optimization problem
+      printOptimalityError = false;
+      Eevalmu0 = E(xk, lk, zlk, printOptimalityError);
+      if(Eevalmu0 < OptTol)
+      {
+         converged = true;
+         if(iAmRoot)
+         {
+            cout << "solved optimization problem :)\n";
+         }
+         break;
+      }
+      
+      if(jOpt > 0) { maxBarrierSolves = 1; }
+      
+      for(int i = 0; i < maxBarrierSolves; i++)
+      {
+         // A-3. Check convergence of the barrier subproblem
+         printOptimalityError = true;
+         Eeval = E(xk, lk, zlk, mu_k, printOptimalityError);
+         if(iAmRoot)
+         {
+            cout << "E = " << Eeval << endl;
+         }
+         if(Eeval < kEps * mu_k)
+         {
+            if(iAmRoot)
+            {
+               cout << "solved barrier subproblem :), for mu = " << mu_k << endl;
+            }
+            // A-3.1. Recompute the barrier parameter
+            mu_k  = max(OptTol / 10., min(kMu * mu_k, pow(mu_k, thetaMu)));
+            // A-3.2. Re-initialize the filter
+            F1.DeleteAll();
+            F2.DeleteAll();
+         }
+         else
+         {
+            break;
+         }
+      }
+    
+      // A-4. Compute the search direction
+      // solve for (uhat, mhat, lhat)
+      if(iAmRoot)
+      {
+         cout << "\n** A-4. IP-Newton solve **\n";
+      }
+      zlhat = 0.0; Xhatuml = 0.0;
+      // why do we have Xhatuml ....???
+      // TO DO: remove Xhatuml in favor of passing Xhat
+      IPNewtonSolve(xk, lk, zlk, zlhat, Xhatuml, mu_k, false); 
+
+      // assign data stack, X = (u, m, l, zl)
+      Xk = 0.0;
+      Xk.GetBlock(0).Set(1.0, xk.GetBlock(0));
+      Xk.GetBlock(1).Set(1.0, xk.GetBlock(1));
+      Xk.GetBlock(2).Set(1.0, lk);
+      Xk.GetBlock(3).Set(1.0, zlk);
+
+      // assign data stack, Xhat = (uhat, mhat, lhat, zlhat)
+      Xhat = 0.0;
+      for(int i = 0; i < 3; i++)
+      {
+         Xhat.GetBlock(i).Set(1.0, Xhatuml.GetBlock(i));
+      }
+      Xhat.GetBlock(3).Set(1.0, zlhat);
+
+      // A-5. Backtracking line search.
+      if(iAmRoot)
+      {
+         cout << "\n** A-5. Linesearch **\n";
+         cout << "mu = " << mu_k << endl;
+      }
+      lineSearch(Xk, Xhat, mu_k);
+
+      if(lineSearchSuccess)
+      {
+         if(iAmRoot)
+         {
+            cout << "lineSearch successful :)\n";
+         }
+         if(!switchCondition || !sufficientDecrease)
+         {
+            F1.Append( (1. - gTheta) * thx0);
+            F2.Append( phx0 - gPhi * thx0);
+         }
+         // ----- A-6: Accept the trial point
+         // print info regarding zl...
+         xk.GetBlock(0).Add(alpha, Xhat.GetBlock(0));
+         xk.GetBlock(1).Add(alpha, Xhat.GetBlock(1));
+         lk.Add(alpha,   Xhat.GetBlock(2));
+         zlk.Add(alphaz, Xhat.GetBlock(3));
+         projectZ(xk, zlk, mu_k);
+      }
+      else
+      {
+         if(iAmRoot)
+         {
+            cout << "lineSearch not successful :(\n";
+            cout << "attempting feasibility restoration with theta = " << thx0 << endl;
+            cout << "no feasibility restoration implemented, exiting now \n";
+         }
+         break;
+      }
+      if(jOpt + 1 == max_iter && iAmRoot) 
+      {  
+         cout << "maximum optimization iterations :(\n";
+      }
+   }
+   // done with optimization routine, just reassign data to xf reference so
+   // that the application code has access to the optimal point
+   xf = 0.0;
+   xf.GetBlock(0).Set(1.0, xk.GetBlock(0));
+   xf.GetBlock(1).Set(1.0, xk.GetBlock(1));
+}
+
+void ParInteriorPointSolver::FormIPNewtonMat(BlockVector & x, Vector & l, Vector &zl, BlockOperator &Ak)
+{
+   // WARNING: Huu, Hum, Hmu, Hmm should all be Hessian terms of the Lagrangian, currently we 
+   //          them by Hessian terms of the objective function and neglect the Hessian of l^T c
+
+   Huu = problem->Duuf(x); 
+   Hum = problem->Dumf(x);
+   Hmu = problem->Dmuf(x);
+   Hmm = problem->Dmmf(x);
+
+   Vector DiagLogBar(dimM); DiagLogBar = 0.0;
+   for(int ii = 0; ii < dimM; ii++)
+   {
+      DiagLogBar(ii) = zl(ii) / (x(ii+dimU) - ml(ii));
+   }
+   if(saveLogBarrierIterates)
+   {
+      std::ofstream diagStream;
+      char diagString[100];
+      snprintf(diagString, 100, "logBarrierHessiandata/D%d.dat", jOpt);
+      diagStream.open(diagString, ios::out | ios::trunc);
+      for(int ii = 0; ii < dimM; ii++)
+      {
+         diagStream << setprecision(30) << DiagLogBar(ii) << endl;
+      }
+      diagStream.close();
+   } 
+
+   SparseMatrix * Ds = new SparseMatrix(DiagLogBar);
+   ParFiniteElementSpace * fes = problem->GetfesM();
+   D = new HypreParMatrix(fes->GetComm(), fes->GlobalTrueVSize(), fes->GetTrueDofOffsets(), Ds);
+   HypreStealOwnership(*D,*Ds);
+   delete Ds;
+  
+   if(Hmm != nullptr)
+   {
+      Wmm = Hmm;
+      Wmm->Add(1.0, *D);
+   }
+   else
+   {
+      Wmm = D;
+   }
+
+   Ju = problem->Duc(x); JuT = Ju->Transpose();
+   Jm = problem->Dmc(x); JmT = Jm->Transpose();
+  
+   //         IP-Newton system matrix
+   //    Ak = [[H_(u,u)  H_(u,m)   J_u^T]
+   //          [H_(m,u)  W_(m,m)   J_m^T]
+   //          [ J_u      J_m       0  ]]
+
+   Ak.SetBlock(0, 0, Huu);                         Ak.SetBlock(0, 2, JuT);
+                           Ak.SetBlock(1, 1, Wmm); Ak.SetBlock(1, 2, JmT);
+   Ak.SetBlock(2, 0,  Ju); Ak.SetBlock(2, 1,  Jm);
+
+   if(Hum != nullptr) { Ak.SetBlock(0, 1, Hum); Ak.SetBlock(1, 0, Hmu); }
+}
+
+// perturbed KKT system solve
+// determine the search direction
+void ParInteriorPointSolver::IPNewtonSolve(BlockVector &x, Vector &l, Vector &zl, Vector &zlhat, BlockVector &Xhat, double mu, bool socSolve)
+{
+   // solve A x = b, where A is the IP-Newton matrix
+   BlockOperator A(block_offsetsuml, block_offsetsuml); BlockVector b(block_offsetsuml); b = 0.0;
+   FormIPNewtonMat(x, l, zl, A);
+
+   //       [grad_u phi + Ju^T l]
+   // b = - [grad_m phi + Jm^T l]
+   //       [          c        ]
+   BlockVector gradphi(block_offsetsx); gradphi = 0.0;
+   BlockVector JTl(block_offsetsx); JTl = 0.0;
+   Dxphi(x, mu, gradphi);
+   
+   (A.GetBlock(0,2)).Mult(l, JTl.GetBlock(0));
+   (A.GetBlock(1,2)).Mult(l, JTl.GetBlock(1));
+
+   for(int ii = 0; ii < 2; ii++)
+   {
+      b.GetBlock(ii).Set(1.0, gradphi.GetBlock(ii));
+      b.GetBlock(ii).Add(1.0, JTl.GetBlock(ii));
+   }
+   if(!socSolve) 
+   {
+      problem->c(x, b.GetBlock(2));
+   }
+   else
+   {
+      b.GetBlock(2).Set(1.0, ckSoc);
+   }
+   b *= -1.0; 
+   Xhat = 0.0;
+
+
+   // Direct solver (default)
+   if(linSolver == 0)
+   {
+      Array2D<HypreParMatrix *> ABlockMatrix(3,3);
+      for(int ii = 0; ii < 3; ii++)
+      {
+      for(int jj = 0; jj < 3; jj++)
+      {
+         if(!A.IsZeroBlock(ii, jj))
+         {
+            ABlockMatrix(ii, jj) = dynamic_cast<HypreParMatrix *>(&(A.GetBlock(ii, jj)));
+	 }
+	 else
+	 {
+	    ABlockMatrix(ii, jj) = nullptr;
+	 }
+      }
+      }
+      
+      HypreParMatrix * Ah = HypreParMatrixFromBlocks(ABlockMatrix);   
+      
+      /* direct solve of the 3x3 IP-Newton linear system */
+      #ifdef MFEM_USE_MUMPS
+        MUMPSSolver ASolver;
+        ASolver.SetPrintLevel(0);
+        ASolver.SetMatrixSymType(MUMPSSolver::MatType::SYMMETRIC_INDEFINITE);
+        ASolver.SetOperator(*Ah);
+        ASolver.Mult(b, Xhat);
+      #else 
+        #ifdef MFEM_USE_MKL_CPARDISO
+          CPardisoSolver ASolver(MPI_COMM_WORLD);
+          ASolver.SetOperator(*Ah);
+          ASolver.Mult(b, Xhat);
+	#else
+	  MFEM_VERIFY(false, "linSolver 0 will not work unless compiled with MUMPS or MKL");
+        #endif
+      #endif
+
+      delete Ah;
+   }
+   else if(linSolver == 1 || linSolver == 2)
+   {
+      // form A = Huu + Ju^T D Ju, Wmm = D for contact
+      HypreParMatrix * Huuloc = dynamic_cast<HypreParMatrix *>(&(A.GetBlock(0, 0)));
+      HypreParMatrix * Wmmloc = dynamic_cast<HypreParMatrix *>(&(A.GetBlock(1, 1)));
+      HypreParMatrix * Juloc  = dynamic_cast<HypreParMatrix *>(&(A.GetBlock(2, 0)));
+      HypreParMatrix * JuTloc = dynamic_cast<HypreParMatrix *>(&(A.GetBlock(0, 2)));
+      
+      
+      HypreParMatrix *JuTDJu   = RAP(Wmmloc, Juloc);     // Ju^T D Ju
+      HypreParMatrix *Areduced = ParAdd(Huuloc, JuTDJu);  // Huu + Ju^T D Ju
+      /* prepare the reduced rhs */
+      // breduced = bu + Ju^T (bm + Wmm bl)
+      Vector breduced(dimU); breduced = 0.0;
+      Vector tempVec(dimM); tempVec = 0.0;
+      Wmmloc->Mult(b.GetBlock(2), tempVec);
+      tempVec.Add(1.0, b.GetBlock(1));
+      JuTloc->Mult(tempVec, breduced);
+      breduced.Add(1.0, b.GetBlock(0));
+      
+      if(linSolver == 1)
+      {
+         // setup the solver for the reduced linear system
+         #ifdef MFEM_USE_MUMPS
+	   MUMPSSolver AreducedSolver;   
+           AreducedSolver.SetPrintLevel(0);
+           AreducedSolver.SetMatrixSymType(MUMPSSolver::MatType::SYMMETRIC_INDEFINITE);
+	   AreducedSolver.SetOperator(*Areduced);
+	   AreducedSolver.Mult(breduced, Xhat.GetBlock(0));
+         #else 
+           #ifdef MFEM_USE_MKL_CPARDISO
+	     CPardisoSolver AreducedSolver(MPI_COMM_WORLD);
+	     AreducedSolver.SetOperator(*Areduced);
+	     AreducedSolver.Mult(breduced, Xhat.GetBlock(0));
+           #else
+	     MFEM_VERIFY(false, "linSolver 1 will not work unless compiled with MUMPS or MKL");
+           #endif
+         #endif
+      }
+      else
+      {
+         HyprePCG AreducedSolver(MPI_COMM_WORLD);
+	 AreducedSolver.SetOperator(*Areduced);
+	 HypreBoomerAMG AreducedPrec;
+         AreducedSolver.SetTol(linSolveTol);
+         AreducedSolver.SetMaxIter(500);
+         AreducedSolver.SetPreconditioner(AreducedPrec);
+         AreducedSolver.SetResidualConvergenceOptions(); // convergence criteria based on residual norm
+	 AreducedSolver.SetPrintLevel(2);
+         AreducedSolver.Mult(breduced, Xhat.GetBlock(0));
+      }
+
+      // now propagate solved uhat to obtain mhat and lhat
+      // xm = Ju xu - bl
+      Juloc->Mult(Xhat.GetBlock(0), Xhat.GetBlock(1));
+      Xhat.GetBlock(1).Add(-1.0, b.GetBlock(2));
+
+      // xl = Wmm xm - bm
+      Wmmloc->Mult(Xhat.GetBlock(1), Xhat.GetBlock(2));
+      Xhat.GetBlock(2).Add(-1.0, b.GetBlock(1));
+
+      delete JuTDJu;
+      delete Areduced;
+   }
+
+   /* backsolve to determine zlhat */
+   for(int ii = 0; ii < dimM; ii++)
+   {
+      zlhat(ii) = -1.*(zl(ii) + (zl(ii) * Xhat(ii + dimU) - mu) / (x(ii + dimU) - ml(ii)) );
+   }
+
+   // free memory
+   delete D;
+   delete JuT;
+   delete JmT;
+   if(Hmm != nullptr)
+   {
+      delete Wmm;
+   }
+}
+
+// here Xhat, X will be BlockVectors w.r.t. the 4 partitioning X = (u, m, l, zl)
+
+void ParInteriorPointSolver::lineSearch(BlockVector& X0, BlockVector& Xhat, double mu)
+{
+   double tau  = max(tauMin, 1.0 - mu);
+   Vector u0   = X0.GetBlock(0);
+   Vector m0   = X0.GetBlock(1);
+   Vector l0   = X0.GetBlock(2);
+   Vector z0   = X0.GetBlock(3);
+   Vector uhat = Xhat.GetBlock(0);
+   Vector mhat = Xhat.GetBlock(1);
+   Vector lhat = Xhat.GetBlock(2);
+   Vector zhat = Xhat.GetBlock(3);
+   double alphaMax  = MaxStepSize(m0, ml, mhat, tau);
+   double alphaMaxz = MaxStepSize(z0, zhat, tau);
+   alphaz = alphaMaxz;
+
+   BlockVector x0(block_offsetsx); x0 = 0.0;
+   x0.GetBlock(0).Set(1.0, u0);
+   x0.GetBlock(1).Set(1.0, m0);
+   
+   BlockVector xhat(block_offsetsx); xhat = 0.0;
+   xhat.GetBlock(0).Set(1.0, uhat);
+   xhat.GetBlock(1).Set(1.0, mhat);
+   
+   BlockVector xtrial(block_offsetsx); xtrial = 0.0;
+   BlockVector Dxphi0(block_offsetsx); Dxphi0 = 0.0;
+   int maxBacktrack = 20;
+   alpha = alphaMax;
+
+   Vector ck0(dimC); ck0 = 0.0;
+   Vector zhatsoc(dimM); zhatsoc = 0.0;
+   BlockVector Xhatumlsoc(block_offsetsuml); Xhatumlsoc = 0.0;
+   BlockVector xhatsoc(block_offsetsx); xhatsoc = 0.0;
+   Vector uhatsoc(dimU); uhatsoc = 0.0;
+   Vector mhatsoc(dimM); mhatsoc = 0.0;
+
+   Dxphi(x0, mu, Dxphi0);
+
+   Dxphi0_xhat = InnerProduct(MPI_COMM_WORLD, Dxphi0, xhat);
+   descentDirection = Dxphi0_xhat < 0. ? true : false;
+   if(descentDirection)
+   {
+      if (iAmRoot)
+      {
+         cout << "is a descent direction for the log-barrier objective\n";
+      }
+   }
+   else
+   {
+      cout << "is not a descent direction for the log-barrier objective\n";
+   }
+   thx0 = theta(x0);
+   phx0 = phi(x0, mu);
+
+   lineSearchSuccess = false;
+   for(int i = 0; i < maxBacktrack; i++)
+   {
+      if (iAmRoot)
+      {
+         cout << "\n--------- alpha = " << alpha << " ---------\n";
+      }
+      // ----- A-5.2. Compute trial point: xtrial = x0 + alpha_i xhat
+      xtrial.Set(1.0, x0);
+      xtrial.Add(alpha, xhat);
+
+      // ------ A-5.3. if not in filter region go to A.5.4 otherwise go to A-5.5.
+      thxtrial = theta(xtrial);
+      phxtrial = phi(xtrial, mu);
+      filterCheck(thxtrial, phxtrial);    
+      if(!inFilterRegion)
+      {
+         if (iAmRoot)
+         {
+            cout << "not in filter region :)\n";
+         }
+         // ------ A.5.4: Check sufficient decrease
+         if(!descentDirection)
+         {
+            switchCondition = false;
+         }
+         else
+         {
+            switchCondition = (alpha * pow(abs(Dxphi0_xhat), sPhi) > delta * pow(thx0, sTheta)) ? true : false;
+         }
+         if (iAmRoot)
+         {
+            cout << "theta(x0) = "     << thx0     << ", thetaMin = "                  << thetaMin             << endl;
+            cout << "theta(xtrial) = " << thxtrial << ", (1-gTheta) *theta(x0) = "     << (1. - gTheta) * thx0 << endl;
+            cout << "phi(xtrial) = "   << phxtrial << ", phi(x0) - gPhi *theta(x0) = " << phx0 - gPhi * thx0   << endl;
+         }      
+         // Case I      
+         if(thx0 <= thetaMin && switchCondition)
+         {
+            sufficientDecrease = (phxtrial <= phx0 + eta * alpha * Dxphi0_xhat) ? true : false;
+            if(sufficientDecrease)
+            {
+               if(iAmRoot) { cout << "Line search successful: sufficient decrease in log-barrier objective.\n"; } 
+               // accept the trial step
+               lineSearchSuccess = true;
+               break;
+            }
+         }
+         else
+         {
+            if(thxtrial <= (1. - gTheta) * thx0 || phxtrial <= phx0 - gPhi * thx0)
+            {
+               if(iAmRoot) { cout << "Line search successful: infeasibility or log-barrier objective decreased.\n"; } 
+               // accept the trial step
+               lineSearchSuccess = true;
+               break;
+            }
+         }
+         // A-5.5: Initialize the second-order correction
+         if((!(thx0 < thxtrial)) && i == 0)
+         {
+            if (iAmRoot)
+            {
+               cout << "second order correction\n";
+            }
+            problem->c(xtrial, ckSoc);
+            problem->c(x0, ck0);
+            ckSoc.Add(alphaMax, ck0);
+            // A-5.6 Compute the second-order correction.
+            IPNewtonSolve(x0, l0, z0, zhatsoc, Xhatumlsoc, mu, true);
+            mhatsoc.Set(1.0, Xhatumlsoc.GetBlock(1));
+            //WARNING: not complete but currently solver isn't entering this region
+         }
+      }
+      else
+      {
+         if (iAmRoot)
+         {
+            cout << "in filter region :(\n"; 
+         }
+      }
+      // include more if needed
+      alpha *= 0.5;
+
+   } 
+}
+
+
+void ParInteriorPointSolver::projectZ(const Vector &x, Vector &z, double mu)
+{
+   double zi;
+   double mudivmml;
+   for(int i = 0; i < dimM; i++)
+   {
+      zi = z(i);
+      mudivmml = mu / (x(i + dimU) - ml(i));
+      z(i) = max(min(zi, kSig * mudivmml), mudivmml / kSig);
+   }
+}
+
+void ParInteriorPointSolver::filterCheck(double th, double ph)
+{
+   inFilterRegion = false;
+   if(th > thetaMax)
+   {
+      inFilterRegion = true;
+   }
+   else
+   {
+      for(int i = 0; i < F1.Size(); i++)
+      {
+         if(th >= F1[i] && ph >= F2[i])
+         {
+            inFilterRegion = true;
+            break;
+         }
+      }
+   }
+}
+
+double ParInteriorPointSolver::E(const BlockVector &x, const Vector &l, const Vector &zl, double mu, bool printEeval)
+{
+   double E1, E2, E3;
+   double sc, sd;
+   BlockVector gradL(block_offsetsx); gradL = 0.0; // stationarity grad L = grad f + J^T l - z
+   Vector cx(dimC); cx = 0.0;     // feasibility c = c(x)
+   Vector comp(dimM); comp = 0.0; // complementarity M Z - mu 1
+
+   DxL(x, l, zl, gradL);
+   E1 = GlobalLpNorm(infinity(), gradL.Normlinf(), MPI_COMM_WORLD); 
+
+   problem->c(x, cx);
+   E2 = GlobalLpNorm(infinity(), cx.Normlinf(), MPI_COMM_WORLD); 
+
+
+   for(int ii = 0; ii < dimM; ii++) 
+   { 
+      comp(ii) = x(dimU + ii) * zl(ii) - mu;
+   }
+   E3 = GlobalLpNorm(infinity(), comp.Normlinf(), MPI_COMM_WORLD); 
+
+   double ll1, zl1;
+
+   zl1 = GlobalLpNorm(1, zl.Norml1(), MPI_COMM_WORLD)/ double(dimCglb + dimMglb); 
+   ll1 = GlobalLpNorm(1, l.Norml1(), MPI_COMM_WORLD);
+   sc = max(sMax, zl1 / (double(dimMglb)) ) / sMax;
+   sd = max(sMax, (ll1 + zl1) / (double(dimCglb + dimMglb))) / sMax;
+   if(iAmRoot && printEeval)
+   {
+      cout << "evaluating optimality error for mu = " << mu << endl;
+      cout << "stationarity measure = "    << E1 / sd << endl;
+      cout << "feasibility measure  = "    << E2      << endl;
+      cout << "complimentarity measure = " << E3 / sc << endl;
+   }
+   return max(max(E1 / sd, E2), E3 / sc);
+}
+
+double ParInteriorPointSolver::E(const BlockVector &x, const Vector &l, const Vector &zl, bool printEeval)
+{
+  return E(x, l, zl, 0.0, printEeval);
+}
+
+double ParInteriorPointSolver::theta(const BlockVector &x)
+{
+  Vector cx(dimC); cx = 0.0;
+  problem->c(x, cx);
+  return sqrt(InnerProduct(MPI_COMM_WORLD,cx, cx));
+}
+
+// log-barrier objective
+double ParInteriorPointSolver::phi(const BlockVector &x, double mu)
+{
+   double fx = problem->CalcObjective(x); 
+   double logBarrierLoc = 0.0;
+   for(int i = 0; i < dimM; i++) 
+   { 
+     logBarrierLoc += log(x(dimU+i)-ml(i));
+   }
+   double logBarrierGlb;
+   MPI_Allreduce(&logBarrierLoc, &logBarrierGlb, 1, MPI_DOUBLE, MPI_SUM, MPI_COMM_WORLD);
+   return fx - mu * logBarrierGlb;
+}
+
+// gradient of log-barrier objective with respect to x = (u, m)
+void ParInteriorPointSolver::Dxphi(const BlockVector &x, double mu, BlockVector &y)
+{
+   problem->CalcObjectiveGrad(x, y);
+   
+   for(int i = 0; i < dimM; i++) 
+   { 
+      y(dimU + i) -= mu / (x(dimU + i));
+   } 
+}
+
+// Lagrangian function evaluation
+// L(x, l, zl) = f(x) + l^T c(x) - zl^T m
+double ParInteriorPointSolver::L(const BlockVector &x, const Vector &l, const Vector &zl)
+{
+   double fx = problem->CalcObjective(x);
+   Vector cx(dimC); problem->c(x, cx);
+   return (fx + InnerProduct(MPI_COMM_WORLD,cx, l) - InnerProduct(MPI_COMM_WORLD, x.GetBlock(1), zl));
+}
+
+void ParInteriorPointSolver::DxL(const BlockVector &x, const Vector &l, const Vector &zl, BlockVector &y)
+{
+   // evaluate the gradient of the objective with respect to the primal variables x = (u, m)
+   BlockVector gradxf(block_offsetsx); gradxf = 0.0;
+   problem->CalcObjectiveGrad(x, gradxf);
+   
+   HypreParMatrix *Jacu, *Jacm, *JacuT, *JacmT;
+   Jacu = problem->Duc(x); 
+   Jacm = problem->Dmc(x);
+   JacuT = Jacu->Transpose();
+   JacmT = Jacm->Transpose();
+   
+   JacuT->Mult(l, y.GetBlock(0));
+   JacmT->Mult(l, y.GetBlock(1));
+   
+   delete JacuT;
+   delete JacmT;
+   
+   y.Add(1.0, gradxf);
+   (y.GetBlock(1)).Add(-1.0, zl);
+}
+
+bool ParInteriorPointSolver::GetConverged() const
+{
+   return converged;
+}
+
+void ParInteriorPointSolver::SetTol(double Tol)
+{
+   OptTol = Tol;
+}
+
+void ParInteriorPointSolver::SetMaxIter(int max_it)
+{
+   max_iter = max_it;
+}
+
+void ParInteriorPointSolver::SetBarrierParameter(double mu_0)
+{
+   mu_k = mu_0;
+}
+
+void ParInteriorPointSolver::SaveLogBarrierHessianIterates(bool save)
+{
+   MFEM_ASSERT(MyRank == 0 || save == false, "currently can only save logbarrier hessian in serial codes");
+   saveLogBarrierIterates = save;
+}
+
+void ParInteriorPointSolver::SetLinearSolver(int LinSolver)
+{
+   linSolver = LinSolver;
+}
+
+void ParInteriorPointSolver::SetLinearSolveTol(double Tol)
+{
+  linSolveTol = Tol;
+}
+
+
+ParInteriorPointSolver::~ParInteriorPointSolver() 
+{
+   F1.DeleteAll();
+   F2.DeleteAll();
+   block_offsetsx.DeleteAll();
+   block_offsetsumlz.DeleteAll();
+   block_offsetsuml.DeleteAll();
+   ml.SetSize(0);
+}
@@ -0,0 +1,82 @@
+#include "mfem.hpp"
+#include "ParProblems.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+#ifndef PARIPSOLVER 
+#define PARIPSOLVER
+
+class ParInteriorPointSolver
+{
+protected:
+    ParGeneralOptProblem* problem;
+    double OptTol;
+    int  max_iter;
+    double mu_k; // \mu_k
+    Vector lk, zlk;
+
+    double sMax, kSig, tauMin, eta, thetaMin, delta, sTheta, sPhi, kMu, thetaMu;
+    double thetaMax, kSoc, gTheta, gPhi, kEps;
+	
+    // filter
+    Array<double> F1, F2;
+	
+    // quantities computed in lineSearch
+    double alpha, alphaz;
+    double thx0, thxtrial;
+    double phx0, phxtrial;
+    bool descentDirection, switchCondition, sufficientDecrease, lineSearchSuccess, inFilterRegion;
+    double Dxphi0_xhat;
+
+    int dimU, dimM, dimC;
+    int dimUglb, dimMglb, dimCglb;
+    Array<int> block_offsetsumlz, block_offsetsuml, block_offsetsx;
+    Vector ml;
+
+    Vector ckSoc;
+    HypreParMatrix * Huu, * Hum, * Hmu, * Hmm, * Wmm, *D, * Ju, * Jm, * JuT, * JmT;
+    
+    int jOpt;
+    bool converged;
+    
+    int MyRank;
+    bool iAmRoot;
+
+    bool saveLogBarrierIterates;
+
+    int linSolver;
+    double linSolveTol;
+public:
+    ParInteriorPointSolver(ParGeneralOptProblem*);
+    double MaxStepSize(Vector& , Vector& , Vector& , double);
+    double MaxStepSize(Vector& , Vector& , double);
+    void Mult(const BlockVector& , BlockVector&);
+    void Mult(const Vector&, Vector &); 
+    void FormIPNewtonMat(BlockVector& , Vector& , Vector& , BlockOperator &);
+    void IPNewtonSolve(BlockVector& , Vector& , Vector& , Vector&, BlockVector& , double, bool);
+    void lineSearch(BlockVector& , BlockVector& , double);
+    void projectZ(const Vector & , Vector &, double);
+    void filterCheck(double, double);
+    double E(const BlockVector &, const Vector &, const Vector &, double, bool);
+    double E(const BlockVector &, const Vector &, const Vector &, bool);
+    bool GetConverged() const;
+    // TO DO: include Hessian of Lagrangian
+    double theta(const BlockVector &);
+    double phi(const BlockVector &, double);
+    void Dxphi(const BlockVector &, double, BlockVector &);
+    double L(const BlockVector &, const Vector &, const Vector &);
+    void DxL(const BlockVector &, const Vector &, const Vector &, BlockVector &);
+    void SetTol(double);
+    void SetMaxIter(int);
+    void SetBarrierParameter(double);    
+    void SaveLogBarrierHessianIterates(bool);
+    void SetLinearSolver(int);
+    void SetLinearSolveTol(double);
+    virtual ~ParInteriorPointSolver();
+};
+
+#endif
@@ -0,0 +1,129 @@
+//                         Obstacle Problem
+//
+//
+// Compile with: make ParObstacleProblem
+//
+// Sample runs: mpirun -np 4 ./ParObstacleProblem
+//
+//
+// Description: This example code demonstrates the use of MFEM to solve the
+//              bound-constrained energy minimization problem
+//
+//                      minimize (||∇u||² + ||u||²) subject to u ≥ ϕ in H¹.
+
+#include "mfem.hpp"
+#include "ParProblems.hpp"
+#include "ParIPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+double dmanufacturedFun(const Vector &);
+double fRhs(const Vector &);
+
+int main(int argc, char *argv[])
+{
+  // Initialize MPI
+   Mpi::Init();
+   int num_procs = Mpi::WorldSize();
+   int myid = Mpi::WorldRank();
+   Hypre::Init();
+
+   int FEorder = 1; // order of the finite elements
+   int linSolver = 2;
+   int maxIPMiters = 30;
+   int ref_levels = 3;
+   OptionsParser args(argc, argv);
+   args.AddOption(&FEorder, "-o", "--order",\
+	 	  "Order of the finite elements.");
+   args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+        "IP-Newton linear system solution strategy.");
+   args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+	 	  "Maximum number of IPM iterations");
+   args.AddOption(&ref_levels, "-r", "--mesh_refinement", \
+		  "Mesh Refinement");
+  
+   args.Parse();
+   if(!args.Good())
+   {
+     args.PrintUsage(cout);
+     return 1;
+   }
+   else
+   {
+     if(Mpi::Root())
+     {  
+       args.PrintOptions(cout);
+     }
+   }
+
+   const char *meshFile = "../../data/inline-quad.mesh";
+   Mesh mesh(meshFile, 1, 1);
+   int dim = mesh.Dimension(); // geometric dimension of the meshed domain
+   {
+      for (int l = 0; l < ref_levels; l++)
+      {
+         mesh.UniformRefinement();
+      }
+   }
+
+   ParMesh pmesh(MPI_COMM_WORLD, mesh);
+
+   FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+   ParFiniteElementSpace   *Vh  = new ParFiniteElementSpace(&pmesh, fec);
+
+   ParObstacleProblem problem(Vh,Vh,&fRhs);
+  
+   int dimD = problem.GetDimD();
+   Vector x0(dimD); x0 = 100.0;
+   Vector xf(dimD); xf = 0.0;
+
+   ParInteriorPointSolver optimizer(&problem); 
+   optimizer.SetTol(1.e-8);
+   optimizer.SetLinearSolveTol(1.e-10);
+   optimizer.SetLinearSolver(linSolver);
+   optimizer.SetMaxIter(maxIPMiters);
+   optimizer.Mult(x0, xf);
+
+   ParGridFunction d_gf(Vh);
+
+   d_gf.SetFromTrueDofs(xf);
+
+
+   FunctionCoefficient dm_fc(dmanufacturedFun); // manufactured solution
+   ParGridFunction dm_gf(Vh);
+   dm_gf.ProjectCoefficient(dm_fc);
+
+   char vishost[] = "localhost";
+   int  visport   = 19916;
+   socketstream exact_sock(vishost, visport);
+   exact_sock.precision(8);
+   exact_sock << "parallel " << num_procs << " " << myid << "\n";
+   exact_sock << "solution\n" << pmesh << dm_gf
+              << "window_title 'Manufactured solution'" << flush;
+
+   socketstream sol_sock(vishost, visport);
+   sol_sock.precision(8);
+   sol_sock << "parallel " << num_procs << " " << myid << "\n";
+   sol_sock << "solution\n" << pmesh << d_gf 
+            << "window_title 'Numerical solution'" << flush;
+
+   delete Vh;
+   delete fec;
+   return 0;
+}
+
+
+double dmanufacturedFun(const Vector &x)
+{
+  return cos(2*M_PI*x(0)) + 0.2 - 2.0*(pow(x(0),3) - 1.5*pow(x(0),2));
+}
+
+double fRhs(const Vector &x)
+{
+  double fx = 0.;
+  fx = 0.2 - 2.0 * (pow(x(0),3)- 1.5*pow(x(0),2.) - 6 * x(0) + 3.) + (1. + pow(2.*M_PI,2))*cos(2.*M_PI*x(0));
+  return fx;
+}
@@ -0,0 +1,212 @@
+#include "mfem.hpp"
+#include "ParProblems.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+ParGeneralOptProblem::ParGeneralOptProblem(ParFiniteElementSpace * fesU_, ParFiniteElementSpace * fesM_) 
+: fesU(fesU_), fesM(fesM_) 
+{ 
+   dimU = fesU->GetTrueVSize();
+   dimM = fesM->GetTrueVSize();
+   dimC = fesM->GetTrueVSize();
+}
+
+void ParGeneralOptProblem::CalcObjectiveGrad(const BlockVector &x, BlockVector &y) const
+{
+   Duf(x, y.GetBlock(0));
+   Dmf(x, y.GetBlock(1));
+}
+
+ParGeneralOptProblem::~ParGeneralOptProblem()
+{
+   block_offsetsx.DeleteAll();
+}
+
+
+// min E(d) s.t. g(d) >= 0
+// min_(d,s) E(d) s.t. c(d,s) := g(d) - s = 0, s >= 0
+ParOptProblem::ParOptProblem(ParFiniteElementSpace * fesU_, 
+                                     ParFiniteElementSpace * fesM_)
+ : ParGeneralOptProblem(fesU_, fesM_), block_offsetsx(3)
+{
+   block_offsetsx[0] = 0;
+   block_offsetsx[1] = dimU;
+   block_offsetsx[2] = dimM;
+   block_offsetsx.PartialSum();
+   ml.SetSize(dimM); ml = 0.0;
+   Vector negIdentDiag(dimM);
+   negIdentDiag = -1.0;
+   SparseMatrix * diag = new SparseMatrix(negIdentDiag);
+   Ih = new HypreParMatrix(fesM->GetComm(), fesM->GlobalTrueVSize(),
+                                             fesM->GetTrueDofOffsets(), diag);
+   HypreStealOwnership(*Ih, *diag);
+   delete diag;
+}
+
+double ParOptProblem::CalcObjective(const BlockVector &x) const { return E(x.GetBlock(0)); }
+
+void ParOptProblem::Duf(const BlockVector &x, Vector &y) const { DdE(x.GetBlock(0), y); }
+
+void ParOptProblem::Dmf(const BlockVector &x, Vector &y) const { y = 0.0; }
+
+HypreParMatrix * ParOptProblem::Duuf(const BlockVector &x) 
+{ 
+   return DddE(x.GetBlock(0)); 
+}
+
+HypreParMatrix * ParOptProblem::Dumf(const BlockVector &x) { return nullptr; }
+
+HypreParMatrix * ParOptProblem::Dmuf(const BlockVector &x) { return nullptr; }
+
+HypreParMatrix * ParOptProblem::Dmmf(const BlockVector &x) { return nullptr; }
+
+void ParOptProblem::c(const BlockVector &x, Vector &y) const // c(u,m) = g(u) - m 
+{
+   g(x.GetBlock(0), y);
+   y.Add(-1.0, x.GetBlock(1));  
+}
+
+HypreParMatrix * ParOptProblem::Duc(const BlockVector &x) 
+{ 
+   return Ddg(x.GetBlock(0)); 
+}
+
+HypreParMatrix * ParOptProblem::Dmc(const BlockVector &x) 
+{ 
+   return Ih;
+} 
+
+ParOptProblem::~ParOptProblem() 
+{
+   delete Ih;
+}
+
+
+// Obstacle Problem, no essential boundary conditions enforced
+// Hessian of energy term is K + M (stiffness + mass)
+ParObstacleProblem::ParObstacleProblem(ParFiniteElementSpace *fesU_, 
+                                       ParFiniteElementSpace *fesM_, 
+                                       double (*fSource)(const Vector &)) : 
+                                       ParOptProblem(fesU_,fesM_), f(dimU), psi(dimU), J(nullptr)
+{
+   Kform = new ParBilinearForm(fesU);
+   Kform->AddDomainIntegrator(new MassIntegrator);
+   Kform->AddDomainIntegrator(new DiffusionIntegrator);
+   Kform->Assemble();
+   Kform->Finalize();
+   Kform->FormSystemMatrix(ess_tdof_list, K);
+
+   FunctionCoefficient fcoeff(fSource);
+   fform = new ParLinearForm(fesU);
+   fform->AddDomainIntegrator(new DomainLFIntegrator(fcoeff));
+   fform->Assemble();
+   Vector F(dimU);
+   fform->ParallelAssemble(F);
+   f.SetSize(dimU);
+   f.Set(1.0, F);
+
+   psi = 0.0;
+   
+   Vector iDiag(dimU); iDiag = 1.0;
+   SparseMatrix * Jacg = new SparseMatrix(iDiag);
+   
+   J = new HypreParMatrix(fesU->GetComm(),fesU->GlobalTrueVSize(),fesU->GetTrueDofOffsets(),Jacg);
+   HypreStealOwnership(*J, *Jacg);
+   delete Jacg;
+}
+
+// Obstacle Problem, essential boundary conditions enforced
+// Hessian of energy term is K (stiffness)
+ParObstacleProblem::ParObstacleProblem(ParFiniteElementSpace *fesU_, 
+                                       ParFiniteElementSpace *fesM_, 
+				       double (*fSource)(const Vector &),
+				       double (*obstacleSource)(const Vector &),
+				       Array<int> tdof_list, Vector &xDC) : ParOptProblem(fesU_,fesM_), f(dimU), psi(dimU), J(nullptr)
+{
+   // elastic energy functional terms	
+   ess_tdof_list = tdof_list;
+   Kform = new ParBilinearForm(fesU);
+   Kform->AddDomainIntegrator(new DiffusionIntegrator);
+   Kform->Assemble();
+   Kform->Finalize();
+   Kform->FormSystemMatrix(ess_tdof_list, K);
+
+   FunctionCoefficient fcoeff(fSource);
+   fform = new ParLinearForm(fesU);
+   fform->AddDomainIntegrator(new DomainLFIntegrator(fcoeff));
+   fform->Assemble();
+   Vector F(dimU);
+   fform->ParallelAssemble(F);
+   f.SetSize(dimU);
+   f.Set(1.0, F);
+   Kform->EliminateVDofsInRHS(ess_tdof_list, xDC, f);
+   
+   // obstacle constraints --  
+   Vector iDiag(dimU); iDiag = 1.0;
+   for(int i = 0; i < ess_tdof_list.Size(); i++)
+   {
+     iDiag(ess_tdof_list[i]) = 0.0;
+   }
+   SparseMatrix * Jacg = new SparseMatrix(iDiag);
+
+   J = new HypreParMatrix(fesU->GetComm(),fesU->GlobalTrueVSize(),fesU->GetTrueDofOffsets(),Jacg);
+   HypreStealOwnership(*J, *Jacg);
+   delete Jacg;
+
+   FunctionCoefficient psi_fc(obstacleSource);
+   ParGridFunction psi_gf(fesU);
+   psi_gf.ProjectCoefficient(psi_fc);
+   psi.Set(1.0, (*psi_gf.GetTrueDofs()));
+   for(int i = 0; i < ess_tdof_list.Size(); i++)
+   {
+     psi(ess_tdof_list[i]) -= 1.e-8;
+   }
+}
+
+
+
+double ParObstacleProblem::E(const Vector &d) const
+{
+   Vector Kd(K.Height()); Kd = 0.0;
+   MFEM_VERIFY(d.Size() == K.Width(), "ParObstacleProblem::E - Inconsistent dimensions");
+   K.Mult(d, Kd);
+   return 0.5 * InnerProduct(MPI_COMM_WORLD, d, Kd) - InnerProduct(MPI_COMM_WORLD, f, d);
+}
+
+void ParObstacleProblem::DdE(const Vector &d, Vector &gradE) const
+{
+   gradE.SetSize(K.Height());
+   MFEM_VERIFY(d.Size() == K.Width(), "ParObstacleProblem::DdE - Inconsistent dimensions");
+   K.Mult(d, gradE);
+   MFEM_VERIFY(f.Size() == K.Height(), "ParObstacleProblem::DdE - Inconsistent dimensions");
+   gradE.Add(-1.0, f);
+}
+
+HypreParMatrix * ParObstacleProblem::DddE(const Vector &d)
+{
+   return &K; 
+}
+
+// g(d) = d >= \psi
+void ParObstacleProblem::g(const Vector &d, Vector &gd) const
+{
+   MFEM_VERIFY(d.Size() == J->Width(), "ParObstacleProblem::g - Inconsistent dimensions");
+   J->Mult(d, gd);
+   MFEM_VERIFY(gd.Size() == J->Height(), "ParObstacleProblem::g - Inconsistent dimensions");
+   gd.Add(-1.0, psi);
+}
+
+HypreParMatrix * ParObstacleProblem::Ddg(const Vector &d)
+{
+   return J;
+}
+
+ParObstacleProblem::~ParObstacleProblem()
+{
+   delete Kform;
+   delete fform;
+   delete J;
+}
@@ -0,0 +1,107 @@
+#include "mfem.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+#ifndef PARPROBLEM_DEFS
+#define PARPROBLEM_DEFS
+
+// abstract ParGeneralOptProblem class
+// of the form
+// min_(u,m) f(u,m) s.t. c(u,m)=0 and m>=ml
+// the primal variable (u, m) is represented as a BlockVector
+// think about supporting general lower and upper bounds (see HiOP user manual) 
+
+class ParGeneralOptProblem
+{
+protected:
+    int dimU, dimM, dimC;
+    ParFiniteElementSpace * fesU = nullptr;
+    ParFiniteElementSpace * fesM = nullptr;
+    Array<int> block_offsetsx;
+    Vector ml;
+public:
+    ParGeneralOptProblem(ParFiniteElementSpace * fesU_, ParFiniteElementSpace * fesM_); // constructor
+    virtual double CalcObjective(const BlockVector &) const = 0;
+    virtual void Duf(const BlockVector &, Vector &) const = 0;
+    virtual void Dmf(const BlockVector &, Vector &) const = 0;
+    void CalcObjectiveGrad(const BlockVector &, BlockVector &) const;
+    virtual HypreParMatrix * Duuf(const BlockVector &) = 0;
+    virtual HypreParMatrix * Dumf(const BlockVector &) = 0;
+    virtual HypreParMatrix * Dmuf(const BlockVector &) = 0;
+    virtual HypreParMatrix * Dmmf(const BlockVector &) = 0;
+    virtual HypreParMatrix * Duc(const BlockVector &) = 0;
+    virtual HypreParMatrix * Dmc(const BlockVector &) = 0;
+    // TO DO: include Hessian terms of constraint c
+    virtual void c(const BlockVector &, Vector &) const = 0;
+    int GetDimU() const { return dimU; };
+    int GetDimM() const { return dimM; }; 
+    int GetDimC() const { return dimC; };
+    ParFiniteElementSpace * GetfesU() {return fesU;}
+    ParFiniteElementSpace * GetfesM() {return fesM;}
+    Vector Getml() const { return ml; };
+    ~ParGeneralOptProblem(); // destructor
+};
+
+
+// abstract ContactProblem class
+// of the form
+// min_d e(d) s.t. g(d) >= 0
+class ParOptProblem : public ParGeneralOptProblem
+{
+protected:
+    Array<int> block_offsetsx;
+    HypreParMatrix * Ih;
+public:
+    ParOptProblem(ParFiniteElementSpace * fesU_, ParFiniteElementSpace * fesM_);        // constructor
+    double CalcObjective(const BlockVector &) const; // objective e
+    void Duf(const BlockVector &, Vector &) const;
+    void Dmf(const BlockVector &, Vector &) const;
+
+    HypreParMatrix * Duuf(const BlockVector &);
+    HypreParMatrix * Dumf(const BlockVector &);
+    HypreParMatrix * Dmuf(const BlockVector &);
+    HypreParMatrix * Dmmf(const BlockVector &);
+    HypreParMatrix * Duc(const BlockVector &);
+    HypreParMatrix * Dmc(const BlockVector &);
+
+    void c(const BlockVector &, Vector &) const;
+    virtual double E(const Vector &) const = 0;               // objective e(d) (energy function)
+    virtual void DdE(const Vector &, Vector &) const = 0;      // gradient of objective De / Dd
+    virtual HypreParMatrix * DddE(const Vector &) = 0;
+    // Hessian of objective D^2 e / D d^2
+    virtual HypreParMatrix * Ddg(const Vector &) = 0;
+    // Jacobian of inequality constraint Dg / Dd
+    virtual void g(const Vector &, Vector &) const = 0;       // inequality constraint g(d) >= 0 (gap function)
+    int GetDimD() const { return fesU->GetTrueVSize(); };
+    int GetDimS() const { return fesM->GetTrueVSize(); };
+    virtual ~ParOptProblem();
+};
+
+class ParObstacleProblem : public ParOptProblem
+{
+protected:
+   // data to define energy objective function e(d) = 0.5 d^T K d - f^T d, g(d) = d >= \psi
+   // stiffness matrix used to define objective
+   ParBilinearForm *Kform;
+   ParLinearForm   *fform;
+   Array<int> ess_tdof_list; // needed for calls to FormSystemMatrix
+   HypreParMatrix  K;
+   HypreParMatrix  *J;
+   ParFiniteElementSpace *Vh;
+   Vector f;
+   Vector psi;
+public :
+   ParObstacleProblem(ParFiniteElementSpace*, ParFiniteElementSpace*, double (*fSource)(const Vector &));
+   ParObstacleProblem(ParFiniteElementSpace*, ParFiniteElementSpace*, double (*fSource)(const Vector &), double (*obstacleSource)(const Vector &), Array<int> tdof_list, Vector &);
+   double E(const Vector &) const;
+   void DdE(const Vector &, Vector &) const;
+   HypreParMatrix* DddE(const Vector &);
+   void g(const Vector &, Vector &) const;
+   HypreParMatrix* Ddg(const Vector &);
+   virtual ~ParObstacleProblem();
+};
+
+#endif
@@ -0,0 +1,173 @@
+//                       Spherical Obstacle Problem
+//
+//
+// Compile with: make ParSphericalObstacleProblem
+//
+// Sample runs: mpirun -np 4 ./ParSphericalObstacleProblem -linSolver 0
+//              mpirun -np 4 ./ParSphericalObstacleProblem -linSolver 1
+//              mpirun -np 4 ./ParSphericalObstacleProblem -linSolver 2
+//
+//
+// Description: This example code demonstrates the use of MFEM to solve the
+//              bound-constrained energy minimization problem
+//
+//                      minimize ||∇u||² subject to u ≥ ϕ in H¹₀.
+
+#include "mfem.hpp"
+#include "ParProblems.hpp"
+#include "ParIPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+double fRhs(const Vector &);
+double spherical_obstacle(const Vector &);
+double exact_solution_obstacle(const Vector &);
+
+
+int main(int argc, char *argv[])
+{
+  // Initialize MPI
+   Mpi::Init();
+   int num_procs = Mpi::WorldSize();
+   int myid = Mpi::WorldRank();
+   Hypre::Init();
+
+   int FEorder = 1; // order of the finite elements
+   int linSolver = 2;
+   int maxIPMiters = 30;
+   int ref_levels = 3;
+   OptionsParser args(argc, argv);
+   args.AddOption(&FEorder, "-o", "--order",\
+	 	  "Order of the finite elements.");
+   args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+        "IP-Newton linear system solution strategy.");
+   args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+	 	  "Maximum number of IPM iterations");
+   args.AddOption(&ref_levels, "-r", "--mesh_refinement", \
+		  "Mesh Refinement");
+  
+   args.Parse();
+   if(!args.Good())
+   {
+     args.PrintUsage(cout);
+     return 1;
+   }
+   else
+   {
+     if(myid == 0)
+     {  
+       args.PrintOptions(cout);
+     }
+   }
+
+   const char *meshFile = "../../data/disk.mesh";
+   Mesh mesh(meshFile, 1, 1);
+   int dim = mesh.Dimension(); // geometric dimension of the meshed domain
+   {
+      for (int l = 0; l < ref_levels; l++)
+      {
+         mesh.UniformRefinement();
+      }
+   }
+
+   ParMesh pmesh(MPI_COMM_WORLD, mesh);
+   
+   FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+   ParFiniteElementSpace   *Vh  = new ParFiniteElementSpace(&pmesh, fec);
+   Array<int> boundary_dofs;
+   Vh->GetBoundaryTrueDofs(boundary_dofs);
+   int dimD = Vh->GetTrueVSize();
+   Vector xDC(dimD); xDC = 0.0;
+
+   ParObstacleProblem problem(Vh, Vh, &fRhs, &spherical_obstacle, boundary_dofs, xDC);
+   Vector x0(dimD); x0.Set(1.0, xDC);
+   Vector xf(dimD); xf = 0.0;
+
+   ParInteriorPointSolver optimizer(&problem); 
+   optimizer.SetTol(1.e-7);
+   optimizer.SetLinearSolveTol(1.e-10);
+   optimizer.SetLinearSolver(linSolver);
+   optimizer.SetMaxIter(maxIPMiters);
+   optimizer.Mult(x0, xf);
+
+   ParGridFunction d_gf(Vh);
+
+   d_gf.SetFromTrueDofs(xf);
+
+
+   FunctionCoefficient dtrue_fc(exact_solution_obstacle); // analytic solution
+   ParGridFunction dtrue_gf(Vh);
+   dtrue_gf.ProjectCoefficient(dtrue_fc);
+
+   double L2error = d_gf.ComputeL2Error(dtrue_fc);
+   if (myid == 0)
+   {
+     cout << "\n|| u_h - u ||_{L^2} = " << L2error << '\n' << endl;
+   } 
+   
+   ParaViewDataCollection paraview_dc("SphericalObstacleProblem", &pmesh);
+   paraview_dc.SetPrefixPath("ParaView");
+   paraview_dc.SetLevelsOfDetail(FEorder);
+   paraview_dc.SetDataFormat(VTKFormat::BINARY);
+   paraview_dc.SetHighOrderOutput(true);
+   paraview_dc.SetCycle(0);
+   paraview_dc.SetTime(0.0);
+   paraview_dc.RegisterField("u(x,y) (analytic)", &dtrue_gf);
+   paraview_dc.RegisterField("u(x,y) (numerical)", &d_gf);
+   paraview_dc.Save();
+
+   delete Vh;
+   delete fec;
+   return 0;
+}
+
+
+double fRhs(const Vector &x)
+{
+  return 0.;
+}
+
+
+double spherical_obstacle(const Vector &pt)
+{
+   double x = pt(0), y = pt(1);
+   double r = sqrt(x*x + y*y);
+   double r0 = 0.5;
+   double beta = 0.9;
+
+   double b = r0*beta;
+   double tmp = sqrt(r0*r0 - b*b);
+   double B = tmp + b*b/tmp;
+   double C = -b/tmp;
+
+   if (r > b)
+   {
+      return B + r * C;
+   }
+   else
+   {
+      return sqrt(r0*r0 - r*r);
+   }
+}
+
+double exact_solution_obstacle(const Vector &pt)
+{
+   double x = pt(0), y = pt(1);
+   double r = sqrt(x*x + y*y);
+   double r0 = 0.5;
+   double a =  0.348982574111686;
+   double A = -0.340129705945858;
+
+   if (r > a)
+   {
+      return A * log(r);
+   }
+   else
+   {
+      return sqrt(r0*r0-r*r);
+   }
+} 
+
@@ -0,0 +1,300 @@
+#include "mfem.hpp"
+#include <fstream>
+#include <iostream>
+#include <set>
+
+using namespace std;
+using namespace mfem;
+
+#ifndef PROBLEM_DEFS
+#define PROBLEM_DEFS
+
+
+
+// abstract GeneralOptProblem class
+// for the problem
+// min_(u,m) f(u,m) 
+// such that c(u,m)=0 and m >= ml
+class GeneralOptProblem
+{
+protected:
+    int dimU, dimM, dimC;
+    Array<int> block_offsetsx;
+    Vector ml;
+public:
+    GeneralOptProblem();
+    virtual double CalcObjective(const BlockVector &) const = 0;
+    virtual void Duf(const BlockVector &, Vector &) const = 0;
+    virtual void Dmf(const BlockVector &, Vector &) const = 0;
+    void CalcObjectiveGrad(const BlockVector &, BlockVector &) const;
+    virtual SparseMatrix* Duuf(const BlockVector &) = 0;
+    virtual SparseMatrix* Dumf(const BlockVector &) = 0;
+    virtual SparseMatrix* Dmuf(const BlockVector &) = 0;
+    virtual SparseMatrix* Dmmf(const BlockVector &) = 0;
+    virtual void c(const BlockVector &, Vector &) const = 0;
+    virtual SparseMatrix* Duc(const BlockVector &) = 0;
+    virtual SparseMatrix* Dmc(const BlockVector &) = 0;
+    virtual SparseMatrix* lDuuc(const BlockVector &, const Vector &) = 0;
+    virtual SparseMatrix* lDumc(const BlockVector &, const Vector &) = 0;
+    virtual SparseMatrix* lDmuc(const BlockVector &, const Vector &) = 0;
+    virtual SparseMatrix* lDmmc(const BlockVector &, const Vector &) = 0;
+    // TO DO: include log-barrier lumped-mass and pass that
+    // to the optimizer
+    //virtual SparseMatrix* GetLogBarrierLumpedMass() = 0;
+    int GetDimU() const { return dimU; };
+    int GetDimM() const { return dimM; }; 
+    int GetDimC() const { return dimC; };
+    Vector Getml() const { return ml; };
+    ~GeneralOptProblem();
+};
+
+
+// abstract OptProblem class
+// of the form
+// min_d e(d) s.t. g(d) >= 0
+class OptProblem : public GeneralOptProblem
+{
+protected:
+    int dimD;
+    int dimS;
+    Array<int> block_offsetsx;
+    SparseMatrix * negIdentity;
+    SparseMatrix * zeroMatum;
+    SparseMatrix * zeroMatmu;
+    SparseMatrix * zeroMatmm;
+public:
+    //OptProblem(int, int);        // constructor
+    OptProblem();
+    void InitializeParentData(int, int);
+    double CalcObjective(const BlockVector &) const; // objective e
+    void Duf(const BlockVector &, Vector &) const;
+    void Dmf(const BlockVector &, Vector &) const;
+    SparseMatrix* Duuf(const BlockVector &);
+    SparseMatrix* Dumf(const BlockVector &);
+    SparseMatrix* Dmuf(const BlockVector &);
+    SparseMatrix* Dmmf(const BlockVector &);
+    void c(const BlockVector &, Vector &) const;
+    SparseMatrix* Duc(const BlockVector &);
+    SparseMatrix* Dmc(const BlockVector &);
+    SparseMatrix* lDuuc(const BlockVector &, const Vector &);
+    SparseMatrix* lDumc(const BlockVector &, const Vector &);
+    SparseMatrix* lDmuc(const BlockVector &, const Vector &);
+    SparseMatrix* lDmmc(const BlockVector &, const Vector &);
+    virtual double E(const Vector &) const = 0;           // objective e(d) (energy function)
+    virtual void DdE(const Vector &, Vector &) const = 0; // gradient of objective De / Dd
+    virtual SparseMatrix* DddE(const Vector &) = 0;       // Hessian of objective D^2 e / D d^2
+    virtual void g(const Vector &, Vector &) const = 0;   // inequality constraint g(d) >= 0 (gap function)
+    virtual SparseMatrix* Ddg(const Vector &) = 0;        // Jacobian of inequality constraint Dg / Dd
+    virtual SparseMatrix* lDddg(const Vector &, const Vector &) = 0;
+    int GetDimD() const { return dimD; };
+    int GetDimS() const { return dimS; };
+    virtual ~OptProblem();
+};
+
+
+class ObstacleProblem : public OptProblem
+{
+protected:
+   // data to define energy objective function e(d) = 0.5 d^T K d - f^T d, g(d) = d + \psi >= 0
+   // stiffness matrix used to define objective
+   BilinearForm *Kform;
+   LinearForm   *fform;
+   Array<int> ess_tdof_list;
+   SparseMatrix  *K;
+   SparseMatrix  *J;
+   SparseMatrix  *Hcl;
+   FiniteElementSpace *Vh;
+   Vector f;
+   Vector psil;
+   Vector psiu;
+   bool twoBounds;
+   Vector xDC;
+   double Ce;
+public : 
+   ObstacleProblem(FiniteElementSpace*, double (*fSource)(const Vector &), double (*obstacleSource)(const Vector &));
+   ObstacleProblem(FiniteElementSpace*, Vector&,  double (*fSource)(const Vector &), double (*obstacleSource)(const Vector &), Array<int> tdof_list);
+   ObstacleProblem(FiniteElementSpace*, Vector &, double (*fSource)(const Vector &), double (*obstacleSourcel)(const Vector &), double (*obstacleSourceu)(const Vector &), Array<int> tdof_list);
+   double E(const Vector &) const;
+   void DdE(const Vector &, Vector &) const;
+   SparseMatrix* DddE(const Vector &);
+   void g(const Vector &, Vector &) const;
+   SparseMatrix* Ddg(const Vector &);
+   SparseMatrix * lDddg(const Vector &, const Vector &);
+   virtual ~ObstacleProblem();
+};
+
+
+SparseMatrix * GenerateProjector(int n, Array<int> ess_tdof_list);
+
+
+class ObstacleProblemVariant : public OptProblem
+{
+protected:
+   // data to define energy objective function e(d) = 0.5 d^T K d - f^T d, g(d) = d + \psi >= 0
+   // stiffness matrix used to define objective
+   BilinearForm *Kform;
+   LinearForm   *fform;
+   Array<int> ess_tdof_list;
+   Array<int> noness_tdof_list;
+   SparseMatrix  *K;
+   SparseMatrix  *RKP; // R K P = R K R^T
+   SparseMatrix  *J;
+   SparseMatrix  *Hcl;
+   SparseMatrix *R;
+   FiniteElementSpace *Vh;
+   Vector f;
+   Vector psil;
+   Vector xDC;
+   double Ce;
+public : 
+   ObstacleProblemVariant(FiniteElementSpace*, Vector&,  double (*fSource)(const Vector &), double (*obstacleSource)(const Vector &), Array<int> tdof_list);
+   double E(const Vector &) const;
+   void DdE(const Vector &, Vector &) const;
+   SparseMatrix* DddE(const Vector &);
+   void g(const Vector &, Vector &) const;
+   SparseMatrix* Ddg(const Vector &);
+   SparseMatrix * lDddg(const Vector &, const Vector &);
+   virtual ~ObstacleProblemVariant();
+};
+
+
+
+
+class QPOptProblem : public OptProblem
+{
+protected:
+  SparseMatrix *K;
+  SparseMatrix *J;
+  SparseMatrix *zeroMatdd;
+  Vector f;
+  Vector g0;
+public:
+  QPOptProblem(const SparseMatrix, const SparseMatrix, const Vector, const Vector);
+  double E(const Vector &) const;
+  void DdE(const Vector &, Vector &) const;
+  SparseMatrix* DddE(const Vector &);
+  void g(const Vector &, Vector &) const;
+  SparseMatrix* Ddg(const Vector &);
+  SparseMatrix * lDddg(const Vector &, const Vector &);
+  virtual ~QPOptProblem();
+};
+
+
+class ExContactBlockTL : public OptProblem
+{
+public:
+   double E(const Vector &) const;
+   void DdE(const Vector &, Vector &) const;
+   SparseMatrix* DddE(const Vector &);
+   void g(const Vector &, Vector &) const;
+   SparseMatrix* Ddg(const Vector &);
+   SparseMatrix * lDddg(const Vector &, const Vector &);
+   FiniteElementSpace GetVh1();
+   FiniteElementSpace GetVh2();
+   SparseMatrix *zeroMatdd;
+public:
+   /** default constructor */
+   ExContactBlockTL(Mesh *, Mesh *, int);
+   
+
+   /** default destructor */
+   virtual ~ExContactBlockTL();
+
+private:
+   void update_g() const;
+
+private:
+   /**@name Methods to block default compiler methods.
+    *
+    * The compiler automatically generates the following three methods.
+    *  Since the default compiler implementation is generally not what
+    *  you want (for all but the most simple classes), we usually
+    *  put the declarations of these methods in the private section
+    *  and never implement them. This prevents the compiler from
+    *  implementing an incorrect "default" behavior without us
+    *  knowing. (See Scott Meyers book, "Effective C++")
+    */
+   ExContactBlockTL(
+      const ExContactBlockTL&
+   );
+
+   ExContactBlockTL& operator=(
+      const ExContactBlockTL&
+   );
+
+   Array<int> attr;
+   Array<int> m_attr;
+   Array<int> s_conn; // connectivity of the second/slave mesh
+   std::string mesh_file1;
+   std::string mesh_file2;
+   Mesh* mesh1;
+   Mesh* mesh2;
+   FiniteElementCollection* fec1;
+   FiniteElementCollection* fec2;
+   FiniteElementSpace* fespace1;
+   FiniteElementSpace* fespace2;
+   Array<int> ess_tdof_list1;
+   Array<int> ess_tdof_list2;
+   GridFunction nodes0;
+   GridFunction* nodes1;
+   GridFunction* nodes2;
+   mutable GridFunction* x1;
+   mutable GridFunction* x2;
+   PWConstCoefficient* lambda1_func;
+   PWConstCoefficient* lambda2_func;
+   PWConstCoefficient* mu1_func;
+   PWConstCoefficient* mu2_func;
+   BilinearForm* a1;
+   BilinearForm* a2;
+
+   mfem::Vector lambda1;
+   mfem::Vector lambda2;
+   mfem::Vector mu1;
+   mfem::Vector mu2;
+   mutable mfem::Vector xyz;
+
+   std::set<int> bdryVerts2;
+
+   int dim;
+   // degrees of freedom of both meshes
+   int ndof_1;
+   int ndof_2;
+   int ndofs;
+   // number of nodes for each mesh
+   int nnd_1;
+   int nnd_2;
+   int nnd;
+
+   int npoints;
+
+   SparseMatrix A1;
+   mfem::Vector B1, X1;
+   SparseMatrix A2;
+   mfem::Vector B2, X2;
+   BlockVector *B;
+   SparseMatrix* K;
+   mutable mfem::Vector gapv;
+   mutable mfem::Vector m_xi;
+   mutable mfem::Vector xs;
+
+   mutable Array<int> m_conn; // only works for linear elements that have 4 vertices!
+   mutable DenseMatrix* coordsm;
+   mutable SparseMatrix* M;
+
+   mutable std::vector<SparseMatrix>* dM;
+
+   Array<int> Dirichlet_dof;
+   Array<double> Dirichlet_val;
+   Array<int> block_offsets;
+public: 
+   Mesh * GetMesh1() {return mesh1;}
+   Mesh * GetMesh2() {return mesh2;}
+   GridFunction & GetMesh1GridFunction() {return *x1;}
+   GridFunction & GetMesh2GridFunction() {return *x2;}
+   Array<int> & GetMesh1DirichletDofs() {return ess_tdof_list1;}
+   Array<int> & GetMesh2DirichletDofs() {return ess_tdof_list2;}
+};
+
+
+
+#endif
@@ -0,0 +1,175 @@
+//                          Spherical Obstacle Problem
+//
+//
+// Compile with: make SphericalobstacleProblem
+//
+// Sample runs: ./SphericalobstacleProblem
+//
+//
+// Description: This example code demonstrates the use of MFEM to solve the
+//              bound-constrained energy minimization problem
+//
+//                      minimize ||∇u||² subject to u ≥ ϕ in H¹₀.
+
+
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include "IPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+double fRhs(const Vector &);
+double spherical_obstacle(const Vector &);
+double exact_solution_obstacle(const Vector &);
+
+int main(int argc, char *argv[])
+{
+  int FEorder = 1;     // finite element order
+  int linSolver = 0;   // linear solver 0 (direct), 1 (iterative) or 2 (iterative)
+  int maxIPMiters = 30;
+  bool iAmRoot = true;
+  int ref_levels = 3;
+  
+  OptionsParser args(argc, argv);
+  args.AddOption(&FEorder, "-o", "--order",\
+		  "Order of the finite elements.");
+  args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+       "IP-Newton linear system solution strategy.");
+  args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+		  "Maximum number of IPM iterations");
+  args.AddOption(&ref_levels, "-r", "--mesh_refinement", \
+		  "Mesh Refinement");
+  
+  args.Parse();
+  if(!args.Good())
+  {
+    args.PrintUsage(cout);
+    return 1;
+  }
+  else
+  {
+    if( iAmRoot )
+    {
+      args.PrintOptions(cout);
+    }
+  }
+
+  const char *meshFile = "../../data/disk.mesh";
+  Mesh *mesh = new Mesh(meshFile, 1, 1);
+  int dim = mesh->Dimension(); // geometric dimension of the domain
+  {
+     for (int l = 0; l < ref_levels; l++)
+     {
+        mesh->UniformRefinement();
+     }
+  }
+  double h_min, h_max, kappa_min, kappa_max;
+  mesh->GetCharacteristics(h_min, h_max, kappa_min, kappa_max);
+
+  FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+  FiniteElementSpace      *Vh  = new FiniteElementSpace(mesh, fec);
+  Array<int> ess_tdof_list;
+  if (mesh->bdr_attributes.Size())
+  {
+     Array<int> ess_bdr(mesh->bdr_attributes.Max());
+     ess_bdr = 1;
+     Vh->GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
+  }
+  
+  int dimD = Vh->GetTrueVSize();
+  Vector x0(dimD); x0 = 0.0;
+  Vector xf(dimD); xf = 0.0;
+  
+  ObstacleProblem problem(Vh, x0, &fRhs, &spherical_obstacle, ess_tdof_list);
+  InteriorPointSolver optimizer(&problem); 
+  
+  optimizer.SetTol(1.e-7);
+  optimizer.SetLinearSolver(linSolver);
+  optimizer.SetMaxIter(maxIPMiters);
+  optimizer.Mult(x0, xf);
+  
+  
+  double Einitial = problem.E(x0);
+  double Efinal = problem.E(xf);
+  cout << "Energy objective at initial point = " << Einitial << endl;
+  cout << "Energy objective at optimizer = " << Efinal << endl;
+
+  GridFunction d_gf(Vh);
+  d_gf = xf;
+
+  FunctionCoefficient dtrue_fc(exact_solution_obstacle); // exact solution
+  GridFunction dtrue_gf(Vh);
+  dtrue_gf.ProjectCoefficient(dtrue_fc);
+  
+  ParaViewDataCollection paraview_dc("BarrierProblemSolution", mesh);
+  paraview_dc.SetPrefixPath("ParaView");
+  paraview_dc.SetLevelsOfDetail(FEorder);
+  paraview_dc.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc.SetHighOrderOutput(true);
+  paraview_dc.SetCycle(0);
+  paraview_dc.SetTime(0.0);
+  paraview_dc.RegisterField("d(x) (numerical)", &d_gf);
+  paraview_dc.RegisterField("d(x) (true)", &dtrue_gf);
+  paraview_dc.Save();
+  
+  FunctionCoefficient exact_coef(exact_solution_obstacle);
+  double L2_error = d_gf.ComputeL2Error(exact_coef);
+  cout << "||u - u_true||_L^2(Omega) = " << L2_error << ", hmax = " << h_max << ", hmin = " << h_min << endl;
+  
+  delete Vh;
+  delete fec;
+  delete mesh;
+  return 0;
+}
+
+
+double fRhs(const Vector &x)
+{
+  return 0.;
+}
+
+
+double spherical_obstacle(const Vector &pt)
+{
+   double x = pt(0), y = pt(1);
+   double r = sqrt(x*x + y*y);
+   double r0 = 0.5;
+   double beta = 0.9;
+
+   double b = r0*beta;
+   double tmp = sqrt(r0*r0 - b*b);
+   double B = tmp + b*b/tmp;
+   double C = -b/tmp;
+
+   if (r > b)
+   {
+      return B + r * C;
+   }
+   else
+   {
+      return sqrt(r0*r0 - r*r);
+   }
+}
+
+double exact_solution_obstacle(const Vector &pt)
+{
+   double x = pt(0), y = pt(1);
+   double r = sqrt(x*x + y*y);
+   double r0 = 0.5;
+   double a =  0.348982574111686;
+   double A = -0.340129705945858;
+
+   if (r > a)
+   {
+      return A * log(r);
+   }
+   else
+   {
+      return sqrt(r0*r0-r*r);
+   }
+} 
+
@@ -0,0 +1,143 @@
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include "IPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+
+double fRhs(const Vector &pt);
+double obstaclel(const Vector &pt);
+double obstacleu(const Vector &pt);
+double dmanufacturedFun(const Vector &pt);
+
+int main(int argc, char *argv[])
+{
+  int FEorder = 1; // order of the finite elements
+  int linSolver = 0;
+  int maxIPMiters = 30;
+  bool iAmRoot = true;
+  int ref_levels = 1; 
+  OptionsParser args(argc, argv);
+  args.AddOption(&FEorder, "-o", "--order",\
+		  "Order of the finite elements.");
+  args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+       "IP-Newton linear system solution strategy.");
+  args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+		  "Maximum number of IPM iterations");
+  args.AddOption(&ref_levels, "-r", "--mesh_refinement", \
+		  "Mesh Refinement");
+  
+  args.Parse();
+  if(!args.Good())
+  {
+    args.PrintUsage(cout);
+    return 1;
+  }
+  else
+  {
+    if( iAmRoot )
+    {
+      args.PrintOptions(cout);
+    }
+  }
+
+  const char *meshFile = "../../data/inline-quad.mesh";
+  Mesh *mesh = new Mesh(meshFile, 1, 1);
+  int dim = mesh->Dimension(); // geometric dimension of the domain
+  for (int l = 0; l < ref_levels; l++)
+  {
+      mesh->UniformRefinement();
+  }
+
+  FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+  FiniteElementSpace      *Vh  = new FiniteElementSpace(mesh, fec);
+  Array<int> ess_tdof_list;
+  if (mesh->bdr_attributes.Size())
+  {
+     Array<int> ess_bdr(mesh->bdr_attributes.Max());
+     ess_bdr = 1;
+     Vh->GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
+  }
+  
+  double DC_val = 0.0;
+  int dimD = Vh->GetTrueVSize();
+
+  Vector x0(dimD); x0 = DC_val;
+  Vector xf(dimD); xf = 0.0;
+
+  ObstacleProblem problem(Vh, x0, &fRhs, &obstaclel, &obstacleu, ess_tdof_list);
+
+  InteriorPointSolver optimizer(&problem); 
+  optimizer.SetTol(1.e-7);
+  optimizer.SetLinearSolver(linSolver);
+  optimizer.SetMaxIter(maxIPMiters);
+  optimizer.Mult(x0, xf);
+  
+
+  GridFunction d_gf(Vh);
+
+  d_gf = xf;
+  
+  FunctionCoefficient dm_fc(dmanufacturedFun); // pseudo-manufactured solution
+  GridFunction dm_gf(Vh);
+  dm_gf.ProjectCoefficient(dm_fc);
+
+  
+  ParaViewDataCollection paraview_dc("BarrierProblemSolution", mesh);
+  paraview_dc.SetPrefixPath("ParaView");
+  paraview_dc.SetLevelsOfDetail(FEorder);
+  paraview_dc.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc.SetHighOrderOutput(true);
+  paraview_dc.SetCycle(0);
+  paraview_dc.SetTime(0.0);
+  paraview_dc.RegisterField("d(x) (numerical)", &d_gf);
+  paraview_dc.RegisterField("d(x) (pseudo-manufactured)", &dm_gf);
+  paraview_dc.Save();
+  
+
+  delete Vh;
+  delete fec;
+  delete mesh;
+  return 0;
+}
+
+
+
+double dmanufacturedFun(const Vector &pt)
+{
+  double alpha = 16.5;
+  return sin(M_PI * pt(1)) * (sin(M_PI * pt(0)) - alpha * pow(pt(0) * (1. - pt(0)), 2));
+}
+
+
+// f(x) forcing term... which enters the objective energy functional
+// E(d) = 0.5 d^T K d - f^T d, where f is a discrete vector representation
+// of f(x). f(x) is such that in the absence of bound-constraints then
+// the solution of the optimization problem satisfies the PDE
+// -div(grad(d)) + d = f + homogeneous Neumann conditions on the unit interval,
+// for d(x) = cos(2 \pi x) + a0 + a3 (x^3 - 1.5 x^2), a2 = 0.2, a3 = -2
+
+double fRhs(const Vector &pt)
+{
+  double alpha = 16.5;
+  double fx;
+  fx = pow(M_PI, 2) * sin(M_PI * pt(0));
+  fx += alpha * (2. * pow(pt(0), 2) + 2. * pow(1.-pt(0), 2) - 8. * pt(0) * (1.-pt(0)));
+  fx += pow(M_PI, 2) * sin(M_PI * pt(0)) * dmanufacturedFun(pt);
+  fx *= sin(M_PI * pt(1));
+  return fx;
+}
+
+double obstaclel(const Vector &pt)
+{
+  return 0.0;
+}
+
+double obstacleu(const Vector &pt)
+{
+  return 0.08;
+}
@@ -0,0 +1,103 @@
+MFEM mesh v1.0
+
+#
+# MFEM Geometry Types (see mesh/geom.hpp):
+#
+# POINT       = 0
+# SEGMENT     = 1
+# TRIANGLE    = 2
+# SQUARE      = 3
+# TETRAHEDRON = 4
+# CUBE        = 5
+# PRISM       = 6
+#
+
+dimension
+3
+
+elements
+9
+1   5     0      1     3     2     8    9     11    10
+1   5     2      3     5     4    10    11    13    12
+1   5     4      5     7     6    12    13    15    14
+1   5     8      9    11    10    16    17    19    18
+1   5     10    11    13    12    18    19    21    20
+1   5     12    13    15    14    20    21    23    22
+1   5     16    17    19    18    24    25    27    26
+1   5     18    19    21    20    26    27    29    28
+1   5     20    21    23    22    28    29    31    30
+
+
+
+# 0 nothing
+# 1 dirichlet bc
+# 2 contact
+boundary
+30
+1   3    1     0     2     3
+1   3    3     2     4     5
+1   3    5     4     6     7
+1   3    24    25    27    26
+1   3    26    27    29    28
+1   3    28    29    31    30
+2   3    2     0     8    10
+2   3    4     2    10    12
+2   3    6     4    12    14
+2  3    10    8    16    18
+2  3    12    10    18    20
+2  3    14    12    20    22
+2  3    18    16    24    26
+2  3    20    18    26    28
+2  3    22    20    28    30
+3  3    1     3    11    9
+3  3     3     5    13    11
+3  3     5     7    15    13
+3  3    9    11    19    17
+3  3    11    13    21    19
+3  3    13    15    23    21
+3  3    17    19    27    25
+3  3    19    21    29    27
+3  3    21    23    31    29
+1  3     8     0     1    9
+1  3    16     8    9    17
+1  3    24    16    17    25
+1  3     6    14    15     7
+1  3    14    22    23    15
+1  3    22    30    31    23
+
+
+vertices
+32
+3
+   -1.0000         0         0
+         0         0         0
+   -1.0000    0.3000         0
+         0    0.3000         0
+   -1.0000    0.6500         0
+         0    0.6500         0
+   -1.0000    1.0000         0
+         0    1.0000         0
+   -1.0000         0    0.3000
+         0         0    0.3000
+   -1.0000    0.3000    0.3500
+         0    0.3000    0.3500
+   -1.0000    0.6500    0.3000
+         0    0.6500    0.3000
+   -1.0000    1.0000    0.3000
+         0    1.0000    0.3000
+   -1.0000         0    0.6500
+         0         0    0.6500
+   -1.0000    0.3000    0.6500
+         0    0.3000    0.6500
+   -1.0000    0.6500    0.6500
+         0    0.6500    0.6500
+   -1.0000    1.0000    0.6500
+         0    1.0000    0.6500
+   -1.0000         0    1.0000
+         0         0    1.0000
+   -1.0000    0.3000    1.0000
+         0    0.3000    1.0000
+   -1.0000    0.6500    1.0000
+         0    0.6500    1.0000
+   -1.0000    1.0000    1.0000
+         0    1.0000    1.0000
@@ -0,0 +1,246 @@
+//                                Quadratic-Programming (QP) Contact example
+//
+// Compile with: make exQPContactBlockTL
+//
+// Sample runs:  ./exQPContactBlockTL
+
+
+#include <fstream>
+#include <iostream>
+#include <array>
+
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include "IPsolver.hpp"
+
+using namespace std;
+using namespace mfem;
+
+
+int main(int argc, char *argv[])
+{
+  int linSolver = 0;
+  int maxIPMiters = 30;
+  bool iAmRoot = true;
+  int ref_levels = 0;
+
+  OptionsParser args(argc, argv);
+  args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+       "IP-Newton linear system solution strategy.");
+  args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+		  "Maximum number of IPM iterations");
+  args.AddOption(&ref_levels, "-r", "--mesh_refinement", \
+		  "Mesh Refinement");
+
+   
+  args.Parse();
+  if(!args.Good())
+  {
+    args.PrintUsage(cout);
+    return 1;
+  }
+  else
+  {
+    if( iAmRoot )
+    {
+      args.PrintOptions(cout);
+    }
+  }
+	
+  Mesh * mesh1 = new Mesh("block1.mesh", 1, 1);
+  Mesh * mesh2 = new Mesh("rotatedblock2.mesh", 1, 1);
+  for(int i = 0; i < ref_levels; i++)
+  {
+     mesh1->UniformRefinement();
+     mesh2->UniformRefinement(); 
+  }
+
+  // Create an instance of the nlp
+  ExContactBlockTL * contact = new ExContactBlockTL(mesh1, mesh2, 1);
+  int ndofs = contact->GetDimD();
+  int nconstraints = contact->GetDimS();
+  
+  // set up a QP-problem
+  // E(d) = 1 / 2 d^T K d + f^T d
+  // g(d) = J d + g0
+  // where K, J, f and g0 are evaluated at d0 (a valid configuration)
+  
+  // to do: seems more appropriate to evaluate at a valid configuration...
+  //        that is one where the Dirichlet conditions hold... need to pull
+  //        this data from contactBlockTL...
+  Vector d0(ndofs); d0 = 0.0;
+  Array<int> ess_tdofs1 = contact->GetMesh1DirichletDofs();
+  Array<int> ess_tdofs2 = contact->GetMesh2DirichletDofs();
+  int sz1 = ess_tdofs1.Size();
+  int sz2 = ess_tdofs2.Size();
+  Array<int> DirichletDofs(sz1+sz2);
+  for (int i = 0; i<sz1; i++)
+  {
+    DirichletDofs[i] = ess_tdofs1[i];
+  }
+  for (int i = 0; i<sz2; i++)
+  {
+    DirichletDofs[i+sz1] = ess_tdofs2[i]+contact->GetVh1().GetTrueVSize();
+  }
+  GridFunction x1 = contact->GetMesh1GridFunction();
+  GridFunction x2 = contact->GetMesh2GridFunction();
+
+  SparseMatrix *K;
+  Vector f(ndofs); f = 0.0;
+  contact->DdE(d0, f); K = contact->DddE(d0);
+  d0.SetVector(x1,0);
+  d0.SetVector(x2,x1.Size());
+  SparseMatrix *J;
+  Vector g0(nconstraints); g0 = 0.0;
+  contact->g(d0, g0); J = contact->Ddg(d0);
+  Vector temp(nconstraints);
+  J->Mult(d0, temp);
+  g0.Add(-1.0, temp);
+
+  // check which rows of the Jacobian are zero!
+  Vector ei(nconstraints); ei = 0.0;
+  Vector JTei(ndofs); JTei = 0.0;
+
+  double normJTei;
+
+  Array<int> nonZeroRows;
+  for(int i = 0; i < nconstraints; i++)
+  {
+    Array<int> col_tmp;
+    Vector v_tmp; v_tmp = 0.0;
+    J->GetRow(i, col_tmp, v_tmp);
+    normJTei = v_tmp.Norml2();
+    if (normJTei > 1.e-12)
+    {
+      nonZeroRows.Append(i);
+    }
+  }
+  mfem::out << J->Height() << " linearized constraints\n";
+  mfem::out << nonZeroRows.Size() << " (reduced) linearized constraints\n"; 
+  
+  // remove zero rows of the gap function Jacobian and corresponding gap function entries
+  SparseMatrix * Jreduced = new SparseMatrix(nonZeroRows.Size(), ndofs);
+  Vector g0reduced(nonZeroRows.Size()); g0reduced = 0.0; 
+  
+  
+  for(int i = 0; i < nonZeroRows.Size(); i++)
+  {
+    Array<int> col_tmp;
+    Vector v_tmp; v_tmp = 0.0;
+    J->GetRow(nonZeroRows[i], col_tmp, v_tmp);
+    
+    /* obtain subset of columns of the given nonZero Jacobian row that are not Dirichlet constrained */
+    bool freeDof;
+    Array<int> free_col_indicies;
+    for(int j = 0; j < col_tmp.Size(); j++)
+    {
+      freeDof = true;
+      for(int k = 0; k < DirichletDofs.Size(); k++)
+      {
+        if(col_tmp[j] == DirichletDofs[k])
+        {
+          freeDof = false;
+        }
+      }
+      if(freeDof)
+      {
+        free_col_indicies.Append(j);
+      }
+    }
+
+    Array<int> col_tmp_reduced(free_col_indicies.Size());
+    Vector v_tmp_reduced(free_col_indicies.Size());
+    for(int j = 0; j < free_col_indicies.Size(); j++)
+    {
+      col_tmp_reduced[j] = col_tmp[free_col_indicies[j]];
+      v_tmp_reduced(j)   = v_tmp(free_col_indicies[j]);
+    }
+
+    Jreduced->SetRow(i, col_tmp_reduced, v_tmp_reduced);
+    g0reduced(i) = g0(nonZeroRows[i]);
+  }
+
+  QPOptProblem *QPContact = new QPOptProblem(*K, *Jreduced, f, g0reduced);
+  
+  InteriorPointSolver * QPContactOptimizer = new InteriorPointSolver(QPContact);
+  QPContactOptimizer->SetTol(1.e-6);
+  QPContactOptimizer->SetLinearSolver(linSolver);
+  Vector x0(ndofs); x0 = 0.0;
+  x0.SetVector(x1,0);
+  x0.SetVector(x2,x1.Size());
+
+  Vector xf(ndofs); xf = 0.0;
+  QPContactOptimizer->Mult(x0, xf);
+  
+  MFEM_VERIFY(QPContactOptimizer->GetConverged(), "Interior point solver did not converge.");
+  double Einitial = QPContact->E(x0);
+  double Efinal = QPContact->E(xf);
+  cout << "Energy objective at initial point = " << Einitial << endl;
+  cout << "Energy objective at QP optimizer = " << Efinal << endl;
+ 
+  
+  
+  int gdim = mesh1->Dimension();
+  FiniteElementCollection * fec = new H1_FECollection(1, gdim);
+  FiniteElementSpace * fespace1 = new FiniteElementSpace(mesh1, fec, gdim, Ordering::byVDIM);
+  FiniteElementSpace * fespace2 = new FiniteElementSpace(mesh2, fec, gdim, Ordering::byVDIM);
+   
+  GridFunction x1_gf(fespace1);
+  GridFunction x2_gf(fespace2);
+
+  int ndof1 = fespace1->GetTrueVSize();
+  int ndof2 = fespace2->GetTrueVSize();
+  int ndof  = ndof1 + ndof2;
+  for(int i = 0; i < ndof1; i++)
+  {
+    x1_gf(i) = xf(i);
+  }
+  for(int i = ndof1; i < ndof; i++)
+  {
+    x2_gf(i - ndof1) = xf(i);
+  }
+
+  mesh1->SetNodalFESpace(fespace1);
+  mesh2->SetNodalFESpace(fespace2);
+  GridFunction *nodes1 = mesh1->GetNodes();
+  GridFunction *nodes2 = mesh2->GetNodes();
+
+  {
+    *nodes1 += x1_gf;
+    *nodes2 += x2_gf;
+  }
+  
+
+  ParaViewDataCollection paraview_dc1("QPContactBody1", mesh1);
+  paraview_dc1.SetPrefixPath("ParaView");
+  paraview_dc1.SetLevelsOfDetail(1);
+  paraview_dc1.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc1.SetHighOrderOutput(true);
+  paraview_dc1.SetCycle(0);
+  paraview_dc1.SetTime(0.0);
+  paraview_dc1.RegisterField("Body1", &x1_gf);
+  paraview_dc1.Save();
+  
+  ParaViewDataCollection paraview_dc2("QPContactBody2", mesh2);
+  paraview_dc2.SetPrefixPath("ParaView");
+  paraview_dc2.SetLevelsOfDetail(1);
+  paraview_dc2.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc2.SetHighOrderOutput(true);
+  paraview_dc2.SetCycle(0);
+  paraview_dc2.SetTime(0.0);
+  paraview_dc2.RegisterField("Body2", &x2_gf);
+  paraview_dc2.Save();
+
+  delete fespace1;
+  delete fespace2;
+  delete fec;
+  delete mesh1;
+  delete mesh2;
+  
+  delete QPContact;
+  delete QPContactOptimizer;
+  
+  delete Jreduced;
+  delete contact;
+  return 0;
+}
@@ -0,0 +1,125 @@
+# Copyright (c) 2010-2023, Lawrence Livermore National Security, LLC. Produced
+# at the Lawrence Livermore National Laboratory. All Rights reserved. See files
+# LICENSE and NOTICE for details. LLNL-CODE-806117.
+#
+# This file is part of the MFEM library. For more information and source code
+# availability visit https://mfem.org.
+#
+# MFEM is free software; you can redistribute it and/or modify it under the
+# terms of the BSD-3 license. We welcome feedback and contributions, see file
+# CONTRIBUTING.md for details.
+
+# Use the MFEM build directory
+MFEM_DIR ?= ../..
+MFEM_BUILD_DIR ?= ../../
+SRC = $(if $(MFEM_DIR:../..=),$(MFEM_DIR)/examples/contact/,)
+CONFIG_MK = $(MFEM_BUILD_DIR)/config/config.mk
+
+MFEM_LIB_FILE = mfem_is_not_built
+-include $(CONFIG_MK)
+
+SEQ_EXAMPLES = ObstacleProblem SphericalObstacleProblem DirichletObstacleProblem exQPContactBlockTL
+PAR_EXAMPLES = ParObstacleProblem
+EXAMPLES = $(SEQ_EXAMPLES) $(PAR_EXAMPLES)
+
+ifeq ($(MFEM_USE_SUITESPARSE),NO)
+$(SEQ_EXAMPLES):
+	$(error MFEM is not configured with SUITESPARSE)
+endif
+
+ifeq ($(MFEM_USE_MUMPS),NO)
+ifeq ($(MFEM_USE_MKL_CPARDISO), NO)
+$(PAR_EXAMPLES):
+	$(error MFEM is not configured with MUMPS or CPARDISO)
+endif
+endif
+
+all: $(EXAMPLES)
+
+ObstacleProblem: ObstacleProblem.o Problems.o IPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) ObstacleProblem.o Problems.o IPsolver.o -o $@ $(MFEM_LIBS)
+
+SphericalObstacleProblem: SphericalObstacleProblem.o Problems.o IPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) SphericalObstacleProblem.o Problems.o IPsolver.o -o $@ $(MFEM_LIBS)
+
+DirichletObstacleProblem: DirichletObstacleProblem.o Problems.o IPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) DirichletObstacleProblem.o Problems.o IPsolver.o -o $@ $(MFEM_LIBS)
+
+DirichletObstacleProblemVariant: DirichletObstacleProblemVariant.o Problems.o IPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) DirichletObstacleProblemVariant.o Problems.o IPsolver.o -o $@ $(MFEM_LIBS)
+
+TwoSidedDirichletObstacleProblem: TwoSidedDirichletObstacleProblem.o Problems.o IPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) TwoSidedDirichletObstacleProblem.o Problems.o IPsolver.o -o $@ $(MFEM_LIBS)
+
+
+exQPContactBlockTL: exQPContactBlockTL.o Problems.o IPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) exQPContactBlockTL.o Problems.o IPsolver.o -o $@ $(MFEM_LIBS)
+
+ParTest: ParTest.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) ParTest.o -o $@ $(MFEM_LIBS)
+
+
+ObstacleProblem.o: $(SRC)ObstacleProblem.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $< 
+
+SphericalObstacleProblem.o: $(SRC)SphericalObstacleProblem.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $< 
+
+DirichletObstacleProblem.o: $(SRC)DirichletObstacleProblem.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $<
+
+DirichletObstacleProblemVariant.o: $(SRC)DirichletObstacleProblemVariant.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $<
+
+TwoSidedDirichletObstacleProblem.o: $(SRC)TwoSidedDirichletObstacleProblem.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $<
+
+exQPContactBlockTL.o: $(SRC)exQPContactBlockTL.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $<
+
+Problems.o: $(SRC)Problems.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $< 
+
+IPsolver.o: $(SRC)IPsolver.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $<
+
+ParObstacleProblem: ParObstacleProblem.o ParProblems.o ParIPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) ParObstacleProblem.o ParProblems.o ParIPsolver.o -o $@ $(MFEM_LIBS)
+
+ParSphericalObstacleProblem: ParSphericalObstacleProblem.o ParProblems.o ParIPsolver.o $(MFEM_LIB_FILE)
+	$(MFEM_CXX) $(MFEM_FLAGS) ParSphericalObstacleProblem.o ParProblems.o ParIPsolver.o -o $@ $(MFEM_LIBS)
+
+ParObstacleProblem.o: $(SRC)ParObstacleProblem.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $< 
+
+ParSphericalObstacleProblem.o: $(SRC)ParSphericalObstacleProblem.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $< 
+
+ParProblems.o: $(SRC)ParProblems.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $< 
+
+ParIPsolver.o: $(SRC)ParIPsolver.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $<
+
+ParTest.o: $(SRC)ParTest.cpp $(CONFIG_MK)
+	$(MFEM_CXX) $(MFEM_FLAGS) -c $<
+
+clean: clean-build 
+
+clean-build:
+	rm -f *.o *~ $(SEQ_EXAMPLES) $(PAR_EXAMPLES)
+	rm -rf *.dSYM *.TVD.*breakpoints
+
+# For out-of-source builds, link the data files from the source tree:
+ifneq ($(SRC),)
+DATA_FILES = block1.mesh rotatedblock2.mesh
+$(DATA_FILES): %: $(SRC)%
+	ln -sf $(<) .
+copy-data: | $(DATA_FILES)
+# For out-of-source builds, the test and sample runs for 'field-interp' need
+# data from the meshing miniapps directory:
+exQPContactBlockTL: | mesh-data
+.PHONY: mesh-data
+mesh-data:
+	$(MAKE) -C ./ copy-data
+endif
@@ -0,0 +1,896 @@
+
+using namespace std;
+using namespace mfem;
+
+void BasisEval(const Vector xi, Vector &N, DenseMatrix &dNdxi) // dNdxi is 2*4
+{
+   N[0] = 0.25*(1-xi[0])*(1-xi[1]);
+   N[1] = 0.25*(1+xi[0])*(1-xi[1]);
+   N[2] = 0.25*(1+xi[0])*(1+xi[1]);
+   N[3] = 0.25*(1-xi[0])*(1+xi[1]);
+
+   dNdxi(0,0) = 0.25*(-1+xi[1]);
+   dNdxi(0,1) = 0.25*(1-xi[1]);
+   dNdxi(0,2) = 0.25*(1+xi[1]);
+   dNdxi(0,3) = 0.25*(-1-xi[1]);
+   dNdxi(1,0) = 0.25*(-1+xi[0]);
+   dNdxi(1,1) = 0.25*(-1-xi[0]);
+   dNdxi(1,2) = 0.25*(1+xi[0]);
+   dNdxi(1,3) = 0.25*(1-xi[0]);
+}
+
+
+void BasisEvalDerivs(const Vector xi, Vector& N, DenseMatrix& dNdxi,
+                     DenseMatrix& dN2dxi)
+{
+   N[0] = 0.25*(1-xi[0])*(1-xi[1]);
+   N[1] = 0.25*(1+xi[0])*(1-xi[1]);
+   N[2] = 0.25*(1+xi[0])*(1+xi[1]);
+   N[3] = 0.25*(1-xi[0])*(1+xi[1]);
+
+   dNdxi.SetSize(2,4); dNdxi = 0.0;
+   dN2dxi.SetSize(3,4);
+   dN2dxi = 0.0;  // first row dxi2, second detadxi, third deta2
+
+   dNdxi(0,0) = 0.25*(-1+xi[1]); dNdxi(0,1) = 0.25*(1-xi[1]);
+   dNdxi(0,2) = 0.25*(1+xi[1]);  dNdxi(0,3) = 0.25*(-1-xi[1]);
+   dNdxi(1,0) = 0.25*(-1+xi[0]); dNdxi(1,1) = 0.25*(-1-xi[0]);
+   dNdxi(1,2) = 0.25*(1+xi[0]);  dNdxi(1,3) = 0.25*(1-xi[0]);
+
+   dN2dxi(1,0) = 0.25; dN2dxi(1,1) = -0.25; dN2dxi(1,2) = 0.25;
+   dN2dxi(1,3) = -0.25;
+}
+
+// returns the vector and matrix form of the shape functions and its derivative
+void BasisVectorDerivs(const Vector xi, DenseMatrix& N, DenseMatrix& dNdxi,
+                       DenseMatrix& ddNdxi)
+{
+   N.SetSize(3,12); N = 0.0;
+   N(0,0) = 0.25*(1-xi[0])*(1-xi[1]); N(0,3) = 0.25*(1+xi[0])*(1-xi[1]);
+   N(0,6) = 0.25*(1+xi[0])*(1+xi[1]); N(0,9) = 0.25*(1-xi[0])*(1+xi[1]);
+
+   N(1,1) = 0.25*(1-xi[0])*(1-xi[1]); N(1,4) = 0.25*(1+xi[0])*(1-xi[1]);
+   N(1,7) = 0.25*(1+xi[0])*(1+xi[1]); N(1,10) = 0.25*(1-xi[0])*(1+xi[1]);
+
+   N(2,2) = 0.25*(1-xi[0])*(1-xi[1]); N(2,5) = 0.25*(1+xi[0])*(1-xi[1]);
+   N(2,8) = 0.25*(1+xi[0])*(1+xi[1]); N(2,11) = 0.25*(1-xi[0])*(1+xi[1]);
+
+   dNdxi.SetSize(3*2, 3*4);  dNdxi = 0.0;
+   dNdxi(0,0) = 0.25*(-1+xi[1]);  dNdxi(0,3) = 0.25*(1-xi[1]);
+   dNdxi(0,6) = 0.25*(1+xi[1]);  dNdxi(0,9) = 0.25*(-1-xi[1]);
+   dNdxi(1,1) = 0.25*(-1+xi[1]);  dNdxi(1,4) = 0.25*(1-xi[1]);
+   dNdxi(1,7) = 0.25*(1+xi[1]);  dNdxi(1,10) = 0.25*(-1-xi[1]);
+   dNdxi(2,2) = 0.25*(-1+xi[1]);  dNdxi(2,5) = 0.25*(1-xi[1]);
+   dNdxi(2,8) = 0.25*(1+xi[1]);  dNdxi(2,11) = 0.25*(-1-xi[1]);
+
+   dNdxi(3,0) = 0.25*(-1+xi[0]);  dNdxi(3,3) = 0.25*(-1-xi[0]);
+   dNdxi(3,6) = 0.25*(1+xi[0]);  dNdxi(3,9) = 0.25*(1-xi[0]);
+   dNdxi(4,1) = 0.25*(-1+xi[0]);  dNdxi(4,4) = 0.25*(-1-xi[0]);
+   dNdxi(4,7) = 0.25*(1+xi[0]);  dNdxi(4,10) = 0.25*(1-xi[0]);
+   dNdxi(5,2) = 0.25*(-1+xi[0]);  dNdxi(5,5) = 0.25*(-1-xi[0]);
+   dNdxi(5,8) = 0.25*(1+xi[0]);  dNdxi(5,11) = 0.25*(1-xi[0]);
+
+   ddNdxi.SetSize(3*4, 3*4); ddNdxi = 0.0;
+   ddNdxi(3,0) = 0.25; ddNdxi(3,3) = -0.25;
+   ddNdxi(3,6) = 0.25; ddNdxi(3,9) = -0.25;
+   ddNdxi(4,1) = 0.25; ddNdxi(4,4) = -0.25;
+   ddNdxi(4,7) = 0.25; ddNdxi(4,10) = -0.25;
+   ddNdxi(5,2) = 0.25; ddNdxi(5,5) = -0.25;
+   ddNdxi(5,8) = 0.25; ddNdxi(5,11) = -0.25;
+
+   ddNdxi(6,0) = 0.25; ddNdxi(6,3) = -0.25;
+   ddNdxi(6,6) = 0.25; ddNdxi(6,9) = -0.25;
+   ddNdxi(7,1) = 0.25; ddNdxi(7,4) = -0.25;
+   ddNdxi(7,7) = 0.25; ddNdxi(7,10) = -0.25;
+   ddNdxi(8,2) = 0.25; ddNdxi(8,5) = -0.25;
+   ddNdxi(8,8) = 0.25; ddNdxi(8,11) = -0.25;
+}
+
+
+void cross(const Vector a, const Vector b, Vector& c)
+{
+   assert(a.Size()==3);
+   c.SetSize(3);
+   c[0] = a[1]*b[2] - a[2]*b[1];
+   c[1] = -a[0]*b[2] + b[0]*a[2];
+   c[2] = a[0]*b[1] - a[1]*b[0];
+
+}
+// a outer b
+void outer(const Vector a, const Vector b, DenseMatrix& c)
+{
+   int m = a.Size();
+   int n = b.Size();
+   assert(c.Height()==m);
+   assert(c.Width() ==n);
+   for (int i=0; i<m; i++)
+   {
+      for (int j=0; j<n; j++)
+      {
+         c(i,j) = a[i]*b[j];
+      }
+   }
+}
+// dphidxi 2*4
+// coords 4*3
+void ComputeNormal(const DenseMatrix& dphidxi, const DenseMatrix& coords,
+                   Vector& normal, double& nnorm)
+{
+
+   DenseMatrix dxdxi(2,3);
+   Mult(dphidxi, coords, dxdxi);
+   Vector dxdxi1(3);
+   Vector dxdxi2(3);
+
+   dxdxi.GetRow(0,dxdxi1);
+   dxdxi.GetRow(1,dxdxi2);
+
+   cross(dxdxi1, dxdxi2, normal);  // is there a cross product?  no
+   // VectorCrossProductCoefficient::Eval has hard-coded cross product
+   nnorm = normal.Norml2( );
+   normal /=  nnorm;
+}
+
+void SlaveToMaster(const DenseMatrix& m_coords, const Vector& s_x, Vector& xi)
+{
+   bool converged = false;
+   bool pt_on_elem = false;
+   int dim = 3;
+   xi.SetSize(dim-1);
+   xi = 0.0;
+   int max_iter = 15;
+   double off_el_xi = 1e-2;
+   double proj_newton_tol = 1e-13;
+   double proj_max_gap = 0.5;
+   Vector gap_v(dim);
+   // warm start from linear solution
+
+   for (int it=0; it<max_iter; it++)
+   {
+      //cout<<it<<endl;
+      Vector m_N(4);
+      m_N = 0.;
+      DenseMatrix m_dN(2,4);
+      m_dN = 0.;
+      DenseMatrix m_dN2(3,4);
+      m_dN2 = 0.;
+      BasisEvalDerivs(xi, m_N, m_dN, m_dN2);
+
+      Vector x_c(dim);
+      m_coords.MultTranspose(m_N, x_c);
+
+      gap_v = s_x;
+      gap_v -= x_c;
+
+      DenseMatrix m_dx(2,3);
+      m_dx = 0.;
+      Mult(m_dN, m_coords, m_dx);
+
+      Vector r(dim-1);
+      r = 0.0;
+      m_dx.Mult(gap_v, r);
+
+      if (r.Normlinf() < proj_newton_tol)
+      {
+         converged = true;
+         break;
+      }
+
+      DenseMatrix drdxi(dim-1,dim-1);
+      drdxi = 0.;
+      MultABt(m_dx, m_dx, drdxi); // m_dx * m_dx.T
+      drdxi *= -1.0;
+
+      DenseMatrix m_dx2(3,3); m_dx2 = 0.0;
+      Mult(m_dN2,m_coords, m_dx2);
+
+      //m_d2x = m_dN(:,:,2) * m_elem_coords(1:4,:);  //m_dN(:,:,2) is 3*4
+      for (int d=0; d<3; d++)
+      {
+         DenseMatrix Mtemp(2,2); Mtemp = 0.0;
+         Mtemp(0,0) = m_dx2(0,d); Mtemp(0,1) = m_dx2(1,d);
+         Mtemp(1,0) = m_dx2(1,d); Mtemp(1,1) = m_dx2(2,d);
+
+         drdxi.Add(gap_v[d], Mtemp);
+      }
+
+      //cond_num = rcond(drdxi);  condition number?
+      //drdxi.TestInversion();
+      DenseMatrixInverse drdxi_inv(drdxi);
+      Vector xi_tmp(dim-1);
+
+      drdxi_inv.Mult(r,xi_tmp);
+      xi -= xi_tmp;
+   }
+   if (!converged)
+   {
+      xi = 0.0;
+   }
+   off_el_xi += 1 ; // tolerance of offset of xi outside [-1,1]
+
+   //cout<<gap_v.Norml2()<<" " <<xi.Normlinf()<<endl;
+   //
+   // Discuss with Frank... what is happening here
+   if (gap_v.Norml2() < proj_max_gap && xi.Normlinf() <= off_el_xi)
+   {
+      pt_on_elem = true;
+   }
+
+   if (pt_on_elem)
+   {
+     //cout << "convergence of node to segment projection? " << converged << endl;
+     //for(int i = 0; i < 2; i++)
+     //{
+     //  cout << "xi_" << i << " = " << xi(i) << endl;
+     //}
+   }
+   MFEM_VERIFY(pt_on_elem == true, "xi went out of bounds");
+   MFEM_VERIFY(converged == true, "projection didn't converge");
+}
+
+
+
+// m_coords is expected to be 4 * 3
+void  ComputeGapJacobian(const Vector x_s, const Vector xi,
+                         const DenseMatrix m_coords,
+                         double& gap, Vector& normal, Vector& dgdxm, Vector& dgdxs)
+{
+   Vector m_N(4);
+   DenseMatrix m_dN(2,4);
+   DenseMatrix m_dN2(3,4);
+   BasisEvalDerivs(xi, m_N, m_dN, m_dN2);
+
+   Vector x_c(3);
+   m_coords.MultTranspose(m_N, x_c);
+
+   Vector gap_v(3); gap_v = 0.0;
+   gap_v  = x_s;
+   gap_v -= x_c;
+
+   DenseMatrix m_dx(2,3);
+   Mult(m_dN, m_coords, m_dx);
+
+   double nnorm = 0;
+   ComputeNormal(m_dN, m_coords, normal, nnorm);
+
+   gap = gap_v * normal; // gap function value, dot product between vectors
+
+   //dr_dx = zeros(2,4,3); % nsegment, nodes in quad, ndim
+
+   DenseMatrix dr_dx_res1(4,3); dr_dx_res1 = 0.;
+   DenseMatrix dr_dx_res2(4,3); dr_dx_res2 = 0.;
+
+   Vector m_dxrow1(3);
+   m_dx.GetRow(0, m_dxrow1);
+   outer(m_N, m_dxrow1, dr_dx_res1);// 4*1 times 1*3
+   dr_dx_res1 *= -1.0;
+
+   Vector m_dxrow2(3);
+   m_dx.GetRow(1, m_dxrow2);
+   outer(m_N, m_dxrow2, dr_dx_res2);// 4*1 times 1*3
+   dr_dx_res2 *= -1.0;
+
+   Vector m_dNrow1(4); m_dN.GetRow(0, m_dNrow1);
+   Vector m_dNrow2(4); m_dN.GetRow(1, m_dNrow2);
+
+   DenseMatrix dr_dx_res1_tmp(4,3); dr_dx_res1_tmp = 0.;
+   DenseMatrix dr_dx_res2_tmp(4,3); dr_dx_res2_tmp = 0.;
+   outer(m_dNrow1, gap_v, dr_dx_res1_tmp);// 4*1 times 1*3
+   outer(m_dNrow2, gap_v, dr_dx_res2_tmp);// 4*1 times 1*3
+
+   dr_dx_res1 += dr_dx_res1_tmp; // outer product in vector?
+   dr_dx_res2 += dr_dx_res2_tmp;
+
+
+   DenseMatrix K_dxidx1(2,2); // 2*2
+   K_dxidx1 = 0.;
+   MultABt(m_dx, m_dx, K_dxidx1); // m_dx * m_dx.T
+
+   Vector v_dxidx2(4);
+   m_coords.Mult(gap_v, v_dxidx2); // m_coords * gap_v;  // 4*3 * 3 = 4
+
+   DenseMatrix K_dxidx2(2,2); K_dxidx2 = 0.0;
+
+   Vector m_dN2row1(4); m_dN2.GetRow(0, m_dN2row1);
+   Vector m_dN2row2(4); m_dN2.GetRow(1, m_dN2row2);
+   Vector m_dN2row3(4); m_dN2.GetRow(2, m_dN2row3);
+   // how to get 2nd order? multidimensional matrix?
+   K_dxidx2(0,0) = m_dN2row1 * v_dxidx2; // how would 4*1 * 1*4 be computed?
+   K_dxidx2(0,1) = m_dN2row2 * v_dxidx2;
+   K_dxidx2(1,0) = m_dN2row2 * v_dxidx2;
+   K_dxidx2(1,1) = m_dN2row3 * v_dxidx2;
+
+   DenseMatrix K_dxidx(2,2);
+   K_dxidx -= K_dxidx1;
+   K_dxidx += K_dxidx2;
+
+   // resize the vectors and matrices
+   Vector dxidx(24); dxidx = 0.0;
+   Vector drdx_r(24); drdx_r = 0.0;
+
+   for (int i=0; i<4; i++)
+   {
+      for (int j=0; j<3; j++)
+      {
+         drdx_r[4*j+i] = dr_dx_res1(i,j);
+         drdx_r[4*j+i+12] = dr_dx_res2(i,j);
+
+      }
+   }
+   //drdx_r(1:4*3,1) = reshape(dr_dx_res(:,:,1),4*3,1);
+   //drdx_r(4*3+1:2*4*3,1) = reshape(dr_dx_res(:,:,2),4*3,1);
+   DenseMatrix drdx_K(24,24); drdx_K = 0.;
+   for (int i =0; i<12; i++)
+   {
+      drdx_K(i,i) = K_dxidx(0,0);
+      drdx_K(i,12+i) = K_dxidx(0,1);
+      drdx_K(12+i,i) = K_dxidx(1,0);
+      drdx_K(12+i,12+i) = K_dxidx(1,1);
+   }
+
+   DenseMatrixInverse drdxK_inv(drdx_K);
+   drdxK_inv.Mult(drdx_r,dxidx);
+   // LinearSolve (drdx_K,drdx_r, dxidx) ; //???
+   dxidx *= -1.0;
+
+
+
+   Vector drdxs_r(6);
+   drdxs_r[0] = m_dx(0,0); drdxs_r[1] = m_dx(0,1); drdxs_r[2] = m_dx(0,2);
+   drdxs_r[3] = m_dx(1,0); drdxs_r[4] = m_dx(1,1); drdxs_r[5] = m_dx(1,2);
+
+   DenseMatrix drdxs_K(6,6); drdxs_K = 0.;
+   for (int i=0; i<3; i++)
+   {
+      drdxs_K(i,i) = K_dxidx(0,0);
+      drdxs_K(i,3+i) = K_dxidx(0,1);
+      drdxs_K(i+3,i) = K_dxidx(1,0);
+      drdxs_K(i+3,i+3) = K_dxidx(1,1);
+   }
+
+   Vector dxidxs(6); dxidxs = 0.0;
+   DenseMatrixInverse drdxsK_inv(drdxs_K);
+   drdxsK_inv.Mult(drdxs_r,dxidxs);
+   dxidxs *= -1.0;
+   //dxidxs = -drdxs_K\drdxs_r;
+
+   //dxidx = reshape(dxidx, 4,3,2); dxidxs = reshape(dxidxs, 1,3,2);
+
+   dgdxm.SetSize(12); dgdxm = 0.;
+   DenseMatrix dgdxm_tmp(4,3);
+   outer(m_N, normal,dgdxm_tmp);
+   for (int i=0; i<4; i++)
+   {
+      for (int j=0; j<3; j++)
+      {
+         dgdxm[3*i+j] = -dgdxm_tmp(i,j);
+      }
+   }
+   //dxidx_M = -m_dN(1:2,:,1) * (m_coords(1:4,:)*normal'); % this turns out to be 0
+
+   dgdxs.SetSize(3);
+   dgdxs += normal;
+   //dgdxs = dgdxs + dxidx_M(1) * dxidxs(:,:,1) + dxidx_M(2) * dxidxs(:,:,2);
+};
+
+void ComputeGapHessian(const Vector x_s, const Vector xi,
+                       const DenseMatrix m_coords,
+                       DenseMatrix& dg2dx)
+{
+   Vector m_N(4);
+   DenseMatrix m_dN(2,4);
+   DenseMatrix m_dN2(3,4);
+   BasisEvalDerivs(xi, m_N, m_dN, m_dN2);
+
+   int dim = 3;
+   int num_dofs1 = dim;
+   int num_dofs2 = 4*dim;
+   int num_dofs = num_dofs1 + num_dofs2;
+   dg2dx.SetSize(num_dofs,num_dofs); dg2dx = 0.0;
+
+   Vector x_c(3);
+   m_coords.MultTranspose(m_N,x_c);
+
+   Vector gap_v(3); gap_v = 0.0;
+   gap_v  = x_s;
+   gap_v -= x_c;
+
+   DenseMatrix m_dx(2,3);
+   Mult(m_dN, m_coords, m_dx);
+
+   DenseMatrix m_dx2(3,3); m_dx2 = 0.0;
+   Mult(m_dN2,m_coords, m_dx2);
+   double nnorm = 0.0;
+   Vector normal(3); normal = 0.0;
+   ComputeNormal(m_dN, m_coords, normal, nnorm);
+
+   double gap = gap_v * normal; // gap function value, dot product between vectors
+
+   DenseMatrix M(2,2); M = 0.0;
+   MultABt(m_dx, m_dx, M);
+
+   DenseMatrix f(2, num_dofs2); f = 0.0;
+
+   for (int d=0; d<3; d++)
+   {
+      DenseMatrix Mtemp(2,2); Mtemp = 0.0;
+      Mtemp(0,0) = m_dx2(0,d); Mtemp(0,1) = m_dx2(1,d);
+      Mtemp(1,0) = m_dx2(1,d); Mtemp(1,1) = m_dx2(2,d);
+
+      M.Add(-gap_v[d], Mtemp);
+
+      Vector m_dxcol(2); m_dx.GetColumn(d, m_dxcol);
+      DenseMatrix ftmp(2,4);
+      outer(m_dxcol, m_N, ftmp);
+      ftmp *= -1;
+      ftmp.Add( gap_v[d], m_dN);   // 2*4
+
+      for (int j=0; j<4; j++)
+      {
+         assert(d+3*j<num_dofs2);
+         f(0,d+j*3) = ftmp(0,j);
+         f(1,d+j*3) = ftmp(1,j);
+      }
+   }
+   //fprintf('hess dxidxm\n');
+   DenseMatrixInverse Minv(M);
+   DenseMatrix dxidxm(2,num_dofs2); dxidxm = 0.0;
+   Minv.Mult(f, dxidxm);
+   //LinearSolve??
+   //dxidxm = M\f;
+
+   DenseMatrix nde2(2,2); nde2 = 0.0;
+   DenseMatrix Nndx2(2,num_dofs2); Nndx2 = 0.0;
+
+   for (int d=0; d<3; d++)
+   {
+      DenseMatrix ndetmp(2,2); ndetmp = 0.0;
+      ndetmp(0,0) = normal(d)*m_dx2(0,d); ndetmp(0,1) = normal(d)*m_dx2(1,d);
+      ndetmp(1,0) = normal(d)*m_dx2(1,d); ndetmp(1,1) = normal(d)*m_dx2(2,d);
+
+      nde2 += ndetmp;
+
+      for (int j=0; j<4; j++)
+      {
+         assert(d+3*j<num_dofs2);
+         Nndx2(0,d+j*3) = normal[d]*m_dN(0,j);
+         Nndx2(1,d+j*3) = normal[d]*m_dN(1,j);
+      }
+
+   }
+
+   DenseMatrix Ndn(2,num_dofs2); Ndn = 0.0;
+   Ndn += Nndx2;
+   AddMult(nde2, dxidxm, Ndn);
+
+
+   DenseMatrix M2(2,2); M2 = 0.0;
+   MultABt(m_dx, m_dx, M2);
+   DenseMatrixInverse M2inv(M2);
+   DenseMatrix diag2(2,2); diag2(0,0) = 1.0; diag2(1,1) = 1.0;
+   DenseMatrix m_con(2,2); m_con = 0.0;
+
+   M2inv.Mult(diag2, m_con);
+
+   DenseMatrix dg2dxm(num_dofs2, num_dofs2); dg2dxm = 0.0;
+
+   DenseMatrix dg2dxm_tmp(num_dofs2,2); dg2dxm_tmp = 0.0;
+   MultAtB(Ndn, m_con, dg2dxm_tmp);
+   Mult(dg2dxm_tmp, Ndn, dg2dxm);
+   dg2dxm *= gap;
+
+   DenseMatrix dg2dxm_tmp2(num_dofs2,num_dofs2); dg2dxm_tmp2 = 0.0;
+   MultAtB(Nndx2, dxidxm, dg2dxm_tmp2);
+   dg2dxm.Add(-1.0, dg2dxm_tmp2);
+
+   dg2dxm_tmp = 0.0;
+   MultAtB(dxidxm, nde2, dg2dxm_tmp);
+
+   AddMult_a(-1.0, dg2dxm_tmp, dxidxm, dg2dxm);
+
+   dg2dxm_tmp2 = 0.0;
+   MultAtB(dxidxm, Nndx2, dg2dxm_tmp2);
+   dg2dxm.Add(-1.0, dg2dxm_tmp2);
+
+   Vector v_dxidx2(4);
+   m_coords.Mult(gap_v, v_dxidx2); // m_coords * gap_v;  // 4*3 * 3 = 4
+
+   DenseMatrix K_dxidx2(2,2); K_dxidx2 = 0.0;
+
+   Vector m_dN2row1(4); m_dN2.GetRow(0, m_dN2row1);
+   Vector m_dN2row2(4); m_dN2.GetRow(1, m_dN2row2);
+   Vector m_dN2row3(4); m_dN2.GetRow(2, m_dN2row3);
+   K_dxidx2(0,0) = m_dN2row1 * v_dxidx2; // how would 4*1 * 1*4 be computed?
+   K_dxidx2(0,1) = m_dN2row2 * v_dxidx2;
+   K_dxidx2(1,0) = m_dN2row2 * v_dxidx2;
+   K_dxidx2(1,1) = m_dN2row3 * v_dxidx2;
+
+   DenseMatrix K_dxidx(2,2);
+   K_dxidx -= M2;
+   K_dxidx += K_dxidx2;
+
+   Vector drdxs_r(6);
+   drdxs_r[0] = m_dx(0,0); drdxs_r[1] = m_dx(0,1); drdxs_r[2] = m_dx(0,2);
+   drdxs_r[3] = m_dx(1,0); drdxs_r[4] = m_dx(1,1); drdxs_r[5] = m_dx(1,2);
+
+   DenseMatrix drdxs_K(6,6); drdxs_K = 0.;
+   for (int i=0; i<3; i++)
+   {
+      drdxs_K(i,i) = K_dxidx(0,0);
+      drdxs_K(i,3+i) = K_dxidx(0,1);
+      drdxs_K(i+3,i) = K_dxidx(1,0);
+      drdxs_K(i+3,i+3) = K_dxidx(1,1);
+   }
+   Vector dxidxs(6);
+
+   DenseMatrixInverse drdxsK_inv(drdxs_K);
+   drdxsK_inv.Mult(drdxs_r,dxidxs);
+   dxidxs *= -1.0;
+   //dxidxs = -drdxs_K\drdxs_r;
+
+   DenseMatrix dxidxs_m(2,3); dxidxs_m = 0.0;
+   dxidxs_m(0,0) = dxidxs[0]; dxidxs_m(0,1) = dxidxs[1]; dxidxs_m(0,2) = dxidxs[2];
+   dxidxs_m(1,0) = dxidxs[3]; dxidxs_m(1,1) = dxidxs[4]; dxidxs_m(1,2) = dxidxs[5];
+
+   DenseMatrix dtao1dxs(3,3); dtao1dxs = 0.0;
+   DenseMatrix dtao2dxs(3,3); dtao2dxs = 0.0;
+
+   Vector dxidxs_row1(3); dxidxs_row1 = 0.0; Vector dxidxs_row2(3);
+   dxidxs_row2 = 0.0;
+   Vector mdx2_row1(3); mdx2_row1 = 0.0; Vector mdx2_row2(3); mdx2_row2 = 0.0;
+   Vector mdx2_row3(3); mdx2_row3 = 0.0;
+   dxidxs_m.GetRow(0,dxidxs_row1);
+   dxidxs_m.GetRow(1,dxidxs_row2);
+   m_dx2.GetRow(0,mdx2_row1);
+   m_dx2.GetRow(1,mdx2_row2);
+   m_dx2.GetRow(2,mdx2_row3);
+
+   DenseMatrix dtaotmp(3,3); dtaotmp = 0.0;
+   outer(mdx2_row1, dxidxs_row1,dtaotmp);
+   dtao1dxs += dtaotmp; dtaotmp = 0.0;
+   outer(mdx2_row2, dxidxs_row1,dtaotmp);
+   dtao1dxs += dtaotmp; dtaotmp = 0.0;
+
+   outer(mdx2_row2, dxidxs_row2, dtaotmp);
+   dtao2dxs += dtaotmp; dtaotmp = 0.0;
+   outer(mdx2_row3, dxidxs_row2, dtaotmp);
+   dtao2dxs += dtaotmp; dtaotmp = 0.0;
+
+   DenseMatrix dtaodxs(3,3); dtaodxs = 0.0; //tao = tao1 cross tao2
+
+   for (int d=0; d<3; d++)
+   {
+      Vector dtao1dxs_tmp(3); dtao1dxs_tmp = 0.0;
+      dtao1dxs.GetColumn(d,dtao1dxs_tmp);
+      Vector m_dxrow(3); m_dx.GetRow(1, m_dxrow);
+
+      Vector dtaodxs_tmp(3); dtaodxs_tmp = 0.0;
+      cross(dtao1dxs_tmp, m_dxrow, dtaodxs_tmp);
+
+      Vector dtaodxs_tmp2(3); dtaodxs_tmp2 = 0.0;
+      m_dx.GetRow(0, m_dxrow);
+      dtao1dxs_tmp = 0.0;  // reuse the same vector for dtao2
+      dtao2dxs.GetColumn(d,dtao1dxs_tmp);
+      cross(m_dxrow, dtao1dxs_tmp, dtaodxs_tmp2);
+
+      dtaodxs_tmp2 += dtaodxs_tmp;
+      dtaodxs.SetCol(d, dtaodxs_tmp2);
+   }
+
+   DenseMatrix dndxs(3,3); dndxs = 0.0; dndxs += dtaodxs; dndxs *= 1.0/nnorm;
+   DenseMatrix dndxs_tmp(3,3); dndxs_tmp = 0.0;
+   outer(normal, normal, dndxs_tmp);
+   AddMult_a(-1/nnorm, dndxs_tmp, dtaodxs, dndxs);
+
+   DenseMatrix dgvdxs(3,3); dgvdxs = 0.0;
+   MultAtB(m_dx, dxidxs_m, dgvdxs);
+   dgvdxs *= -1;
+   for (int d=0; d<3; d++)
+   {
+      dgvdxs(d,d) += 1.0;
+   }
+   //dxidxs: 2*3
+
+   DenseMatrix dg2dxs(3,3); dg2dxs = 0.0;
+   DenseMatrix dg2dxs_tmp(3,2); dg2dxs_tmp = 0.0;
+   MultAtB(dxidxs_m, nde2, dg2dxs_tmp);
+   AddMult_a(-1.0, dg2dxs_tmp, dxidxs_m, dg2dxs);
+   DenseMatrix dg2dxs_tmp2(3,3); dg2dxs_tmp2 = 0.0;
+   MultAtB(dgvdxs, dndxs, dg2dxs_tmp2);
+   dg2dxs += dg2dxs_tmp2;
+   dg2dxs_tmp2 = 0.0;
+   MultAtB(dndxs, dndxs_tmp, dg2dxs_tmp2);
+   AddMult(dg2dxs_tmp2, dgvdxs, dg2dxs);
+
+   DenseMatrix Ne(3,12), Be(6,12), dBe(12,12);
+   BasisVectorDerivs(xi, Ne, Be, dBe);
+
+   DenseMatrix dtao1dxm(3,12); dtao1dxm.CopyRows(Be, 0, 2);
+   DenseMatrix dtao2dxm(3,12); dtao2dxm.CopyRows(Be, 3, 5);
+
+   Vector m_coords_v(12);
+   for (int i=0; i<4; i++)
+   {
+      for (int j=0; j<3; j++)
+      {
+         m_coords_v[i*3+j] = m_coords(i,j);
+      }
+   }
+
+   for (int i=0; i<2; i++)
+   {
+      Vector dxidxm_tmp(num_dofs2); dxidxm_tmp = 0.0;
+      dxidxm.GetRow(i,dxidxm_tmp);
+
+      DenseMatrix dBe_tmp(3,12);
+      dBe_tmp.CopyRows(dBe,i*3,(i+1)*3-1);
+
+      DenseMatrix dtaodxm_tmp(12,12); dtaodxm_tmp = 0.0;
+      outer(m_coords_v, dxidxm_tmp, dtaodxm_tmp);
+      AddMult(dBe_tmp, dtaodxm_tmp, dtao1dxm);
+
+      //dtao1dxm += dBe(:,:,i)*reshape(m_coords(1:4,:)',12,1)*reshape(dxidxm(i,:),1,12); % 3*12
+      dBe_tmp = 0.0;
+      dBe_tmp.CopyRows(dBe,(i+2)*3,(i+3)*3-1);
+      AddMult(dBe_tmp, dtaodxm_tmp, dtao2dxm);
+
+   }
+
+   DenseMatrix dtaodxm(3,12); dtaodxm = 0.0;//tao = tao1 cross tao2
+
+   for (int d=0; d<12; d++)
+   {
+      Vector dtaodxm_tmp(3); dtaodxm_tmp = 0.0;
+      Vector dtaodxm_tmp2(3); dtaodxm_tmp2 = 0.0;
+      Vector tmp1(3); tmp1 = 0.0;  dtao1dxm.GetColumn(d,tmp1);
+      Vector m_dxrow2(3); m_dx.GetRow(1, m_dxrow2);
+      Vector m_dxrow1(3); m_dx.GetRow(0, m_dxrow1);
+      Vector tmp2(3); tmp2 = 0.0;  dtao2dxm.GetColumn(d,tmp2);
+
+      cross(tmp1, m_dxrow2, dtaodxm_tmp);
+      cross(m_dxrow1,tmp2, dtaodxm_tmp2);
+      dtaodxm_tmp += dtaodxm_tmp2;
+
+      dtaodxm.SetCol(d, dtaodxm_tmp);
+   }
+
+   DenseMatrix dndxm(3,12); dndxm = 0.0;
+   dndxm += dtaodxm;
+   dndxm *= 1.0/nnorm;
+   AddMult_a(-1/nnorm, dndxs_tmp, dtaodxm, dndxm); //dndxs_tmp = normal'*normal
+
+   DenseMatrix dgvdxm(3,12); dgvdxm = 0.0;
+   dgvdxm -= Ne;
+
+   for (int i=0; i<2; i++)
+   {
+      Vector dxidxm_tmp(num_dofs2); dxidxm_tmp = 0.0;
+      dxidxm.GetRow(i,dxidxm_tmp);
+
+      DenseMatrix Be_tmp(3,12);
+      Be_tmp.CopyRows(Be,i*3,(i+1)*3-1);
+
+      DenseMatrix dgvdxm_tmp(12,12); dgvdxm_tmp = 0.0;
+      outer(m_coords_v, dxidxm_tmp, dgvdxm_tmp);
+      AddMult_a(-1.0, Be_tmp, dgvdxm_tmp, dgvdxm);
+
+   }
+
+   DenseMatrix dg2dxsxm(3,12); dg2dxsxm = 0.0;
+   DenseMatrix dg2dxsxm_tmp(3,3); dg2dxsxm_tmp = 0.0;
+   MultAtB(dgvdxs, dndxm, dg2dxsxm);
+
+   MultAtB(dndxs, dndxs_tmp, dg2dxsxm_tmp);
+   AddMult(dg2dxsxm_tmp, dgvdxm, dg2dxsxm); // += dndxs'*normal'*normal*dgvdxm;
+
+   DenseMatrix dgvdxsxmn(3,12); dgvdxsxmn = 0.0;
+   DenseMatrix dgvdxsxmn_tmp(3,2); dgvdxsxmn_tmp = 0.0;
+   MultAtB(dxidxs_m, nde2, dgvdxsxmn_tmp);   //dxidxs_m: 2*3
+
+   AddMult_a(-1.0, dgvdxsxmn_tmp, dxidxm, dgvdxsxmn);
+
+
+   for (int i =0; i<2; i++)
+   {
+      DenseMatrix Be_tmp(3,12);
+      Be_tmp.CopyRows(Be,i*3,(i+1)*3-1);
+
+      Vector dxidxs_row(3); dxidxs_row = 0.0; dxidxs_m.GetRow(i,dxidxs_row);
+      DenseMatrix dgvdxsxmn_tmp2(3,3); dgvdxsxmn_tmp2 = 0.0;
+      outer(dxidxs_row, normal, dgvdxsxmn_tmp2);
+      AddMult_a(-1.0, dgvdxsxmn_tmp2, Be_tmp, dgvdxsxmn);
+   }
+
+   dg2dxsxm += dgvdxsxmn;
+
+   DenseMatrix dg2dxmxs(12,3); dg2dxmxs = 0.0;
+   DenseMatrix dg2dxmxs_tmp(12,3); dg2dxmxs_tmp = 0.0;
+   MultAtB(dgvdxm, dndxs, dg2dxmxs);
+   MultAtB(dndxm, dndxs_tmp, dg2dxmxs_tmp);
+   AddMult(dg2dxmxs_tmp, dgvdxs, dg2dxmxs);
+
+   DenseMatrix dgvdxmxsn(12,3); dgvdxmxsn = 0.0;
+   DenseMatrix dgvdxmxsn_tmp(12,2); dgvdxmxsn_tmp = 0.0;
+
+   MultAtB(dxidxm, nde2, dgvdxmxsn_tmp);
+   dgvdxmxsn_tmp *= -1.0;
+   AddMult(dgvdxmxsn_tmp, dxidxs_m, dgvdxmxsn);
+
+   for (int i =0; i<2; i++)
+   {
+      DenseMatrix Be_tmp(3,12);
+      Be_tmp.CopyRows(Be,i*3,(i+1)*3-1);
+      Be_tmp.Transpose(); // Be is now 12*3
+
+      Vector dxidxs_row(3); dxidxs_row = 0.0; dxidxs_m.GetRow(i,dxidxs_row);
+      DenseMatrix dgvdxmxsn_tmp2(3,3); dgvdxmxsn_tmp2 = 0.0;
+      outer(normal, dxidxs_row, dgvdxmxsn_tmp2);
+      AddMult_a(-1.0, Be_tmp, dgvdxmxsn_tmp2, dgvdxmxsn);
+
+   }
+
+   dg2dxmxs += dgvdxmxsn;
+
+   dg2dx.CopyMN(dg2dxs, 0, 0);
+   dg2dx.CopyMN(dg2dxm, 3, 3);
+   dg2dx.CopyMN(dg2dxsxm, 0, 3);
+   dg2dx.CopyMN(dg2dxmxs, 3, 0);
+
+};
+
+
+
+void NodeSegConPairs(const Vector x1, const Vector xi2,
+                     const DenseMatrix coords2,
+                     double& node_g, Vector& node_dg, DenseMatrix& node_dg2)
+{
+   double gap = 0.0;
+   Vector normal(3); normal = 0.0;
+   Vector dgdxm(12); dgdxm = 0.0;
+   Vector dgdxs(3); dgdxs = 0.0;
+
+   ComputeGapJacobian(x1, xi2, coords2, gap, normal, dgdxm, dgdxs);
+   node_g = gap;
+
+   node_dg.SetSize(12+3);
+   for (int i=0; i<3; i++) { node_dg[i] = dgdxs[i]; }
+   for (int i=0; i<12; i++) { node_dg[i+3] = dgdxm[i]; }
+
+   DenseMatrix dg2dx(15,15); dg2dx = 0.0;
+   DenseMatrix dgvdxmxsn(12,3); dgvdxmxsn = 0.0;
+   ComputeGapHessian(x1, xi2, coords2, dg2dx);
+
+   node_dg2.SetSize(15,15);
+   node_dg2 = dg2dx;
+
+   /*
+   if(obj.space1.conns{e1}(i)==150)  % for debugging purpose
+
+   v1 = 1:3;
+   v2 = 1:12;
+   %v1 = ones(1,3)
+   %v2 = ones(1,12)
+   v2 = reshape(v2,4,3);
+   x1n1 = x1 + 0.01*v1;
+   coords2n1 = coords2 + 0.001*v2;
+      [xi2n1, gapv1, ~, ~] = SlaveToMaster(obj, coords2n1, x1n1);
+      [gapn1, n1,dgdxmn1, dgdxsn1] = ComputeGapJacobian(obj, x1n1, xi2n1, coords2n1);
+   x1n2 = x1 - 0.01*v1;
+   coords2n2 = coords2 - 0.001*v2;
+      [xi2n2, gapv2, ~, ~] = SlaveToMaster(obj, coords2n2, x1n2);
+      [gapn2, n2,dgdxmn2, dgdxsn2] = ComputeGapJacobian(obj, x1n2, xi2n2, coords2n2);
+   fprintf('fd\n');
+      %gapv1-gapv2
+   [dgdxsn1(:)',dgdxmn1(:)'] - [dgdxsn2(:)',dgdxmn2(:)']
+
+   %dgdxsn1-dgdxsn2
+   fprintf('code\n');
+   v2n = v2';
+   %dg2dx(1:3,1:3)*0.04*ones(3,1)
+   temp = zeros(12,3);
+   for i = 1:4
+   temp1 = dg2dx(3+(i-1)*3+1:3+i*3,1:3);
+   temp((i-1)*3+1:i*3,:) = temp1';
+   end
+   temp2 = zeros(3,12);
+   for i = 1:4
+   temp3 = dg2dx(1:3,3+(i-1)*3+1:3+i*3);
+   temp2(:,(i-1)*3+1:i*3) = temp3';
+   end
+   %dg2dx
+   %dg2dx(4:end,1:3) = temp;
+   %dg2dx(1:3,4:end) = temp2;
+   %dgvdxm * 0.002*v2n(:)
+   (dg2dx*[0.02*v1(:)',0.002*v2n(:)']')'
+   %dg2dx(4:end,1:3)
+   end*/
+
+};
+
+
+// coordsm : (npoints*4, 3) use what class?
+// m_conn: (npoints*4)
+void Assemble_Contact(const int m, const int npoints, const int ndofs,
+                      const Vector x_s,
+                      const Vector xi, const DenseMatrix coordsm, const Array<int> s_conn,
+                      const Array<int> m_conn, Vector& g, SparseMatrix& M,
+                      std::vector<SparseMatrix>& dM)
+{
+   int ndim = 3;
+
+   g.SetSize(m);
+   g = 0.0;
+
+   //SparseMatrix M(m, n);  // M needs to be the correct size
+
+   //dM.resize(m); // needs to clear?
+
+   double g_tmp = 0.;
+   Vector dg(4*ndim+ndim);
+   dg = 0.;
+   DenseMatrix dg2(4*ndim+ndim,4*ndim+ndim);
+   dg2 = 0.;
+
+   for (int i=0; i<npoints; i++)
+   {
+      Vector x1(ndim);
+      x1[0] = x_s[i*ndim];
+      x1[1] = x_s[i*ndim+1];
+      x1[2] = x_s[i*ndim+2];
+
+      Vector xi2(ndim-1);
+      xi2[0] = xi[i*(ndim-1)];
+      xi2[1] = xi[i*(ndim-1)+1];
+
+      DenseMatrix coords2(4,3);
+      coords2.CopyRows(coordsm, i*4,(i+1)*4-1);
+
+      //how to get coords2?
+      dg = 0.0;
+      dg2 = 0.;
+      NodeSegConPairs(x1, xi2, coords2, g_tmp, dg, dg2);
+      //x1.Print();
+      //xi2.Print();
+      //coords2.Print();
+      g[s_conn[i]] = g_tmp; // should be unique
+      Array<int> m_conn_i(4);
+      m_conn.GetSubArray(4*i, 4, m_conn_i);
+
+      Array<int> node_conn(5);
+      node_conn[0] = s_conn[i];
+      for (int j=0; j<4; j++)
+      {
+         node_conn[j+1] = m_conn_i[j];
+      }
+
+      Array<int>  M_i_tmp(1);
+      M_i_tmp[0] = s_conn[i];
+
+      //j_idx = (node_conn-1)*obj.disp_field.num_components +repmat((1:obj.disp_field.num_components)', 1, length(node_conn{i}));
+      Array<int> j_idx(5*ndim); j_idx = 0;
+      for (int j=0; j< 5; j++)
+      {
+         for (int k=0; k<ndim; k++)
+         {
+            j_idx[j*ndim+k] = node_conn[j]*ndim+k;
+         }
+      }
+      DenseMatrix M_v_tmp(1, ndim*(4+1));   // SetData now?
+      M_v_tmp.SetRow(0, dg);
+
+      M.AddSubMatrix(M_i_tmp, j_idx, M_v_tmp);
+
+      Array<int> dM_i(ndim*(4+1));
+      Array<int> dM_j(ndim*(4+1));
+
+      for (int j=0; j< ndim*(4+1); j++)
+      {
+         dM_i[j] = j_idx[j];
+         dM_j[j] = j_idx[j];
+      }
+      //dg2.Print();
+      //dM[s_conn[i]].Print();
+      dM[s_conn[i]].AddSubMatrix(dM_i,dM_j, dg2);
+   }
+};
+
@@ -0,0 +1,119 @@
+#include "mfem.hpp"
+#include "Problems.hpp"
+#include "IPsolver.hpp"
+#include <fstream>
+#include <iostream>
+
+using namespace std;
+using namespace mfem;
+
+
+
+double dmanufacturedFun(const Vector &);
+double fRhs(const Vector &);
+double obstacle(const Vector &);
+
+int main(int argc, char *argv[])
+{
+  int FEorder = 1; // order of the finite elements
+  int linSolver = 0;
+  int maxIPMiters = 30;
+  bool iAmRoot = true;
+  
+  OptionsParser args(argc, argv);
+  args.AddOption(&FEorder, "-o", "--order",\
+		  "Order of the finite elements.");
+  args.AddOption(&linSolver, "-linSolver", "--linearSolver", \
+       "IP-Newton linear system solution strategy.");
+  args.AddOption(&maxIPMiters, "-IPMiters", "--IPMiters",\
+		  "Maximum number of IPM iterations");
+  
+  args.Parse();
+  if(!args.Good())
+  {
+    args.PrintUsage(cout);
+    return 1;
+  }
+  else
+  {
+    if( iAmRoot )
+    {
+      args.PrintOptions(cout);
+    }
+  }
+
+  const char *meshFile = "../../data/inline-quad.mesh";
+  Mesh *mesh = new Mesh(meshFile, 1, 1);
+  int dim = mesh->Dimension(); // geometric dimension of the domain
+  {
+     int ref_levels = 3;
+     for (int l = 0; l < ref_levels; l++)
+     {
+        mesh->UniformRefinement();
+     }
+  }
+
+  FiniteElementCollection *fec = new H1_FECollection(FEorder, dim);
+  FiniteElementSpace      *Vh  = new FiniteElementSpace(mesh, fec);
+  ObstacleProblem problem(Vh, &fRhs, &obstacle);
+  
+  int dimD = problem.GetDimD();
+  Vector x0(dimD); x0 = 0.0;
+  Vector xf(dimD); xf = 0.0;
+
+  InteriorPointSolver optimizer(&problem); 
+  optimizer.SetTol(1.e-7);
+  optimizer.SetLinearSolver(linSolver);
+  optimizer.SetMaxIter(maxIPMiters);
+  optimizer.Mult(x0, xf);
+
+  GridFunction d_gf(Vh);
+
+  d_gf = xf;
+
+  FunctionCoefficient dm_fc(dmanufacturedFun); // pseudo-manufactured solution
+  GridFunction dm_gf(Vh);
+  dm_gf.ProjectCoefficient(dm_fc);
+  
+  ParaViewDataCollection paraview_dc("BarrierProblemSolution", mesh);
+  paraview_dc.SetPrefixPath("ParaView");
+  paraview_dc.SetLevelsOfDetail(FEorder);
+  paraview_dc.SetDataFormat(VTKFormat::BINARY);
+  paraview_dc.SetHighOrderOutput(true);
+  paraview_dc.SetCycle(0);
+  paraview_dc.SetTime(0.0);
+  paraview_dc.RegisterField("d(x) (numerical)", &d_gf);
+  paraview_dc.RegisterField("d(x) (pseudo-manufactured)", &dm_gf);
+  paraview_dc.Save();
+  
+  
+  delete Vh;
+  delete fec;
+  delete mesh;
+  return 0;
+}
+
+
+double dmanufacturedFun(const Vector &x)
+{
+  return cos(2*M_PI*x(0)) + 0.2 - 2.0*(pow(x(0),3) - 1.5*pow(x(0),2));
+}
+
+
+// f(x) forcing term... which enters the objective energy functional
+// E(d) = 0.5 d^T K d - f^T d, where f is a discrete vector representation
+// of f(x). f(x) is such that in the absence of bound-constraints then
+// the solution of the optimization problem satisfies the PDE
+// -div(grad(d)) + d = f + homogeneous Neumann conditions on the unit interval,
+// for d(x) = cos(2 \pi x) + a0 + a3 (x^3 - 1.5 x^2), a2 = 0.2, a3 = -2
+double fRhs(const Vector &x)
+{
+  double fx = 0.;
+  fx = 0.2 - 2.0 * (pow(x(0),3)- 1.5*pow(x(0),2.) - 6 * x(0) + 3.) + (1. + pow(2.*M_PI,2))*cos(2.*M_PI*x(0));
+  return fx;
+}
+
+double obstacle(const Vector &x)
+{
+  return 0.0;
+}
@@ -0,0 +1,70 @@
+
+MFEM mesh v1.0
+
+#
+# MFEM Geometry Types (see mesh/geom.hpp):
+#
+# POINT       = 0
+# SEGMENT     = 1
+# TRIANGLE    = 2
+# SQUARE      = 3
+# TETRAHEDRON = 4
+# CUBE        = 5
+# PRISM       = 6
+#
+
+dimension
+3
+
+# 1 nothing
+elements
+4
+1   5     0     1     3     2     6     7    9     8
+1   5     2     3     5     4     8     9    11    10
+1   5     6     7     9     8    12    13    15    14
+1   5     8     9    11    10    14    15    17    16
+
+# 0 nothing
+# 1 dirichlet bc
+# 2 contact
+boundary
+16
+1   3     1     0     2     3
+1   3     3     2     4     5
+1   3    12    13    15    14
+1   3    14    15    17    16
+3   3     2     0     6     8
+3   3     4     2     8    10
+3   3     8     6    12    14
+3   3    10     8    14    16
+2   3     1     3     9     7
+2   3     3     5    11    9
+2   3     7    9    15    13
+2   3     9    11    17    15
+1   3     6     0     1     7
+1   3    12     6     7    13
+1   3     4    10    11     5
+1   3    10    16    17    11
+
+vertices
+18
+3
+
+0.000000000000 0.145770950245 0.443895630208
+0.507100000000 0.145770950245 0.443895630208
+0.000000000000 0.350937660019 0.294833290227
+0.507100000000 0.350937660019 0.294833290227
+0.000000000000 0.556104369792 0.145770950245
+0.507100000000 0.556104369792 0.145770950245
+0.000000000000 0.294833290227 0.649062339981
+0.507100000000 0.294833290227 0.649062339981
+0.000000000000 0.500000000000 0.500000000000
+0.507100000000 0.500000000000 0.500000000000
+0.000000000000 0.705166709773 0.350937660019
+0.507100000000 0.705166709773 0.350937660019
+0.000000000000 0.443895630208 0.854229049755
+0.507100000000 0.443895630208 0.854229049755
+0.000000000000 0.649062339981 0.705166709773
+0.507100000000 0.649062339981 0.705166709773
+0.000000000000 0.854229049755 0.556104369792
+0.507100000000 0.854229049755 0.556104369792
@@ -22,7 +22,7 @@ using namespace mfem;
 int main(int argc, char *argv[])
 {
   // 1. Parse command line options.
-   string mesh_file = "../data/star.mesh";
+   const char *mesh_file = "../data/star.mesh";
   int order = 1;

   OptionsParser args(argc, argv);
@@ -26,7 +26,7 @@ int main(int argc, char *argv[])
   Hypre::Init();

   // 2. Parse command line options.
-   string mesh_file = "../data/star.mesh";
+   const char *mesh_file = "../data/star.mesh";
   int order = 1;

   OptionsParser args(argc, argv);
@@ -5,7 +5,6 @@
 // Sample runs:  mpirun -np 4 ex13p -m ../data/star.mesh
 //               mpirun -np 4 ex13p -m ../data/square-disc.mesh -o 2 -n 4
 //               mpirun -np 4 ex13p -m ../data/beam-tet.mesh
-//               mpirun -np 4 ex13p -m ../data/beam-tet.mesh -nc -o 2 -rs 1
 //               mpirun -np 4 ex13p -m ../data/beam-hex.mesh
 //               mpirun -np 4 ex13p -m ../data/escher.mesh
 //               mpirun -np 4 ex13p -m ../data/fichera.mesh
@@ -55,7 +54,6 @@ int main(int argc, char *argv[])
   int par_ref_levels = 1;
   int order = 1;
   int nev = 5;
-   bool nc = false;
   bool visualization = 1;
   const char *device_config = "cpu";

@@ -71,9 +69,6 @@ int main(int argc, char *argv[])
                  " isoparametric space.");
   args.AddOption(&nev, "-n", "--num-eigs",
                  "Number of desired eigenmodes.");
-   args.AddOption(&nc, "-nc", "--non-conforming", "-c",
-                  "--conforming",
-                  "Mark the mesh as nonconforming before partitioning.");
   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
                  "--no-visualization",
                  "Enable or disable GLVis visualization.");
@@ -103,10 +98,6 @@ int main(int argc, char *argv[])
   //    and volume meshes with the same code.
   Mesh *mesh = new Mesh(mesh_file, 1, 1);
   int dim = mesh->Dimension();
-   if (nc)
-   {
-      mesh->EnsureNCMesh(true);
-   }

   // 5. Refine the serial mesh on all processors to increase the resolution. In
   //    this example we do 'ref_levels' of uniform refinement (2 by default, or
@@ -13,7 +13,6 @@
 //               mpirun -np 4 ex15p -m ../data/square-disc-nurbs.mesh
 //               mpirun -np 4 ex15p -m ../data/disc-nurbs.mesh
 //               mpirun -np 4 ex15p -m ../data/fichera.mesh -tf 0.5
-//               mpirun -np 4 ex15p -m ../data/fichera-mixed.mesh -tf 0.5
 //               mpirun -np 4 ex15p -m ../data/ball-nurbs.mesh -tf 0.5
 //               mpirun -np 4 ex15p -m ../data/mobius-strip.mesh
 //               mpirun -np 4 ex15p -m ../data/amr-quad.mesh
@@ -32,7 +32,6 @@
 //               We recommend viewing Example 22 before viewing this example.

 #include "mfem.hpp"
-#include <memory>
 #include <fstream>
 #include <iostream>

@@ -45,7 +44,7 @@ using namespace std;
 using namespace mfem;

 // Class for setting up a simple Cartesian PML region
-class PML
+class CartesianPML
 {
 private:
   Mesh *mesh;
@@ -70,7 +69,7 @@ private:

 public:
   // Constructor
-   PML(Mesh *mesh_,Array2D<double> length_);
+   CartesianPML(Mesh *mesh_,Array2D<double> length_);

   // Return Computational Domain Boundary
   Array2D<double> GetCompDomainBdr() {return comp_dom_bdr;}
@@ -92,12 +91,12 @@ public:
 class PMLDiagMatrixCoefficient : public VectorCoefficient
 {
 private:
-   PML * pml = nullptr;
-   void (*Function)(const Vector &, PML *, Vector &);
+   CartesianPML * pml = nullptr;
+   void (*Function)(const Vector &, CartesianPML *, Vector &);
 public:
-   PMLDiagMatrixCoefficient(int dim, void(*F)(const Vector &, PML *,
+   PMLDiagMatrixCoefficient(int dim, void(*F)(const Vector &, CartesianPML *,
                                              Vector &),
-                            PML * pml_)
+                            CartesianPML * pml_)
      : VectorCoefficient(dim), pml(pml_), Function(F)
   {}

@@ -126,13 +125,13 @@ void source(const Vector &x, Vector & f);

 // Functions for computing the necessary coefficients after PML stretching.
 // J is the Jacobian matrix of the stretching function
-void detJ_JT_J_inv_Re(const Vector &x, PML * pml, Vector &D);
-void detJ_JT_J_inv_Im(const Vector &x, PML * pml, Vector &D);
-void detJ_JT_J_inv_abs(const Vector &x, PML * pml, Vector &D);
+void detJ_JT_J_inv_Re(const Vector &x, CartesianPML * pml, Vector &D);
+void detJ_JT_J_inv_Im(const Vector &x, CartesianPML * pml, Vector &D);
+void detJ_JT_J_inv_abs(const Vector &x, CartesianPML * pml, Vector &D);

-void detJ_inv_JT_J_Re(const Vector &x, PML * pml, Vector &D);
-void detJ_inv_JT_J_Im(const Vector &x, PML * pml, Vector &D);
-void detJ_inv_JT_J_abs(const Vector &x, PML * pml, Vector &D);
+void detJ_inv_JT_J_Re(const Vector &x, CartesianPML * pml, Vector &D);
+void detJ_inv_JT_J_Im(const Vector &x, CartesianPML * pml, Vector &D);
+void detJ_inv_JT_J_abs(const Vector &x, CartesianPML * pml, Vector &D);

 Array2D<double> comp_domain_bdr;
 Array2D<double> domain_bdr;
@@ -268,7 +267,7 @@ int main(int argc, char *argv[])
         length = 0.25;
         break;
   }
-   PML * pml = new PML(mesh,length);
+   CartesianPML * pml = new CartesianPML(mesh,length);
   comp_domain_bdr = pml->GetCompDomainBdr();
   domain_bdr = pml->GetDomainBdr();

@@ -468,14 +467,16 @@ int main(int argc, char *argv[])
      offsets[2] = fespace->GetTrueVSize();
      offsets.PartialSum();

-      std::unique_ptr<Operator> pc_r;
-      std::unique_ptr<Operator> pc_i;
+      Operator *pc_r = nullptr;
+      Operator *pc_i = nullptr;
      double s = (conv == ComplexOperator::HERMITIAN) ? -1.0 : 1.0;
      if (pa)
      {
         // Jacobi Smoother
-         pc_r.reset(new OperatorJacobiSmoother(prec, ess_tdof_list));
-         pc_i.reset(new ScaledOperator(pc_r.get(), s));
+         OperatorJacobiSmoother *d00 = new OperatorJacobiSmoother(prec, ess_tdof_list);
+         ScaledOperator *d11 = new ScaledOperator(d00, s);
+         pc_r = d00;
+         pc_i = d11;
      }
      else
      {
@@ -484,13 +485,15 @@ int main(int argc, char *argv[])
         prec.FormSystemMatrix(ess_tdof_list, PCOpAh);

         // Gauss-Seidel Smoother
-         pc_r.reset(new GSSmoother(*PCOpAh.As<SparseMatrix>()));
-         pc_i.reset(new ScaledOperator(pc_r.get(), s));
+         GSSmoother *gs00 = new GSSmoother(*PCOpAh.As<SparseMatrix>());
+         ScaledOperator *gs11 = new ScaledOperator(gs00, s);
+         pc_r = gs00;
+         pc_i = gs11;
      }

      BlockDiagonalPreconditioner BlockDP(offsets);
-      BlockDP.SetDiagonalBlock(0, pc_r.get());
-      BlockDP.SetDiagonalBlock(1, pc_i.get());
+      BlockDP.SetDiagonalBlock(0, pc_r);
+      BlockDP.SetDiagonalBlock(1, pc_i);

      GMRESSolver gmres;
      gmres.SetPrintLevel(1);
@@ -804,7 +807,7 @@ void E_bdr_data_Im(const Vector &x, Vector &E)
   }
 }

-void detJ_JT_J_inv_Re(const Vector &x, PML * pml, Vector &D)
+void detJ_JT_J_inv_Re(const Vector &x, CartesianPML * pml, Vector &D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det(1.0, 0.0);
@@ -821,7 +824,7 @@ void detJ_JT_J_inv_Re(const Vector &x, PML * pml, Vector &D)
   }
 }

-void detJ_JT_J_inv_Im(const Vector &x, PML * pml, Vector &D)
+void detJ_JT_J_inv_Im(const Vector &x, CartesianPML * pml, Vector &D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -838,7 +841,7 @@ void detJ_JT_J_inv_Im(const Vector &x, PML * pml, Vector &D)
   }
 }

-void detJ_JT_J_inv_abs(const Vector &x, PML * pml, Vector &D)
+void detJ_JT_J_inv_abs(const Vector &x, CartesianPML * pml, Vector &D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -855,7 +858,7 @@ void detJ_JT_J_inv_abs(const Vector &x, PML * pml, Vector &D)
   }
 }

-void detJ_inv_JT_J_Re(const Vector &x, PML * pml, Vector &D)
+void detJ_inv_JT_J_Re(const Vector &x, CartesianPML * pml, Vector &D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det(1.0, 0.0);
@@ -880,7 +883,7 @@ void detJ_inv_JT_J_Re(const Vector &x, PML * pml, Vector &D)
   }
 }

-void detJ_inv_JT_J_Im(const Vector &x, PML * pml, Vector &D)
+void detJ_inv_JT_J_Im(const Vector &x, CartesianPML * pml, Vector &D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -904,7 +907,7 @@ void detJ_inv_JT_J_Im(const Vector &x, PML * pml, Vector &D)
   }
 }

-void detJ_inv_JT_J_abs(const Vector &x, PML * pml, Vector &D)
+void detJ_inv_JT_J_abs(const Vector &x, CartesianPML * pml, Vector &D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -928,14 +931,14 @@ void detJ_inv_JT_J_abs(const Vector &x, PML * pml, Vector &D)
   }
 }

-PML::PML(Mesh *mesh_, Array2D<double> length_)
+CartesianPML::CartesianPML(Mesh *mesh_, Array2D<double> length_)
   : mesh(mesh_), length(length_)
 {
   dim = mesh->Dimension();
   SetBoundaries();
 }

-void PML::SetBoundaries()
+void CartesianPML::SetBoundaries()
 {
   comp_dom_bdr.SetSize(dim, 2);
   dom_bdr.SetSize(dim, 2);
@@ -950,7 +953,7 @@ void PML::SetBoundaries()
   }
 }

-void PML::SetAttributes(Mesh *mesh_)
+void CartesianPML::SetAttributes(Mesh *mesh_)
 {
   // Initialize bdr attributes
   for (int i = 0; i < mesh_->GetNBE(); ++i)
@@ -999,8 +1002,8 @@ void PML::SetAttributes(Mesh *mesh_)
   mesh_->SetAttributes();
 }

-void PML::StretchFunction(const Vector &x,
-                          vector<complex<double>> &dxs)
+void CartesianPML::StretchFunction(const Vector &x,
+                                   vector<complex<double>> &dxs)
 {
   complex<double> zi = complex<double>(0., 1.);

@@ -44,7 +44,7 @@ using namespace std;
 using namespace mfem;

 // Class for setting up a simple Cartesian PML region
-class PML
+class CartesianPML
 {
 private:
   Mesh *mesh;
@@ -69,7 +69,7 @@ private:

 public:
   // Constructor
-   PML(Mesh *mesh_,Array2D<double> length_);
+   CartesianPML(Mesh *mesh_,Array2D<double> length_);

   // Return Computational Domain Boundary
   Array2D<double> GetCompDomainBdr() {return comp_dom_bdr;}
@@ -91,12 +91,12 @@ public:
 class PMLDiagMatrixCoefficient : public VectorCoefficient
 {
 private:
-   PML * pml = nullptr;
-   void (*Function)(const Vector &, PML *, Vector &);
+   CartesianPML * pml = nullptr;
+   void (*Function)(const Vector &, CartesianPML *, Vector &);
 public:
-   PMLDiagMatrixCoefficient(int dim, void(*F)(const Vector &, PML *,
+   PMLDiagMatrixCoefficient(int dim, void(*F)(const Vector &, CartesianPML *,
                                              Vector &),
-                            PML * pml_)
+                            CartesianPML * pml_)
      : VectorCoefficient(dim), pml(pml_), Function(F)
   {}

@@ -125,13 +125,13 @@ void source(const Vector &x, Vector & f);

 // Functions for computing the necessary coefficients after PML stretching.
 // J is the Jacobian matrix of the stretching function
-void detJ_JT_J_inv_Re(const Vector &x, PML * pml, Vector & D);
-void detJ_JT_J_inv_Im(const Vector &x, PML * pml, Vector & D);
-void detJ_JT_J_inv_abs(const Vector &x, PML * pml, Vector & D);
+void detJ_JT_J_inv_Re(const Vector &x, CartesianPML * pml, Vector & D);
+void detJ_JT_J_inv_Im(const Vector &x, CartesianPML * pml, Vector & D);
+void detJ_JT_J_inv_abs(const Vector &x, CartesianPML * pml, Vector & D);

-void detJ_inv_JT_J_Re(const Vector &x, PML * pml, Vector & D);
-void detJ_inv_JT_J_Im(const Vector &x, PML * pml, Vector & D);
-void detJ_inv_JT_J_abs(const Vector &x, PML * pml, Vector & D);
+void detJ_inv_JT_J_Re(const Vector &x, CartesianPML * pml, Vector & D);
+void detJ_inv_JT_J_Im(const Vector &x, CartesianPML * pml, Vector & D);
+void detJ_inv_JT_J_abs(const Vector &x, CartesianPML * pml, Vector & D);

 Array2D<double> comp_domain_bdr;
 Array2D<double> domain_bdr;
@@ -295,7 +295,7 @@ int main(int argc, char *argv[])
         length = 0.25;
         break;
   }
-   PML * pml = new PML(mesh,length);
+   CartesianPML * pml = new CartesianPML(mesh,length);
   comp_domain_bdr = pml->GetCompDomainBdr();
   domain_bdr = pml->GetDomainBdr();

@@ -478,11 +478,11 @@ int main(int argc, char *argv[])
   if (!pa && mumps_solver)
   {
      HypreParMatrix *A = Ah.As<ComplexHypreParMatrix>()->GetSystemMatrix();
-      MUMPSSolver mumps(A->GetComm());
+      MUMPSSolver mumps;
      mumps.SetPrintLevel(0);
      mumps.SetMatrixSymType(MUMPSSolver::MatType::UNSYMMETRIC);
      mumps.SetOperator(*A);
-      mumps.Mult(B, X);
+      mumps.Mult(B,X);
      delete A;
   }
 #endif
@@ -524,14 +524,16 @@ int main(int argc, char *argv[])
      offsets[2] = fespace->GetTrueVSize();
      offsets.PartialSum();

-      std::unique_ptr<Operator> pc_r;
-      std::unique_ptr<Operator> pc_i;
+      Operator *pc_r = nullptr;
+      Operator *pc_i = nullptr;
      int s = (conv == ComplexOperator::HERMITIAN) ? -1.0 : 1.0;
      if (pa)
      {
         // Jacobi Smoother
-         pc_r.reset(new OperatorJacobiSmoother(prec, ess_tdof_list));
-         pc_i.reset(new ScaledOperator(pc_r.get(), s));
+         OperatorJacobiSmoother *d00 = new OperatorJacobiSmoother(prec, ess_tdof_list);
+         ScaledOperator *d11 = new ScaledOperator(d00, s);
+         pc_r = d00;
+         pc_i = d11;
      }
      else
      {
@@ -539,13 +541,15 @@ int main(int argc, char *argv[])
         prec.FormSystemMatrix(ess_tdof_list, PCOpAh);

         // Hypre AMS
-         pc_r.reset(new HypreAMS(*PCOpAh.As<HypreParMatrix>(), fespace));
-         pc_i.reset(new ScaledOperator(pc_r.get(), s));
+         HypreAMS *ams00 = new HypreAMS(*PCOpAh.As<HypreParMatrix>(), fespace);
+         ScaledOperator *ams11 = new ScaledOperator(ams00, s);
+         pc_r = ams00;
+         pc_i = ams11;
      }

      BlockDiagonalPreconditioner BlockDP(offsets);
-      BlockDP.SetDiagonalBlock(0, pc_r.get());
-      BlockDP.SetDiagonalBlock(1, pc_i.get());
+      BlockDP.SetDiagonalBlock(0, pc_r);
+      BlockDP.SetDiagonalBlock(1, pc_i);

      GMRESSolver gmres(MPI_COMM_WORLD);
      gmres.SetPrintLevel(1);
@@ -880,7 +884,7 @@ void E_bdr_data_Im(const Vector &x, Vector &E)
   }
 }

-void detJ_JT_J_inv_Re(const Vector &x, PML * pml, Vector & D)
+void detJ_JT_J_inv_Re(const Vector &x, CartesianPML * pml, Vector & D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det(1.0, 0.0);
@@ -897,7 +901,7 @@ void detJ_JT_J_inv_Re(const Vector &x, PML * pml, Vector & D)
   }
 }

-void detJ_JT_J_inv_Im(const Vector &x, PML * pml, Vector & D)
+void detJ_JT_J_inv_Im(const Vector &x, CartesianPML * pml, Vector & D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -914,7 +918,7 @@ void detJ_JT_J_inv_Im(const Vector &x, PML * pml, Vector & D)
   }
 }

-void detJ_JT_J_inv_abs(const Vector &x, PML * pml, Vector & D)
+void detJ_JT_J_inv_abs(const Vector &x, CartesianPML * pml, Vector & D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -931,7 +935,7 @@ void detJ_JT_J_inv_abs(const Vector &x, PML * pml, Vector & D)
   }
 }

-void detJ_inv_JT_J_Re(const Vector &x, PML * pml, Vector & D)
+void detJ_inv_JT_J_Re(const Vector &x, CartesianPML * pml, Vector & D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det(1.0, 0.0);
@@ -956,7 +960,7 @@ void detJ_inv_JT_J_Re(const Vector &x, PML * pml, Vector & D)
   }
 }

-void detJ_inv_JT_J_Im(const Vector &x, PML * pml, Vector & D)
+void detJ_inv_JT_J_Im(const Vector &x, CartesianPML * pml, Vector & D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -980,7 +984,7 @@ void detJ_inv_JT_J_Im(const Vector &x, PML * pml, Vector & D)
   }
 }

-void detJ_inv_JT_J_abs(const Vector &x, PML * pml, Vector & D)
+void detJ_inv_JT_J_abs(const Vector &x, CartesianPML * pml, Vector & D)
 {
   vector<complex<double>> dxs(dim);
   complex<double> det = 1.0;
@@ -1004,14 +1008,14 @@ void detJ_inv_JT_J_abs(const Vector &x, PML * pml, Vector & D)
   }
 }

-PML::PML(Mesh *mesh_, Array2D<double> length_)
+CartesianPML::CartesianPML(Mesh *mesh_, Array2D<double> length_)
   : mesh(mesh_), length(length_)
 {
   dim = mesh->Dimension();
   SetBoundaries();
 }

-void PML::SetBoundaries()
+void CartesianPML::SetBoundaries()
 {
   comp_dom_bdr.SetSize(dim, 2);
   dom_bdr.SetSize(dim, 2);
@@ -1026,7 +1030,7 @@ void PML::SetBoundaries()
   }
 }

-void PML::SetAttributes(ParMesh *pmesh)
+void CartesianPML::SetAttributes(ParMesh *pmesh)
 {
   // Initialize bdr attributes
   for (int i = 0; i < pmesh->GetNBE(); ++i)
@@ -1076,8 +1080,8 @@ void PML::SetAttributes(ParMesh *pmesh)
   pmesh->SetAttributes();
 }

-void PML::StretchFunction(const Vector &x,
-                          vector<complex<double>> &dxs)
+void CartesianPML::StretchFunction(const Vector &x,
+                                   vector<complex<double>> &dxs)
 {
   complex<double> zi = complex<double>(0., 1.);

@@ -63,7 +63,6 @@ int main(int argc, char *argv[])
   int order = 1;
   bool static_cond = false;
   bool pa = false;
-   bool nc = false;
   const char *device_config = "cpu";
   bool visualization = 1;

@@ -78,9 +77,6 @@ int main(int argc, char *argv[])
                  "--no-static-condensation", "Enable static condensation.");
   args.AddOption(&pa, "-pa", "--partial-assembly", "-no-pa",
                  "--no-partial-assembly", "Enable Partial Assembly.");
-   args.AddOption(&nc, "-nc", "--non-conforming", "-c",
-                  "--conforming",
-                  "Mark the mesh as nonconforming before partitioning.");
   args.AddOption(&device_config, "-d", "--device",
                  "Device configuration string, see Device::Configure().");
   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
@@ -106,11 +102,6 @@ int main(int argc, char *argv[])
   Mesh *mesh = new Mesh(mesh_file, 1, 1);
   dim = mesh->Dimension();
   int sdim = mesh->SpaceDimension();
-   if (nc)
-   {
-      // Can set to false to use conformal refinement for simplices.
-      mesh->EnsureNCMesh(true);
-   }

   // 4. Refine the mesh to increase the resolution. In this example we do
   //    'ref_levels' of uniform refinement. We choose 'ref_levels' to be the
@@ -1,621 +0,0 @@
-//                                MFEM Example 34
-//
-// Compile with: make ex34
-//
-// Sample runs:  ex34 -o 2
-//               ex34 -o 2 -pa -hex
-//
-// Device sample runs:
-//               ex34 -o 2 -pa -hex -d cuda
-//               ex34 -o 2 -no-pa -d cuda
-//
-// Description:  This example code solves a simple magnetostatic problem
-//               curl curl A = J where the current density J is computed on a
-//               subset of the domain as J = -sigma grad phi. We discretize the
-//               vector potential with Nedelec finite elements, the scalar
-//               potential with Lagrange finite elements, and the current
-//               density with Raviart-Thomas finite elements.
-//
-//               The example demonstrates the use of a SubMesh to compute the
-//               scalar potential and its associated current density which is
-//               then transferred to the original mesh and used as a source
-//               function.
-//
-//               Note that this example takes certain liberties with the
-//               current density which is not necessarily divergence free
-//               as it should be. This was done to focus on the use of the
-//               SubMesh to transfer information between a full mesh and a
-//               sub-domain. A more rigorous implementation might employ an
-//               H(div) saddle point solver to obtain a divergence free J on
-//               the SubMesh. It would then also need to ensure that the r.h.s.
-//               of curl curl A = J does in fact lie in the range of the weak
-//               curl operator by performing a divergence cleaning procedure
-//               before the solve. After divergence cleaning the delta
-//               parameter would probably not be needed.
-//
-//               This example is designed to make use of a specific mesh which
-//               has a known configuration of elements and boundary attributes.
-//               Other meshes could be used but extra care would be required to
-//               properly define the SubMesh and the necessary boundaries.
-//
-//               We recommend viewing examples 1 and 3 before viewing this
-//               example.
-
-#include "mfem.hpp"
-#include <fstream>
-#include <iostream>
-
-using namespace std;
-using namespace mfem;
-
-static bool pa_ = false;
-static bool algebraic_ceed_ = false;
-
-void ComputeCurrentDensityOnSubMesh(int order,
-                                    const Array<int> &phi0_attr,
-                                    const Array<int> &phi1_attr,
-                                    const Array<int> &jn_zero_attr,
-                                    GridFunction &j_cond);
-
-int main(int argc, char *argv[])
-{
-   // 1. Parse command-line options.
-   const char *mesh_file = "../data/fichera-mixed.mesh";
-   Array<int> cond_attr;
-   Array<int> submesh_elems;
-   Array<int> sym_plane_attr;
-   Array<int> phi0_attr;
-   Array<int> phi1_attr;
-   Array<int> jn_zero_attr;
-   int ref_levels = 1;
-   int order = 1;
-   double delta_const = 1e-6;
-   bool mixed = true;
-   bool static_cond = false;
-   const char *device_config = "cpu";
-   bool visualization = true;
-
-   OptionsParser args(argc, argv);
-   args.AddOption(&ref_levels, "-r", "--refine",
-                  "Number of times to refine the mesh uniformly.");
-   args.AddOption(&order, "-o", "--order",
-                  "Finite element order (polynomial degree).");
-   args.AddOption(&delta_const, "-mc", "--magnetic-cond",
-                  "Magnetic Conductivity");
-   args.AddOption(&static_cond, "-sc", "--static-condensation", "-no-sc",
-                  "--no-static-condensation", "Enable static condensation.");
-   args.AddOption(&mixed, "-mixed", "--mixed-mesh", "-hex",
-                  "--hex-mesh", "Mixed mesh of hexahedral mesh.");
-   args.AddOption(&pa_, "-pa", "--partial-assembly", "-no-pa",
-                  "--no-partial-assembly", "Enable Partial Assembly.");
-   args.AddOption(&device_config, "-d", "--device",
-                  "Device configuration string, see Device::Configure().");
-#ifdef MFEM_USE_CEED
-   args.AddOption(&algebraic_ceed_, "-a", "--algebraic", "-no-a", "--no-algebraic",
-                  "Use algebraic Ceed solver");
-#endif
-   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
-                  "--no-visualization",
-                  "Enable or disable GLVis visualization.");
-
-   args.Parse();
-   if (!args.Good())
-   {
-      args.PrintUsage(cout);
-      return 1;
-   }
-   args.PrintOptions(cout);
-
-   if (!mixed || pa_)
-   {
-      mesh_file = "../data/fichera.mesh";
-   }
-
-   if (submesh_elems.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0)
-      {
-         submesh_elems.SetSize(5);
-         submesh_elems[0] = 0;
-         submesh_elems[1] = 2;
-         submesh_elems[2] = 3;
-         submesh_elems[3] = 4;
-         submesh_elems[4] = 9;
-      }
-      else if (strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         submesh_elems.SetSize(7);
-         submesh_elems[0] = 10;
-         submesh_elems[1] = 14;
-         submesh_elems[2] = 34;
-         submesh_elems[3] = 36;
-         submesh_elems[4] = 37;
-         submesh_elems[5] = 38;
-         submesh_elems[6] = 39;
-      }
-   }
-   if (sym_plane_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         sym_plane_attr.SetSize(8);
-         sym_plane_attr[0] =  9;
-         sym_plane_attr[1] = 10;
-         sym_plane_attr[2] = 11;
-         sym_plane_attr[3] = 12;
-         sym_plane_attr[4] = 13;
-         sym_plane_attr[5] = 14;
-         sym_plane_attr[6] = 15;
-         sym_plane_attr[7] = 16;
-      }
-   }
-   if (phi0_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         phi0_attr.Append(2);
-      }
-   }
-   if (phi1_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         phi1_attr.Append(23);
-      }
-   }
-   if (jn_zero_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         jn_zero_attr.Append(25);
-      }
-      for (int i=0; i<sym_plane_attr.Size(); i++)
-      {
-         jn_zero_attr.Append(sym_plane_attr[i]);
-      }
-   }
-
-   // 2. Enable hardware devices such as GPUs, and programming models such as
-   //    CUDA, OCCA, RAJA and OpenMP based on command line options.
-   Device device(device_config);
-   device.Print();
-
-   // 3. Read the (serial) mesh from the given mesh file on all processors.  We
-   //    can handle triangular, quadrilateral, tetrahedral, hexahedral, surface
-   //    and volume meshes with the same code.
-   Mesh mesh(mesh_file, 1, 1);
-   int dim = mesh.Dimension();
-
-   if (!mixed || pa_)
-   {
-      mesh.UniformRefinement();
-
-      if (ref_levels > 0)
-      {
-         ref_levels--;
-      }
-   }
-
-   int submesh_attr = -1;
-   if (cond_attr.Size() == 0 && submesh_elems.Size() > 0)
-   {
-      int max_attr = mesh.attributes.Max();
-      submesh_attr = max_attr + 1;
-
-      for (int i=0; i<submesh_elems.Size(); i++)
-      {
-         mesh.SetAttribute(submesh_elems[i], submesh_attr);
-      }
-      mesh.SetAttributes();
-
-      if (cond_attr.Size() == 0)
-      {
-         cond_attr.Append(submesh_attr);
-      }
-   }
-
-   // 4. Refine the serial mesh on all processors to increase the resolution. In
-   //    this example we do 'ref_levels' of uniform refinement.
-   {
-      for (int l = 0; l < ref_levels; l++)
-      {
-         mesh.UniformRefinement();
-      }
-   }
-
-   // 5b. Extract a submesh covering a portion of the domain
-   SubMesh mesh_cond(SubMesh::CreateFromDomain(mesh, cond_attr));
-
-   // 6. Define a suitable finite element space on the SubMesh and compute
-   //    the current density as an H(div) field.
-   RT_FECollection fec_cond_rt(order - 1, dim);
-   FiniteElementSpace fes_cond_rt(&mesh_cond, &fec_cond_rt);
-   GridFunction j_cond(&fes_cond_rt);
-
-   ComputeCurrentDensityOnSubMesh(order, phi0_attr, phi1_attr, jn_zero_attr,
-                                  j_cond);
-
-   // 6a. Save the SubMesh and associated current density in parallel. This
-   //     output can be viewed later using GLVis:
-   //        "glvis -np <np> -m cond_mesh -g cond_j"
-   {
-      ostringstream mesh_name, cond_name;
-      mesh_name << "cond.mesh";
-      cond_name << "cond_j.gf";
-
-      ofstream mesh_ofs(mesh_name.str().c_str());
-      mesh_ofs.precision(8);
-      mesh_cond.Print(mesh_ofs);
-
-      ofstream cond_ofs(cond_name.str().c_str());
-      cond_ofs.precision(8);
-      j_cond.Save(cond_ofs);
-   }
-   // 6b. Send the current density, computed on the SubMesh, to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream port_sock(vishost, visport);
-      port_sock.precision(8);
-      port_sock << "solution\n" << mesh_cond << j_cond
-                << "window_title 'Conductor J'"
-                << "window_geometry 400 0 400 350" << flush;
-   }
-
-   // 7. Define a parallel finite element space on the full mesh. Here we use
-   //    the H(curl) finite elements for the vector potential and H(div) for the
-   //    current density.
-   ND_FECollection fec_nd(order, dim);
-   RT_FECollection fec_rt(order - 1, dim);
-   FiniteElementSpace fespace_nd(&mesh, &fec_nd);
-   FiniteElementSpace fespace_rt(&mesh, &fec_rt);
-
-   GridFunction j_full(&fespace_rt);
-   j_full = 0.0;
-   mesh_cond.Transfer(j_cond, j_full);
-
-   // 7a. Send the transferred current density to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock(vishost, visport);
-      sol_sock.precision(8);
-      sol_sock << "solution\n" << mesh << j_full
-               << "window_title 'J Full'"
-               << "window_geometry 400 430 400 350" << flush;
-   }
-
-   // 8. Determine the list of true (i.e. parallel conforming) essential
-   //    boundary dofs. In this example, the boundary conditions are defined by
-   //    marking all the boundary attributes except for those on a symmetry
-   //    plane as essential (Dirichlet) and converting them to a list of true
-   //    dofs.
-   Array<int> ess_tdof_list;
-   Array<int> ess_bdr;
-   if (mesh.bdr_attributes.Size())
-   {
-      ess_bdr.SetSize(mesh.bdr_attributes.Max());
-      ess_bdr = 1;
-      for (int i=0; i<sym_plane_attr.Size(); i++)
-      {
-         ess_bdr[sym_plane_attr[i]-1] = 0;
-      }
-      fespace_nd.GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
-   }
-
-   //  9. Set up the parallel linear form b(.) which corresponds to the
-   //     right-hand side of the FEM linear system, which in this case is
-   //     (J,W_i) where J is given by the function H(div) field transferred
-   //     from the SubMesh and W_i are the basis functions in the finite
-   //     element fespace.
-   VectorGridFunctionCoefficient jCoef(&j_full);
-   LinearForm b(&fespace_nd);
-   b.AddDomainIntegrator(new VectorFEDomainLFIntegrator(jCoef));
-   b.Assemble();
-
-   // 10. Define the solution vector x as a parallel finite element grid
-   //     function corresponding to fespace. Initialize x to zero.
-   GridFunction x(&fespace_nd);
-   x = 0.0;
-
-   // 11. Set up the parallel bilinear form corresponding to the EM diffusion
-   //     operator curl muinv curl + delta I, by adding the curl-curl and the
-   //     mass domain integrators. For standard magnetostatics equations choose
-   //     delta << 1. Larger values of delta should make the linear system
-   //     easier to solve at the expense of resembling a diffusive quasistatic
-   //     magnetic field.  A reasonable balance must be found whenever the mesh
-   //     or problem setup is altered.
-   ConstantCoefficient muinv(1.0);
-   ConstantCoefficient delta(delta_const);
-   BilinearForm a(&fespace_nd);
-   if (pa_) { a.SetAssemblyLevel(AssemblyLevel::PARTIAL); }
-   a.AddDomainIntegrator(new CurlCurlIntegrator(muinv));
-   a.AddDomainIntegrator(new VectorFEMassIntegrator(delta));
-
-   // 12. Assemble the parallel bilinear form and the corresponding linear
-   //     system, applying any necessary transformations such as: parallel
-   //     assembly, eliminating boundary conditions, applying conforming
-   //     constraints for non-conforming AMR, static condensation, etc.
-   if (static_cond) { a.EnableStaticCondensation(); }
-   a.Assemble();
-
-   OperatorPtr A;
-   Vector B, X;
-   a.FormLinearSystem(ess_tdof_list, x, b, A, X, B);
-
-   // 13. Solve the system AX=B
-   if (pa_) // Jacobi preconditioning in partial assembly mode
-   {
-      cout << "\nSolving for magnetic vector potential "
-           << "using CG with a Jacobi preconditioner" << endl;
-
-      OperatorJacobiSmoother M(a, ess_tdof_list);
-      PCG(*A, M, B, X, 1, 1000, 1e-12, 0.0);
-   }
-   else
-   {
-#ifndef MFEM_USE_SUITESPARSE
-      cout << "\nSolving for magnetic vector potential "
-           << "using CG with a Gauss-Seidel preconditioner" << endl;
-
-      // 13a. Define a simple symmetric Gauss-Seidel preconditioner and use
-      //     it to solve the system Ax=b with PCG.
-      GSSmoother M((SparseMatrix&)(*A));
-      PCG(*A, M, B, X, 1, 500, 1e-12, 0.0);
-#else
-      cout << "\nSolving for magnetic vector potential "
-           << "using UMFPack" << endl;
-
-      // 13a. If MFEM was compiled with SuiteSparse, use UMFPACK to solve the
-      //     system.
-      UMFPackSolver umf_solver;
-      umf_solver.Control[UMFPACK_ORDERING] = UMFPACK_ORDERING_METIS;
-      umf_solver.SetOperator(*A);
-      umf_solver.Mult(B, X);
-#endif
-   }
-
-   // 14. Recover the parallel grid function corresponding to X. This is the
-   //     local finite element solution on each processor.
-   a.RecoverFEMSolution(X, b, x);
-
-   // 15. Save the refined mesh and the solution in parallel. This output can
-   //     be viewed later using GLVis: "glvis -np <np> -m mesh -g sol".
-   {
-      ostringstream mesh_name, sol_name;
-      mesh_name << "refined.mesh";
-      sol_name << "sol.gf";
-
-      ofstream mesh_ofs(mesh_name.str().c_str());
-      mesh_ofs.precision(8);
-      mesh.Print(mesh_ofs);
-
-      ofstream sol_ofs(sol_name.str().c_str());
-      sol_ofs.precision(8);
-      x.Save(sol_ofs);
-   }
-
-   // 16. Send the solution by socket to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock(vishost, visport);
-      sol_sock.precision(8);
-      sol_sock << "solution\n" << mesh << x
-               << "window_title 'Vector Potential'"
-               << "window_geometry 800 0 400 350" << flush;
-   }
-
-   // 17. Compute the magnetic flux as the curl of the solution
-   DiscreteLinearOperator curl(&fespace_nd, &fespace_rt);
-   curl.AddDomainInterpolator(new CurlInterpolator);
-   curl.Assemble();
-   curl.Finalize();
-
-   GridFunction dx(&fespace_rt);
-   curl.Mult(x, dx);
-
-   // 18. Save the curl of the solution in parallel. This output can be viewed
-   //     later using GLVis: "glvis -np <np> -m mesh -g dsol".
-   {
-      ostringstream dsol_name;
-      dsol_name << "dsol.gf";
-
-      ofstream dsol_ofs(dsol_name.str().c_str());
-      dsol_ofs.precision(8);
-      dx.Save(dsol_ofs);
-   }
-
-   // 19. Send the curl of the solution by socket to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock(vishost, visport);
-      sol_sock.precision(8);
-      sol_sock << "solution\n" << mesh << dx
-               << "window_title 'Magnetic Flux'"
-               << "window_geometry 1200 0 400 350" << flush;
-   }
-
-   // 20. Clean exit
-   return 0;
-}
-
-void ComputeCurrentDensityOnSubMesh(int order,
-                                    const Array<int> &phi0_attr,
-                                    const Array<int> &phi1_attr,
-                                    const Array<int> &jn_zero_attr,
-                                    GridFunction &j_cond)
-{
-   // Extract the finite element space and mesh on which j_cond is defined
-   FiniteElementSpace &fes_cond_rt = *j_cond.FESpace();
-   Mesh &mesh_cond = *fes_cond_rt.GetMesh();
-   int dim  = mesh_cond.Dimension();
-
-   // Define a parallel finite element space on the SubMesh. Here we use the H1
-   // finite elements for the electrostatic potential.
-   H1_FECollection fec_h1(order, dim);
-   FiniteElementSpace fes_cond_h1(&mesh_cond, &fec_h1);
-
-   // Define the conductivity coefficient and the boundaries associated with the
-   // fixed potentials phi0 and phi1 which will drive the current.
-   ConstantCoefficient sigmaCoef(1.0);
-   Array<int> ess_bdr_phi(mesh_cond.bdr_attributes.Max());
-   Array<int> ess_bdr_j(mesh_cond.bdr_attributes.Max());
-   Array<int> ess_bdr_tdof_phi;
-   ess_bdr_phi = 0;
-   ess_bdr_j   = 0;
-   for (int i=0; i<phi0_attr.Size(); i++)
-   {
-      ess_bdr_phi[phi0_attr[i]-1] = 1;
-   }
-   for (int i=0; i<phi1_attr.Size(); i++)
-   {
-      ess_bdr_phi[phi1_attr[i]-1] = 1;
-   }
-   for (int i=0; i<jn_zero_attr.Size(); i++)
-   {
-      ess_bdr_j[jn_zero_attr[i]-1] = 1;
-   }
-   fes_cond_h1.GetEssentialTrueDofs(ess_bdr_phi, ess_bdr_tdof_phi);
-
-   // Setup the bilinear form corresponding to -Div(sigma Grad phi)
-   BilinearForm a_h1(&fes_cond_h1);
-   a_h1.AddDomainIntegrator(new DiffusionIntegrator(sigmaCoef));
-   a_h1.Assemble();
-
-   // Set the r.h.s. to zero
-   LinearForm b_h1(&fes_cond_h1);
-   b_h1 = 0.0;
-
-   // Setup the boundary conditions on phi
-   ConstantCoefficient one(1.0);
-   ConstantCoefficient zero(0.0);
-   GridFunction phi_h1(&fes_cond_h1);
-   phi_h1 = 0.0;
-
-   Array<int> bdr0(mesh_cond.bdr_attributes.Max()); bdr0 = 0;
-   for (int i=0; i<phi0_attr.Size(); i++)
-   {
-      bdr0[phi0_attr[i]-1] = 1;
-   }
-   phi_h1.ProjectBdrCoefficient(zero, bdr0);
-
-   Array<int> bdr1(mesh_cond.bdr_attributes.Max()); bdr1 = 0;
-   for (int i=0; i<phi1_attr.Size(); i++)
-   {
-      bdr1[phi1_attr[i]-1] = 1;
-   }
-   phi_h1.ProjectBdrCoefficient(one, bdr1);
-
-   {
-      OperatorPtr A;
-      Vector B, X;
-      a_h1.FormLinearSystem(ess_bdr_tdof_phi, phi_h1, b_h1, A, X, B);
-
-      // Solve the linear system
-      if (!pa_)
-      {
-#ifndef MFEM_USE_SUITESPARSE
-         cout << "\nSolving for electric potential using PCG "
-              << "with a Gauss-Seidel preconditioner" << endl;
-
-         // Use a simple symmetric Gauss-Seidel preconditioner with PCG.
-         GSSmoother M((SparseMatrix&)(*A));
-         PCG(*A, M, B, X, 1, 200, 1e-12, 0.0);
-#else
-         cout << "\nSolving for electric potential using UMFPack" << endl;
-
-         // If MFEM was compiled with SuiteSparse,
-         // use UMFPACK to solve the system.
-         UMFPackSolver umf_solver;
-         umf_solver.Control[UMFPACK_ORDERING] = UMFPACK_ORDERING_METIS;
-         umf_solver.SetOperator(*A);
-         umf_solver.Mult(B, X);
-#endif
-      }
-      else
-      {
-         cout << "\nSolving for electric potential using CG" << endl;
-
-         if (UsesTensorBasis(fes_cond_h1))
-         {
-            if (algebraic_ceed_)
-            {
-               ceed::AlgebraicSolver M(a_h1, ess_bdr_tdof_phi);
-               PCG(*A, M, B, X, 1, 400, 1e-12, 0.0);
-            }
-            else
-            {
-               OperatorJacobiSmoother M(a_h1, ess_bdr_tdof_phi);
-               PCG(*A, M, B, X, 1, 400, 1e-12, 0.0);
-            }
-         }
-         else
-         {
-            CG(*A, B, X, 1, 400, 1e-12, 0.0);
-         }
-      }
-      a_h1.RecoverFEMSolution(X, b_h1, phi_h1);
-   }
-
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream port_sock(vishost, visport);
-      port_sock.precision(8);
-      port_sock << "solution\n" << mesh_cond << phi_h1
-                << "window_title 'Conductor Potential'"
-                << "window_geometry 0 0 400 350" << flush;
-   }
-
-   // Solve for the current density J = -sigma Grad phi with boundary conditions
-   // J.n = 0 on the walls of the conductor but not on the ports where phi=0 and
-   // phi=1.
-
-   // J will be computed in H(div) so we need an RT mass matrix
-   BilinearForm m_rt(&fes_cond_rt);
-   m_rt.AddDomainIntegrator(new VectorFEMassIntegrator);
-   m_rt.Assemble();
-
-   // Assemble the (sigma Grad phi) operator
-   MixedBilinearForm d_h1(&fes_cond_h1, &fes_cond_rt);
-   d_h1.AddDomainIntegrator(new MixedVectorGradientIntegrator(sigmaCoef));
-   d_h1.Assemble();
-
-   // Compute the r.h.s, b_rt = sigma E = -sigma Grad phi
-   LinearForm b_rt(&fes_cond_rt);
-   d_h1.Mult(phi_h1, b_rt);
-   b_rt *= -1.0;
-
-   // Apply the necessary boundary conditions and solve for J in H(div)
-   cout << "\nSolving for current density in H(Div) "
-        << "using diagonally scaled CG" << endl;
-   cout << "Size of linear system: "
-        << fes_cond_rt.GetTrueVSize() << endl;
-
-   Array<int> ess_bdr_tdof_rt;
-   OperatorPtr M;
-   Vector B, X;
-
-   fes_cond_rt.GetEssentialTrueDofs(ess_bdr_j, ess_bdr_tdof_rt);
-
-   j_cond = 0.0;
-   m_rt.FormLinearSystem(ess_bdr_tdof_rt, j_cond, b_rt, M, X, B);
-
-   CGSolver cg;
-   cg.SetRelTol(1e-12);
-   cg.SetMaxIter(2000);
-   cg.SetPrintLevel(1);
-   cg.SetOperator(*M);
-   cg.Mult(B, X);
-   m_rt.RecoverFEMSolution(X, b_rt, j_cond);
-}
@@ -1,648 +0,0 @@
-//                       MFEM Example 34 - Parallel Version
-//
-// Compile with: make ex34p
-//
-// Sample runs:  mpirun -np 4 ex34p -o 2
-//               mpirun -np 4 ex34p -o 2 -hex -pa
-//
-// Device sample runs:
-//               mpirun -np 4 ex34p -o 2 -hex -pa -d cuda
-//               mpirun -np 4 ex34p -o 2 -no-pa -d cuda
-//
-// Description:  This example code solves a simple magnetostatic problem
-//               curl curl A = J where the current density J is computed on a
-//               subset of the domain as J = -sigma grad phi. We discretize the
-//               vector potential with Nedelec finite elements, the scalar
-//               potential with Lagrange finite elements, and the current
-//               density with Raviart-Thomas finite elements.
-//
-//               The example demonstrates the use of a SubMesh to compute the
-//               scalar potential and its associated current density which is
-//               then transferred to the original mesh and used as a source
-//               function.
-//
-//               Note that this example takes certain liberties with the
-//               current density which is not necessarily divergence free
-//               as it should be. This was done to focus on the use of the
-//               SubMesh to transfer information between a full mesh and a
-//               sub-domain. A more rigorous implementation might employ an
-//               H(div) saddle point solver to obtain a divergence free J on
-//               the SubMesh. It would then also need to ensure that the r.h.s.
-//               of curl curl A = J does in fact lie in the range of the weak
-//               curl operator by performing a divergence cleaning procedure
-//               before the solve. After divergence cleaning the delta
-//               parameter would probably not be needed.
-//
-//               This example is designed to make use of a specific mesh which
-//               has a known configuration of elements and boundary attributes.
-//               Other meshes could be used but extra care would be required to
-//               properly define the SubMesh and the necessary boundaries.
-//
-//               We recommend viewing examples 1 and 3 before viewing this
-//               example.
-
-#include "mfem.hpp"
-#include <fstream>
-#include <iostream>
-
-using namespace std;
-using namespace mfem;
-
-void ComputeCurrentDensityOnSubMesh(int order,
-                                    const Array<int> &phi0_attr,
-                                    const Array<int> &phi1_attr,
-                                    const Array<int> &jn_zero_attr,
-                                    ParGridFunction &j_cond);
-
-int main(int argc, char *argv[])
-{
-   // 1. Initialize MPI and HYPRE.
-   Mpi::Init(argc, argv);
-   int num_procs = Mpi::WorldSize();
-   int myid = Mpi::WorldRank();
-   Hypre::Init();
-
-   // 2. Parse command-line options.
-   const char *mesh_file = "../data/fichera-mixed.mesh";
-   Array<int> cond_attr;
-   Array<int> submesh_elems;
-   Array<int> sym_plane_attr;
-   Array<int> phi0_attr;
-   Array<int> phi1_attr;
-   Array<int> jn_zero_attr;
-   int ser_ref_levels = 1;
-   int par_ref_levels = 1;
-   int order = 1;
-   double delta_const = 1e-6;
-   bool mixed = true;
-   bool static_cond = false;
-   bool pa = false;
-   const char *device_config = "cpu";
-   bool visualization = true;
-#ifdef MFEM_USE_AMGX
-   bool useAmgX = false;
-#endif
-
-   OptionsParser args(argc, argv);
-   args.AddOption(&ser_ref_levels, "-rs", "--refine-serial",
-                  "Number of times to refine the mesh uniformly in serial.");
-   args.AddOption(&par_ref_levels, "-rp", "--refine-parallel",
-                  "Number of times to refine the mesh uniformly in parallel.");
-   args.AddOption(&order, "-o", "--order",
-                  "Finite element order (polynomial degree).");
-   args.AddOption(&delta_const, "-mc", "--magnetic-cond",
-                  "Magnetic Conductivity");
-   args.AddOption(&static_cond, "-sc", "--static-condensation", "-no-sc",
-                  "--no-static-condensation", "Enable static condensation.");
-   args.AddOption(&mixed, "-mixed", "--mixed-mesh", "-hex",
-                  "--hex-mesh", "Mixed mesh of hexahedral mesh.");
-   args.AddOption(&pa, "-pa", "--partial-assembly", "-no-pa",
-                  "--no-partial-assembly", "Enable Partial Assembly.");
-   args.AddOption(&device_config, "-d", "--device",
-                  "Device configuration string, see Device::Configure().");
-   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
-                  "--no-visualization",
-                  "Enable or disable GLVis visualization.");
-#ifdef MFEM_USE_AMGX
-   args.AddOption(&useAmgX, "-amgx", "--useAmgX", "-no-amgx",
-                  "--no-useAmgX",
-                  "Enable or disable AmgX in MatrixFreeAMS.");
-#endif
-
-   args.Parse();
-   if (!args.Good())
-   {
-      if (myid == 0)
-      {
-         args.PrintUsage(cout);
-      }
-      return 1;
-   }
-   if (myid == 0)
-   {
-      args.PrintOptions(cout);
-   }
-
-   if (!mixed || pa)
-   {
-      mesh_file = "../data/fichera.mesh";
-   }
-
-   if (submesh_elems.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0)
-      {
-         submesh_elems.SetSize(5);
-         submesh_elems[0] = 0;
-         submesh_elems[1] = 2;
-         submesh_elems[2] = 3;
-         submesh_elems[3] = 4;
-         submesh_elems[4] = 9;
-      }
-      else if (strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         submesh_elems.SetSize(7);
-         submesh_elems[0] = 10;
-         submesh_elems[1] = 14;
-         submesh_elems[2] = 34;
-         submesh_elems[3] = 36;
-         submesh_elems[4] = 37;
-         submesh_elems[5] = 38;
-         submesh_elems[6] = 39;
-      }
-   }
-   if (sym_plane_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         sym_plane_attr.SetSize(8);
-         sym_plane_attr[0] =  9;
-         sym_plane_attr[1] = 10;
-         sym_plane_attr[2] = 11;
-         sym_plane_attr[3] = 12;
-         sym_plane_attr[4] = 13;
-         sym_plane_attr[5] = 14;
-         sym_plane_attr[6] = 15;
-         sym_plane_attr[7] = 16;
-      }
-   }
-   if (phi0_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         phi0_attr.Append(2);
-      }
-   }
-   if (phi1_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         phi1_attr.Append(23);
-      }
-   }
-   if (jn_zero_attr.Size() == 0)
-   {
-      if (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-          strcmp(mesh_file, "../data/fichera.mesh") == 0)
-      {
-         jn_zero_attr.Append(25);
-      }
-      for (int i=0; i<sym_plane_attr.Size(); i++)
-      {
-         jn_zero_attr.Append(sym_plane_attr[i]);
-      }
-   }
-
-   // 3. Enable hardware devices such as GPUs, and programming models such as
-   //    CUDA, OCCA, RAJA and OpenMP based on command line options.
-   Device device(device_config);
-   if (myid == 0) { device.Print(); }
-
-   // 4. Read the (serial) mesh from the given mesh file on all processors.  We
-   //    can handle triangular, quadrilateral, tetrahedral, hexahedral, surface
-   //    and volume meshes with the same code.
-   Mesh *mesh = new Mesh(mesh_file, 1, 1);
-   int dim = mesh->Dimension();
-
-   if (!mixed || pa)
-   {
-      mesh->UniformRefinement();
-
-      if (ser_ref_levels > 0)
-      {
-         ser_ref_levels--;
-      }
-      else
-      {
-         par_ref_levels--;
-      }
-   }
-
-   int submesh_attr = -1;
-   if (cond_attr.Size() == 0 && submesh_elems.Size() > 0)
-   {
-      int max_attr = mesh->attributes.Max();
-      submesh_attr = max_attr + 1;
-
-      for (int i=0; i<submesh_elems.Size(); i++)
-      {
-         mesh->SetAttribute(submesh_elems[i], submesh_attr);
-      }
-      mesh->SetAttributes();
-
-      if (cond_attr.Size() == 0)
-      {
-         cond_attr.Append(submesh_attr);
-      }
-   }
-
-   // 5. Refine the serial mesh on all processors to increase the resolution. In
-   //    this example we do 'ref_levels' of uniform refinement.
-   {
-      int ref_levels = ser_ref_levels;
-      for (int l = 0; l < ref_levels; l++)
-      {
-         mesh->UniformRefinement();
-      }
-   }
-
-   // 6. Define a parallel mesh by a partitioning of the serial mesh. Refine
-   //    this mesh further in parallel to increase the resolution. Once the
-   //    parallel mesh is defined, the serial mesh can be deleted.
-   ParMesh pmesh(MPI_COMM_WORLD, *mesh);
-   delete mesh;
-   {
-      for (int l = 0; l < par_ref_levels; l++)
-      {
-         pmesh.UniformRefinement();
-      }
-   }
-
-   // 6b. Extract a submesh covering a portion of the domain
-   ParSubMesh pmesh_cond(ParSubMesh::CreateFromDomain(pmesh, cond_attr));
-
-   // 7. Define a suitable finite element space on the SubMesh and compute
-   //    the current density as an H(div) field.
-   RT_FECollection fec_cond_rt(order - 1, dim);
-   ParFiniteElementSpace fes_cond_rt(&pmesh_cond, &fec_cond_rt);
-   ParGridFunction j_cond(&fes_cond_rt);
-
-   ComputeCurrentDensityOnSubMesh(order, phi0_attr, phi1_attr, jn_zero_attr,
-                                  j_cond);
-
-   // 7a. Save the SubMesh and associated current density in parallel. This
-   //     output can be viewed later using GLVis:
-   //        "glvis -np <np> -m cond_mesh -g cond_j"
-   {
-      ostringstream mesh_name, cond_name;
-      mesh_name << "cond_mesh." << setfill('0') << setw(6) << myid;
-      cond_name << "cond_j." << setfill('0') << setw(6) << myid;
-
-      ofstream mesh_ofs(mesh_name.str().c_str());
-      mesh_ofs.precision(8);
-      pmesh_cond.Print(mesh_ofs);
-
-      ofstream cond_ofs(cond_name.str().c_str());
-      cond_ofs.precision(8);
-      j_cond.Save(cond_ofs);
-   }
-   // 7b. Send the current density, computed on the SubMesh, to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream port_sock(vishost, visport);
-      port_sock << "parallel " << num_procs << " " << myid << "\n";
-      port_sock.precision(8);
-      port_sock << "solution\n" << pmesh_cond << j_cond
-                << "window_title 'Conductor J'"
-                << "window_geometry 400 0 400 350" << flush;
-   }
-
-   // 8. Define a parallel finite element space on the full mesh. Here we use
-   //    the H(curl) finite elements for the vector potential and H(div) for the
-   //    current density.
-   ND_FECollection fec_nd(order, dim);
-   RT_FECollection fec_rt(order - 1, dim);
-   ParFiniteElementSpace fespace_nd(&pmesh, &fec_nd);
-   ParFiniteElementSpace fespace_rt(&pmesh, &fec_rt);
-
-   ParGridFunction j_full(&fespace_rt);
-   j_full = 0.0;
-   pmesh_cond.Transfer(j_cond, j_full);
-
-   // 8a. Send the transferred current density to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock(vishost, visport);
-      sol_sock << "parallel " << num_procs << " " << myid << "\n";
-      sol_sock.precision(8);
-      sol_sock << "solution\n" << pmesh << j_full
-               << "window_title 'J Full'"
-               << "window_geometry 400 430 400 350" << flush;
-   }
-
-   // 9. Determine the list of true (i.e. parallel conforming) essential
-   //    boundary dofs. In this example, the boundary conditions are defined
-   //    by marking all the boundary attributes except for those on a symmetry
-   //    plane as essential (Dirichlet) and converting them to a list of
-   //    true dofs.
-   Array<int> ess_tdof_list;
-   Array<int> ess_bdr;
-   if (pmesh.bdr_attributes.Size())
-   {
-      ess_bdr.SetSize(pmesh.bdr_attributes.Max());
-      ess_bdr = 1;
-      for (int i=0; i<sym_plane_attr.Size(); i++)
-      {
-         ess_bdr[sym_plane_attr[i]-1] = 0;
-      }
-      fespace_nd.GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
-   }
-
-   // 10. Set up the parallel linear form b(.) which corresponds to the
-   //     right-hand side of the FEM linear system, which in this case is
-   //     (J,W_i) where J is given by the function H(div) field transferred
-   //     from the SubMesh and W_i are the basis functions in the finite
-   //     element fespace.
-   VectorGridFunctionCoefficient jCoef(&j_full);
-   ParLinearForm b(&fespace_nd);
-   b.AddDomainIntegrator(new VectorFEDomainLFIntegrator(jCoef));
-   b.Assemble();
-
-   // 11. Define the solution vector x as a parallel finite element grid
-   //     function corresponding to fespace. Initialize x to zero.
-   ParGridFunction x(&fespace_nd);
-   x = 0.0;
-
-   // 12. Set up the parallel bilinear form corresponding to the EM diffusion
-   //     operator curl muinv curl + delta I, by adding the curl-curl and the
-   //     mass domain integrators. For standard magnetostatics equations choose
-   //     delta << 1. Larger values of delta should make the linear system
-   //     easier to solve at the expense of resembling a diffusive quasistatic
-   //     magnetic field.  A reasonable balance must be found whenever the mesh
-   //     or problem setup is altered.
-   ConstantCoefficient muinv(1.0);
-   ConstantCoefficient delta(delta_const);
-   ParBilinearForm a(&fespace_nd);
-   if (pa) { a.SetAssemblyLevel(AssemblyLevel::PARTIAL); }
-   a.AddDomainIntegrator(new CurlCurlIntegrator(muinv));
-   a.AddDomainIntegrator(new VectorFEMassIntegrator(delta));
-
-   // 13. Assemble the parallel bilinear form and the corresponding linear
-   //     system, applying any necessary transformations such as: parallel
-   //     assembly, eliminating boundary conditions, applying conforming
-   //     constraints for non-conforming AMR, static condensation, etc.
-   if (static_cond) { a.EnableStaticCondensation(); }
-   a.Assemble();
-
-   OperatorPtr A;
-   Vector B, X;
-   a.FormLinearSystem(ess_tdof_list, x, b, A, X, B);
-
-   if (myid == 0)
-   {
-      cout << "\nSolving for magnetic vector potential "
-           << "using CG with AMS" << endl;
-   }
-
-   // 14. Solve the system AX=B using PCG with an AMS preconditioner.
-   if (pa)
-   {
-#ifdef MFEM_USE_AMGX
-      MatrixFreeAMS ams(a, *A, fespace_nd, &muinv, &delta, NULL, ess_bdr,
-                        useAmgX);
-#else
-      MatrixFreeAMS ams(a, *A, fespace_nd, &muinv, &delta, NULL, ess_bdr);
-#endif
-      CGSolver cg(MPI_COMM_WORLD);
-      cg.SetRelTol(1e-12);
-      cg.SetMaxIter(1000);
-      cg.SetPrintLevel(1);
-      cg.SetOperator(*A);
-      cg.SetPreconditioner(ams);
-      cg.Mult(B, X);
-   }
-   else
-   {
-      if (myid == 0)
-      {
-         cout << "Size of linear system: "
-              << A.As<HypreParMatrix>()->GetGlobalNumRows() << endl;
-      }
-
-      ParFiniteElementSpace *prec_fespace =
-         (a.StaticCondensationIsEnabled() ? a.SCParFESpace() : &fespace_nd);
-      HypreAMS ams(*A.As<HypreParMatrix>(), prec_fespace);
-      HyprePCG pcg(*A.As<HypreParMatrix>());
-      pcg.SetTol(1e-12);
-      pcg.SetMaxIter(500);
-      pcg.SetPrintLevel(2);
-      pcg.SetPreconditioner(ams);
-      pcg.Mult(B, X);
-   }
-
-   // 15. Recover the parallel grid function corresponding to X. This is the
-   //     local finite element solution on each processor.
-   a.RecoverFEMSolution(X, b, x);
-
-   // 16. Save the refined mesh and the solution in parallel. This output can
-   //     be viewed later using GLVis: "glvis -np <np> -m mesh -g sol".
-   {
-      ostringstream mesh_name, sol_name;
-      mesh_name << "mesh." << setfill('0') << setw(6) << myid;
-      sol_name << "sol." << setfill('0') << setw(6) << myid;
-
-      ofstream mesh_ofs(mesh_name.str().c_str());
-      mesh_ofs.precision(8);
-      pmesh.Print(mesh_ofs);
-
-      ofstream sol_ofs(sol_name.str().c_str());
-      sol_ofs.precision(8);
-      x.Save(sol_ofs);
-   }
-
-   // 17. Send the solution by socket to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock(vishost, visport);
-      sol_sock << "parallel " << num_procs << " " << myid << "\n";
-      sol_sock.precision(8);
-      sol_sock << "solution\n" << pmesh << x
-               << "window_title 'Vector Potential'"
-               << "window_geometry 800 0 400 350" << flush;
-   }
-
-   // 18. Compute the magnetic flux as the curl of the solution
-   ParDiscreteLinearOperator curl(&fespace_nd, &fespace_rt);
-   curl.AddDomainInterpolator(new CurlInterpolator);
-   curl.Assemble();
-   curl.Finalize();
-
-   ParGridFunction dx(&fespace_rt);
-   curl.Mult(x, dx);
-
-   // 19. Save the curl of the solution in parallel. This output can
-   //     be viewed later using GLVis: "glvis -np <np> -m mesh -g dsol".
-   {
-      ostringstream dsol_name;
-      dsol_name << "dsol." << setfill('0') << setw(6) << myid;
-
-      ofstream dsol_ofs(dsol_name.str().c_str());
-      dsol_ofs.precision(8);
-      dx.Save(dsol_ofs);
-   }
-
-   // 20. Send the curl of the solution by socket to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock(vishost, visport);
-      sol_sock << "parallel " << num_procs << " " << myid << "\n";
-      sol_sock.precision(8);
-      sol_sock << "solution\n" << pmesh << dx
-               << "window_title 'Magnetic Flux'"
-               << "window_geometry 1200 0 400 350" << flush;
-   }
-
-   // 21. Clean exit
-   return 0;
-}
-
-void ComputeCurrentDensityOnSubMesh(int order,
-                                    const Array<int> &phi0_attr,
-                                    const Array<int> &phi1_attr,
-                                    const Array<int> &jn_zero_attr,
-                                    ParGridFunction &j_cond)
-{
-   // Extract the finite element space and mesh on which j_cond is defined
-   ParFiniteElementSpace &fes_cond_rt = *j_cond.ParFESpace();
-   ParMesh &pmesh_cond = *fes_cond_rt.GetParMesh();
-   int myid = fes_cond_rt.GetMyRank();
-   int dim  = pmesh_cond.Dimension();
-
-   // Define a parallel finite element space on the SubMesh. Here we use the
-   // H1 finite elements for the electrostatic potential.
-   H1_FECollection fec_h1(order, dim);
-   ParFiniteElementSpace fes_cond_h1(&pmesh_cond, &fec_h1);
-
-   // Define the conductivity coefficient and the boundaries associated with the
-   // fixed potentials phi0 and phi1 which will drive the current.
-   ConstantCoefficient sigmaCoef(1.0);
-   Array<int> ess_bdr_phi(pmesh_cond.bdr_attributes.Max());
-   Array<int> ess_bdr_j(pmesh_cond.bdr_attributes.Max());
-   Array<int> ess_bdr_tdof_phi;
-   ess_bdr_phi = 0;
-   ess_bdr_j   = 0;
-   for (int i=0; i<phi0_attr.Size(); i++)
-   {
-      ess_bdr_phi[phi0_attr[i]-1] = 1;
-   }
-   for (int i=0; i<phi1_attr.Size(); i++)
-   {
-      ess_bdr_phi[phi1_attr[i]-1] = 1;
-   }
-   for (int i=0; i<jn_zero_attr.Size(); i++)
-   {
-      ess_bdr_j[jn_zero_attr[i]-1] = 1;
-   }
-   fes_cond_h1.GetEssentialTrueDofs(ess_bdr_phi, ess_bdr_tdof_phi);
-
-   // Setup the bilinear form corresponding to -Div(sigma Grad phi)
-   ParBilinearForm a_h1(&fes_cond_h1);
-   a_h1.AddDomainIntegrator(new DiffusionIntegrator(sigmaCoef));
-   a_h1.Assemble();
-
-   // Set the r.h.s. to zero
-   ParLinearForm b_h1(&fes_cond_h1);
-   b_h1 = 0.0;
-
-   // Setup the boundary conditions on phi
-   ConstantCoefficient one(1.0);
-   ConstantCoefficient zero(0.0);
-   ParGridFunction phi_h1(&fes_cond_h1);
-   phi_h1 = 0.0;
-
-   Array<int> bdr0(pmesh_cond.bdr_attributes.Max()); bdr0 = 0;
-   for (int i=0; i<phi0_attr.Size(); i++)
-   {
-      bdr0[phi0_attr[i]-1] = 1;
-   }
-   phi_h1.ProjectBdrCoefficient(zero, bdr0);
-
-   Array<int> bdr1(pmesh_cond.bdr_attributes.Max()); bdr1 = 0;
-   for (int i=0; i<phi1_attr.Size(); i++)
-   {
-      bdr1[phi1_attr[i]-1] = 1;
-   }
-   phi_h1.ProjectBdrCoefficient(one, bdr1);
-
-   // Solve the linear system using algebraic multigrid
-   {
-      if (myid == 0)
-      {
-         cout << "\nSolving for electric potential "
-              << "using CG with AMG" << endl;
-      }
-      OperatorPtr A;
-      Vector B, X;
-      a_h1.FormLinearSystem(ess_bdr_tdof_phi, phi_h1, b_h1, A, X, B);
-
-      HypreBoomerAMG prec;
-      CGSolver cg(MPI_COMM_WORLD);
-      cg.SetRelTol(1e-12);
-      cg.SetMaxIter(2000);
-      cg.SetPrintLevel(1);
-      cg.SetPreconditioner(prec);
-      cg.SetOperator(*A);
-      cg.Mult(B, X);
-      a_h1.RecoverFEMSolution(X, b_h1, phi_h1);
-   }
-   {
-      int num_procs = fes_cond_h1.GetNRanks();
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream port_sock(vishost, visport);
-      port_sock << "parallel " << num_procs << " " << myid << "\n";
-      port_sock.precision(8);
-      port_sock << "solution\n" << pmesh_cond << phi_h1
-                << "window_title 'Conductor Potential'"
-                << "window_geometry 0 0 400 350" << flush;
-   }
-
-   // Solve for the current density J = -sigma Grad phi with boundary conditions
-   // J.n = 0 on the walls of the conductor but not on the ports where phi=0 and
-   // phi=1.
-
-   // J will be computed in H(div) so we need an RT mass matrix
-   ParBilinearForm m_rt(&fes_cond_rt);
-   m_rt.AddDomainIntegrator(new VectorFEMassIntegrator);
-   m_rt.Assemble();
-
-   // Assemble the (sigma Grad phi) operator
-   ParMixedBilinearForm d_h1(&fes_cond_h1, &fes_cond_rt);
-   d_h1.AddDomainIntegrator(new MixedVectorGradientIntegrator(sigmaCoef));
-   d_h1.Assemble();
-
-   // Compute the r.h.s, b_rt = sigma E = -sigma Grad phi
-   ParLinearForm b_rt(&fes_cond_rt);
-   d_h1.Mult(phi_h1, b_rt);
-   b_rt *= -1.0;
-
-   // Apply the necessary boundary conditions and solve for J in H(div)
-   HYPRE_BigInt glb_size_rt = fes_cond_rt.GlobalTrueVSize();
-   if (myid == 0)
-   {
-      cout << "\nSolving for current density in H(Div) "
-           << "using diagonally scaled CG" << endl;
-      cout << "Size of linear system: "
-           << glb_size_rt << endl;
-   }
-   Array<int> ess_bdr_tdof_rt;
-   OperatorPtr M;
-   Vector B, X;
-
-   fes_cond_rt.GetEssentialTrueDofs(ess_bdr_j, ess_bdr_tdof_rt);
-
-   j_cond = 0.0;
-   m_rt.FormLinearSystem(ess_bdr_tdof_rt, j_cond, b_rt, M, X, B);
-
-   HypreDiagScale prec;
-
-   CGSolver cg(MPI_COMM_WORLD);
-   cg.SetRelTol(1e-12);
-   cg.SetMaxIter(2000);
-   cg.SetPrintLevel(1);
-   cg.SetPreconditioner(prec);
-   cg.SetOperator(*M);
-   cg.Mult(B, X);
-   m_rt.RecoverFEMSolution(X, b_rt, j_cond);
-}
@@ -1,815 +0,0 @@
-//                       MFEM Example 35 - Parallel Version
-//
-// Compile with: make ex35p
-//
-// Sample runs:  mpirun -np 4 ex35p -p 0 -o 2
-//               mpirun -np 4 ex35p -p 0 -o 2 -pbc '22 23 24' -em 0
-//               mpirun -np 4 ex35p -p 1 -o 1 -rp 2
-//               mpirun -np 4 ex35p -p 1 -o 2
-//               mpirun -np 4 ex35p -p 2 -o 1 -rp 2 -c 15
-//
-// Device sample runs:
-//
-// Description:  This example code demonstrates the use of MFEM to define and
-//               solve simple complex-valued linear systems. It implements three
-//               variants of a damped harmonic oscillator:
-//
-//               1) A scalar H1 field
-//                  -Div(a Grad u) - omega^2 b u + i omega c u = 0
-//
-//               2) A vector H(Curl) field
-//                  Curl(a Curl u) - omega^2 b u + i omega c u = 0
-//
-//               3) A vector H(Div) field
-//                  -Grad(a Div u) - omega^2 b u + i omega c u = 0
-//
-//               In each case the field is driven by a forced oscillation, with
-//               angular frequency omega, imposed at the boundary or a portion
-//               of the boundary. The spatial variation of the boundary
-//               condition is computed as an eigenmode of an appropriate
-//               operator defined on a portion of the boundary i.e. a port
-//               boundary condition.
-//
-//               In electromagnetics the coefficients are typically named the
-//               permeability, mu = 1/a, permittivity, epsilon = b, and
-//               conductivity, sigma = c. The user can specify these constants
-//               using either set of names.
-//
-//               This example demonstrates how to transfer fields computed on a
-//               boundary generated SubMesh to the full mesh and apply them as
-//               boundary conditions. The default mesh and corresponding
-//               boundary attributes were chosen to verify proper behavior on
-//               both triangular and quadrilateral faces of tetrahedral,
-//               wedge-shaped, and hexahedral elements.
-//
-//               The example also demonstrates how to display a time-varying
-//               solution as a sequence of fields sent to a single GLVis socket.
-//
-//               We recommend viewing examples 11, 13, and 22 before viewing
-//               this example.
-
-#include "mfem.hpp"
-#include <fstream>
-#include <iostream>
-
-using namespace std;
-using namespace mfem;
-
-static double mu_ = 1.0;
-static double epsilon_ = 1.0;
-static double sigma_ = 2.0;
-
-void SetPortBC(int prob, int dim, int mode, ParGridFunction &port_bc);
-
-int main(int argc, char *argv[])
-{
-   // 1. Initialize MPI and HYPRE.
-   Mpi::Init(argc, argv);
-   int num_procs = Mpi::WorldSize();
-   int myid = Mpi::WorldRank();
-   Hypre::Init();
-
-   // 2. Parse command-line options.
-   const char *mesh_file = "../data/fichera-mixed.mesh";
-   int ser_ref_levels = 1;
-   int par_ref_levels = 1;
-   int order = 1;
-   Array<int> port_bc_attr;
-   int prob = 0;
-   int mode = 1;
-   double freq = -1.0;
-   double omega = 2.0 * M_PI;
-   double a_coef = 0.0;
-   bool herm_conv = true;
-   bool slu_solver  = false;
-   bool visualization = 1;
-   bool mixed = true;
-   bool pa = false;
-   const char *device_config = "cpu";
-
-   OptionsParser args(argc, argv);
-   args.AddOption(&mesh_file, "-m", "--mesh",
-                  "Mesh file to use.");
-   args.AddOption(&ser_ref_levels, "-rs", "--refine-serial",
-                  "Number of times to refine the mesh uniformly in serial.");
-   args.AddOption(&par_ref_levels, "-rp", "--refine-parallel",
-                  "Number of times to refine the mesh uniformly in parallel.");
-   args.AddOption(&order, "-o", "--order",
-                  "Finite element order (polynomial degree).");
-   args.AddOption(&prob, "-p", "--problem-type",
-                  "Choose between 0: H_1, 1: H(Curl), or 2: H(Div) "
-                  "damped harmonic oscillator.");
-   args.AddOption(&mode, "-em", "--eigenmode",
-                  "Choose the index of the port eigenmode.");
-   args.AddOption(&a_coef, "-a", "--stiffness-coef",
-                  "Stiffness coefficient (spring constant or 1/mu).");
-   args.AddOption(&epsilon_, "-b", "--mass-coef",
-                  "Mass coefficient (or epsilon).");
-   args.AddOption(&sigma_, "-c", "--damping-coef",
-                  "Damping coefficient (or sigma).");
-   args.AddOption(&mu_, "-mu", "--permeability",
-                  "Permeability of free space (or 1/(spring constant)).");
-   args.AddOption(&epsilon_, "-eps", "--permittivity",
-                  "Permittivity of free space (or mass constant).");
-   args.AddOption(&sigma_, "-sigma", "--conductivity",
-                  "Conductivity (or damping constant).");
-   args.AddOption(&freq, "-f", "--frequency",
-                  "Frequency (in Hz).");
-   args.AddOption(&port_bc_attr, "-pbc", "--port-bc-attr",
-                  "Attributes of port boundary condition");
-   args.AddOption(&herm_conv, "-herm", "--hermitian", "-no-herm",
-                  "--no-hermitian", "Use convention for Hermitian operators.");
-#ifdef MFEM_USE_SUPERLU
-   args.AddOption(&slu_solver, "-slu", "--superlu", "-no-slu",
-                  "--no-superlu", "Use the SuperLU Solver.");
-#endif
-   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
-                  "--no-visualization",
-                  "Enable or disable GLVis visualization.");
-   args.AddOption(&mixed, "-mixed", "--mixed-mesh", "-hex",
-                  "--hex-mesh", "Mixed mesh of hexahedral mesh.");
-   args.AddOption(&pa, "-pa", "--partial-assembly", "-no-pa",
-                  "--no-partial-assembly", "Enable Partial Assembly.");
-   args.AddOption(&device_config, "-d", "--device",
-                  "Device configuration string, see Device::Configure().");
-   args.Parse();
-   if (!args.Good())
-   {
-      if (myid == 0)
-      {
-         args.PrintUsage(cout);
-      }
-      return 1;
-   }
-
-   if (!mixed || pa)
-   {
-      mesh_file = "../data/fichera.mesh";
-   }
-
-   if ( a_coef != 0.0 )
-   {
-      mu_ = 1.0 / a_coef;
-   }
-   if ( freq > 0.0 )
-   {
-      omega = 2.0 * M_PI * freq;
-   }
-   if (port_bc_attr.Size() == 0 &&
-       (strcmp(mesh_file, "../data/fichera-mixed.mesh") == 0 ||
-        strcmp(mesh_file, "../data/fichera.mesh") == 0))
-   {
-      port_bc_attr.SetSize(4);
-      port_bc_attr[0] =  7;
-      port_bc_attr[1] =  8;
-      port_bc_attr[2] = 11;
-      port_bc_attr[3] = 12;
-   }
-
-   if (myid == 0)
-   {
-      args.PrintOptions(cout);
-   }
-
-   MFEM_VERIFY(prob >= 0 && prob <=2,
-               "Unrecognized problem type: " << prob);
-
-   ComplexOperator::Convention conv =
-      herm_conv ? ComplexOperator::HERMITIAN : ComplexOperator::BLOCK_SYMMETRIC;
-
-   // 3. Enable hardware devices such as GPUs, and programming models such as
-   //    CUDA, OCCA, RAJA and OpenMP based on command line options.
-   Device device(device_config);
-   if (myid == 0) { device.Print(); }
-
-   // 4. Read the (serial) mesh from the given mesh file on all processors. We
-   //    can handle triangular, quadrilateral, tetrahedral, hexahedral, surface
-   //    and volume meshes with the same code.
-   Mesh *mesh = new Mesh(mesh_file, 1, 1);
-   int dim = mesh->Dimension();
-
-   // 5. Refine the serial mesh on all processors to increase the resolution.
-   for (int l = 0; l < ser_ref_levels; l++)
-   {
-      mesh->UniformRefinement();
-   }
-
-   // 6a. Define a parallel mesh by a partitioning of the serial mesh. Refine
-   //    this mesh further in parallel to increase the resolution. Once the
-   //    parallel mesh is defined, the serial mesh can be deleted.
-   ParMesh pmesh(MPI_COMM_WORLD, *mesh);
-   delete mesh;
-   for (int l = 0; l < par_ref_levels; l++)
-   {
-      pmesh.UniformRefinement();
-   }
-
-   // 6b. Extract a submesh covering a portion of the boundary
-   ParSubMesh pmesh_port(ParSubMesh::CreateFromBoundary(pmesh, port_bc_attr));
-
-   // 7a. Define a parallel finite element space on the parallel mesh. Here we
-   //     use continuous Lagrange, Nedelec, or Raviart-Thomas finite elements
-   //     of the specified order.
-   if (dim == 1 && prob != 0 )
-   {
-      if (myid == 0)
-      {
-         cout << "Switching to problem type 0, H1 basis functions, "
-              << "for 1 dimensional mesh." << endl;
-      }
-      prob = 0;
-   }
-
-   FiniteElementCollection *fec = NULL;
-   switch (prob)
-   {
-      case 0:  fec = new H1_FECollection(order, dim);      break;
-      case 1:  fec = new ND_FECollection(order, dim);      break;
-      case 2:  fec = new RT_FECollection(order - 1, dim);  break;
-      default: break; // This should be unreachable
-   }
-   ParFiniteElementSpace fespace(&pmesh, fec);
-   HYPRE_BigInt size = fespace.GlobalTrueVSize();
-   if (myid == 0)
-   {
-      cout << "Number of finite element unknowns: " << size << endl;
-   }
-
-   // 7b. Define a parallel finite element space on the sub-mesh. Here we
-   //    use continuous Lagrange, Nedelec, or L2 finite elements of
-   //    the specified order.
-   FiniteElementCollection *fec_port = NULL;
-   switch (prob)
-   {
-      case 0:  fec_port = new H1_FECollection(order, dim-1);      break;
-      case 1:
-         if (dim == 3)
-         {
-            fec_port = new ND_FECollection(order, dim-1);
-         }
-         else
-         {
-            fec_port = new L2_FECollection(order - 1, dim-1,
-                                           BasisType::GaussLegendre,
-                                           FiniteElement::INTEGRAL);
-         }
-         break;
-      case 2:  fec_port = new L2_FECollection(order - 1, dim-1,
-                                                 BasisType::GaussLegendre,
-                                                 FiniteElement::INTEGRAL); break;
-      default: break; // This should be unreachable
-   }
-   ParFiniteElementSpace fespace_port(&pmesh_port, fec_port);
-   HYPRE_BigInt size_port = fespace_port.GlobalTrueVSize();
-   if (myid == 0)
-   {
-      cout << "Number of finite element port BC unknowns: " << size_port
-           << endl;
-   }
-
-   // 8a. Define a parallel grid function on the SubMesh which will contain
-   //     the field to be applied as a port boundary condition.
-   ParGridFunction port_bc(&fespace_port);
-   port_bc = 0.0;
-
-   SetPortBC(prob, dim, mode, port_bc);
-
-   // 8b. Save the SubMesh and associated port boundary condition in parallel.
-   //     This output can be viewed later using GLVis:
-   //        "glvis -np <np> -m port_mesh -g port_mode"
-   {
-      ostringstream mesh_name, port_name;
-      mesh_name << "port_mesh." << setfill('0') << setw(6) << myid;
-      port_name << "port_mode." << setfill('0') << setw(6) << myid;
-
-      ofstream mesh_ofs(mesh_name.str().c_str());
-      mesh_ofs.precision(8);
-      pmesh_port.Print(mesh_ofs);
-
-      ofstream port_ofs(port_name.str().c_str());
-      port_ofs.precision(8);
-      port_bc.Save(port_ofs);
-   }
-   // 8c. Send the port bc, computed on the SubMesh, to a GLVis server.
-   if (visualization && dim == 3)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream port_sock(vishost, visport);
-      port_sock << "parallel " << num_procs << " " << myid << "\n";
-      port_sock.precision(8);
-      port_sock << "solution\n" << pmesh_port << port_bc
-                << "window_title 'Port BC'"
-                << "window_geometry 0 0 400 350" << flush;
-   }
-
-   // 9. Determine the list of true (i.e. parallel conforming) essential
-   //    boundary dofs. In this example, the boundary conditions are defined
-   //    using an eigenmode of the appropriate type computed on the SubMesh.
-   Array<int> ess_tdof_list;
-   Array<int> ess_bdr;
-   if (pmesh.bdr_attributes.Size())
-   {
-      ess_bdr.SetSize(pmesh.bdr_attributes.Max());
-      ess_bdr = 1;
-      fespace.GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
-   }
-
-   // 10. Set up the parallel linear form b(.) which corresponds to the
-   //     right-hand side of the FEM linear system.
-   ParComplexLinearForm b(&fespace, conv);
-   b.Vector::operator=(0.0);
-
-   // 11a. Define the solution vector u as a parallel complex finite element
-   //      grid function corresponding to fespace. Initialize u to equal zero.
-   ParComplexGridFunction u(&fespace);
-   u = 0.0;
-   pmesh_port.Transfer(port_bc, u.real());
-
-   // 11b. Send the transferred port bc field to a GLVis server.
-   {
-      ParGridFunction full_bc(&fespace);
-      ParTransferMap port_to_full(port_bc, full_bc);
-
-      full_bc = 0.0;
-      port_to_full.Transfer(port_bc, full_bc);
-
-      if (visualization)
-      {
-         char vishost[] = "localhost";
-         int  visport   = 19916;
-         socketstream full_sock(vishost, visport);
-         full_sock << "parallel " << num_procs << " " << myid << "\n";
-         full_sock.precision(8);
-         full_sock << "solution\n" << pmesh << full_bc
-                   << "window_title 'Transferred BC'"
-                   << "window_geometry 400 0 400 350"<< flush;
-      }
-   }
-
-   // 12. Set up the parallel sesquilinear form a(.,.) on the finite element
-   //     space corresponding to the damped harmonic oscillator operator of the
-   //     appropriate type:
-   //
-   //     0) A scalar H1 field
-   //        -Div(a Grad) - omega^2 b + i omega c
-   //
-   //     1) A vector H(Curl) field
-   //        Curl(a Curl) - omega^2 b + i omega c
-   //
-   //     2) A vector H(Div) field
-   //        -Grad(a Div) - omega^2 b + i omega c
-   //
-   ConstantCoefficient stiffnessCoef(1.0/mu_);
-   ConstantCoefficient massCoef(-omega * omega * epsilon_);
-   ConstantCoefficient lossCoef(omega * sigma_);
-   ConstantCoefficient negMassCoef(omega * omega * epsilon_);
-
-   ParSesquilinearForm a(&fespace, conv);
-   if (pa) { a.SetAssemblyLevel(AssemblyLevel::PARTIAL); }
-   switch (prob)
-   {
-      case 0:
-         a.AddDomainIntegrator(new DiffusionIntegrator(stiffnessCoef),
-                               NULL);
-         a.AddDomainIntegrator(new MassIntegrator(massCoef),
-                               new MassIntegrator(lossCoef));
-         break;
-      case 1:
-         a.AddDomainIntegrator(new CurlCurlIntegrator(stiffnessCoef),
-                               NULL);
-         a.AddDomainIntegrator(new VectorFEMassIntegrator(massCoef),
-                               new VectorFEMassIntegrator(lossCoef));
-         break;
-      case 2:
-         a.AddDomainIntegrator(new DivDivIntegrator(stiffnessCoef),
-                               NULL);
-         a.AddDomainIntegrator(new VectorFEMassIntegrator(massCoef),
-                               new VectorFEMassIntegrator(lossCoef));
-         break;
-      default: break; // This should be unreachable
-   }
-
-   // 13. Assemble the parallel bilinear form and the corresponding linear
-   //     system, applying any necessary transformations such as: parallel
-   //     assembly, eliminating boundary conditions, applying conforming
-   //     constraints for non-conforming AMR, etc.
-   a.Assemble();
-
-   OperatorHandle A;
-   Vector B, U;
-
-   a.FormLinearSystem(ess_tdof_list, u, b, A, U, B);
-
-   if (myid == 0)
-   {
-      cout << "Size of linear system: "
-           << 2 * size << endl << endl;
-   }
-
-   if (!slu_solver)
-   {
-      // 14a. Set up the parallel bilinear form for the preconditioner
-      //      corresponding to the appropriate operator
-      //
-      //      0) A scalar H1 field
-      //         -Div(a Grad) - omega^2 b + i omega c
-      //
-      //      1) A vector H(Curl) field
-      //         Curl(a Curl) + omega^2 b + i omega c
-      //
-      //      2) A vector H(Div) field
-      //         -Grad(a Div) - omega^2 b + i omega c
-      ParBilinearForm pcOp(&fespace);
-      if (pa) { pcOp.SetAssemblyLevel(AssemblyLevel::PARTIAL); }
-      switch (prob)
-      {
-         case 0:
-            pcOp.AddDomainIntegrator(new DiffusionIntegrator(stiffnessCoef));
-            pcOp.AddDomainIntegrator(new MassIntegrator(massCoef));
-            pcOp.AddDomainIntegrator(new MassIntegrator(lossCoef));
-            break;
-         case 1:
-            pcOp.AddDomainIntegrator(new CurlCurlIntegrator(stiffnessCoef));
-            pcOp.AddDomainIntegrator(new VectorFEMassIntegrator(negMassCoef));
-            pcOp.AddDomainIntegrator(new VectorFEMassIntegrator(lossCoef));
-            break;
-         case 2:
-            pcOp.AddDomainIntegrator(new DivDivIntegrator(stiffnessCoef));
-            pcOp.AddDomainIntegrator(new VectorFEMassIntegrator(massCoef));
-            pcOp.AddDomainIntegrator(new VectorFEMassIntegrator(lossCoef));
-            break;
-         default: break; // This should be unreachable
-      }
-      pcOp.Assemble();
-
-      // 14b. Define and apply a parallel FGMRES solver for AU=B with a block
-      //      diagonal preconditioner based on the appropriate multigrid
-      //      preconditioner from hypre.
-      Array<int> blockTrueOffsets;
-      blockTrueOffsets.SetSize(3);
-      blockTrueOffsets[0] = 0;
-      blockTrueOffsets[1] = A->Height() / 2;
-      blockTrueOffsets[2] = A->Height() / 2;
-      blockTrueOffsets.PartialSum();
-
-      BlockDiagonalPreconditioner BDP(blockTrueOffsets);
-
-      Operator * pc_r = NULL;
-      Operator * pc_i = NULL;
-
-      if (pa)
-      {
-         pc_r = new OperatorJacobiSmoother(pcOp, ess_tdof_list);
-      }
-      else
-      {
-         OperatorHandle PCOp;
-         pcOp.FormSystemMatrix(ess_tdof_list, PCOp);
-
-         switch (prob)
-         {
-            case 0:
-               pc_r = new HypreBoomerAMG(*PCOp.As<HypreParMatrix>());
-               break;
-            case 1:
-               pc_r = new HypreAMS(*PCOp.As<HypreParMatrix>(), &fespace);
-               break;
-            case 2:
-               if (dim == 2 )
-               {
-                  pc_r = new HypreAMS(*PCOp.As<HypreParMatrix>(), &fespace);
-               }
-               else
-               {
-                  pc_r = new HypreADS(*PCOp.As<HypreParMatrix>(), &fespace);
-               }
-               break;
-            default: break; // This should be unreachable
-         }
-      }
-      pc_i = new ScaledOperator(pc_r,
-                                (conv == ComplexOperator::HERMITIAN) ?
-                                -1.0:1.0);
-
-      BDP.SetDiagonalBlock(0, pc_r);
-      BDP.SetDiagonalBlock(1, pc_i);
-      BDP.owns_blocks = 1;
-
-      FGMRESSolver fgmres(MPI_COMM_WORLD);
-      fgmres.SetPreconditioner(BDP);
-      fgmres.SetOperator(*A.Ptr());
-      fgmres.SetRelTol(1e-6);
-      fgmres.SetMaxIter(1000);
-      fgmres.SetPrintLevel(1);
-      fgmres.Mult(B, U);
-   }
-#ifdef MFEM_USE_SUPERLU
-   else
-   {
-      // 14. Solve using a direct solver
-      // Transform to monolithic HypreParMatrix
-      HypreParMatrix *A_hyp = A.As<ComplexHypreParMatrix>()->GetSystemMatrix();
-      SuperLURowLocMatrix SA(*A_hyp);
-      SuperLUSolver superlu(MPI_COMM_WORLD);
-      superlu.SetPrintStatistics(true);
-      superlu.SetSymmetricPattern(false);
-      superlu.SetColumnPermutation(superlu::PARMETIS);
-      superlu.SetOperator(SA);
-      superlu.Mult(B, U);
-      delete A_hyp;
-   }
-#endif
-
-   // 15. Recover the parallel grid function corresponding to U. This is the
-   //     local finite element solution on each processor.
-   a.RecoverFEMSolution(U, b, u);
-
-   // 16. Save the refined mesh and the solution in parallel. This output can be
-   //     viewed later using GLVis: "glvis -np <np> -m mesh -g sol_r" or
-   //     "glvis -np <np> -m mesh -g sol_i".
-   {
-      ostringstream mesh_name, sol_r_name, sol_i_name;
-      mesh_name << "mesh." << setfill('0') << setw(6) << myid;
-      sol_r_name << "sol_r." << setfill('0') << setw(6) << myid;
-      sol_i_name << "sol_i." << setfill('0') << setw(6) << myid;
-
-      ofstream mesh_ofs(mesh_name.str().c_str());
-      mesh_ofs.precision(8);
-      pmesh.Print(mesh_ofs);
-
-      ofstream sol_r_ofs(sol_r_name.str().c_str());
-      ofstream sol_i_ofs(sol_i_name.str().c_str());
-      sol_r_ofs.precision(8);
-      sol_i_ofs.precision(8);
-      u.real().Save(sol_r_ofs);
-      u.imag().Save(sol_i_ofs);
-   }
-
-   // 17. Send the solution by socket to a GLVis server.
-   if (visualization)
-   {
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock_r(vishost, visport);
-      sol_sock_r << "parallel " << num_procs << " " << myid << "\n";
-      sol_sock_r.precision(8);
-      sol_sock_r << "solution\n" << pmesh << u.real()
-                 << "window_title 'Solution: Real Part'"
-                 << "window_geometry 800 0 400 350" << flush;
-
-      MPI_Barrier(MPI_COMM_WORLD);
-
-      socketstream sol_sock_i(vishost, visport);
-      sol_sock_i << "parallel " << num_procs << " " << myid << "\n";
-      sol_sock_i.precision(8);
-      sol_sock_i << "solution\n" << pmesh << u.imag()
-                 << "window_title 'Solution: Imaginary Part'"
-                 << "window_geometry 1200 0 400 350" << flush;
-   }
-   if (visualization)
-   {
-      ParGridFunction u_t(&fespace);
-      u_t = u.real();
-      char vishost[] = "localhost";
-      int  visport   = 19916;
-      socketstream sol_sock(vishost, visport);
-      sol_sock << "parallel " << num_procs << " " << myid << "\n";
-      sol_sock.precision(8);
-      sol_sock << "solution\n" << pmesh << u_t
-               << "window_title 'Harmonic Solution (t = 0.0 T)'"
-               << "window_geometry 0 432 600 450"
-               << "pause\n" << flush;
-      if (myid == 0)
-         cout << "GLVis visualization paused."
-              << " Press space (in the GLVis window) to resume it.\n";
-      int num_frames = 32;
-      int i = 0;
-      while (sol_sock)
-      {
-         double t = (double)(i % num_frames) / num_frames;
-         ostringstream oss;
-         oss << "Harmonic Solution (t = " << t << " T)";
-
-         add(cos( 2.0 * M_PI * t), u.real(),
-             sin(-2.0 * M_PI * t), u.imag(), u_t);
-         sol_sock << "parallel " << num_procs << " " << myid << "\n";
-         sol_sock << "solution\n" << pmesh << u_t
-                  << "window_title '" << oss.str() << "'" << flush;
-         i++;
-      }
-   }
-
-   // 18. Free the used memory.
-   delete fec_port;
-   delete fec;
-
-   return 0;
-}
-
-/**
-   Solves the eigenvalue problem -Div(Grad x) = lambda x with homogeneous
-   Dirichlet boundary conditions on the boundary of the domain. Returns mode
-   number "mode" (counting from zero) in the ParGridFunction "x".
-*/
-void ScalarWaveGuide(int mode, ParGridFunction &x)
-{
-   int nev = std::max(mode + 2, 5);
-   int seed = 75;
-
-   ParFiniteElementSpace &fespace = *x.ParFESpace();
-   ParMesh &pmesh = *fespace.GetParMesh();
-
-   Array<int> ess_bdr;
-   if (pmesh.bdr_attributes.Size())
-   {
-      ess_bdr.SetSize(pmesh.bdr_attributes.Max());
-      ess_bdr = 1;
-   }
-
-   ParBilinearForm a(&fespace);
-   a.AddDomainIntegrator(new DiffusionIntegrator);
-   a.Assemble();
-   a.EliminateEssentialBCDiag(ess_bdr, 1.0);
-   a.Finalize();
-
-   ParBilinearForm m(&fespace);
-   m.AddDomainIntegrator(new MassIntegrator);
-   m.Assemble();
-   // shift the eigenvalue corresponding to eliminated dofs to a large value
-   m.EliminateEssentialBCDiag(ess_bdr, numeric_limits<double>::min());
-   m.Finalize();
-
-   HypreParMatrix *A = a.ParallelAssemble();
-   HypreParMatrix *M = m.ParallelAssemble();
-
-   HypreBoomerAMG amg(*A);
-   amg.SetPrintLevel(0);
-
-   HypreLOBPCG lobpcg(MPI_COMM_WORLD);
-   lobpcg.SetNumModes(nev);
-   lobpcg.SetRandomSeed(seed);
-   lobpcg.SetPreconditioner(amg);
-   lobpcg.SetMaxIter(200);
-   lobpcg.SetTol(1e-8);
-   lobpcg.SetPrecondUsageMode(1);
-   lobpcg.SetPrintLevel(1);
-   lobpcg.SetMassMatrix(*M);
-   lobpcg.SetOperator(*A);
-   lobpcg.Solve();
-
-   x = lobpcg.GetEigenvector(mode);
-
-   delete A;
-   delete M;
-}
-
-/**
-   Solves the eigenvalue problem -Curl(Curl x) = lambda x with homogeneous
-   Dirichlet boundary conditions, on the tangential component of x, on the
-   boundary of the domain. Returns mode number "mode" (counting from zero) in
-   the ParGridFunction "x".
-*/
-void VectorWaveGuide(int mode, ParGridFunction &x)
-{
-   int nev = std::max(mode + 2, 5);
-
-   ParFiniteElementSpace &fespace = *x.ParFESpace();
-   ParMesh &pmesh = *fespace.GetParMesh();
-
-   Array<int> ess_bdr;
-   if (pmesh.bdr_attributes.Size())
-   {
-      ess_bdr.SetSize(pmesh.bdr_attributes.Max());
-      ess_bdr = 1;
-   }
-
-   ParBilinearForm a(&fespace);
-   a.AddDomainIntegrator(new CurlCurlIntegrator);
-   a.Assemble();
-   a.EliminateEssentialBCDiag(ess_bdr, 1.0);
-   a.Finalize();
-
-   ParBilinearForm m(&fespace);
-   m.AddDomainIntegrator(new VectorFEMassIntegrator);
-   m.Assemble();
-   // shift the eigenvalue corresponding to eliminated dofs to a large value
-   m.EliminateEssentialBCDiag(ess_bdr, numeric_limits<double>::min());
-   m.Finalize();
-
-   HypreParMatrix *A = a.ParallelAssemble();
-   HypreParMatrix *M = m.ParallelAssemble();
-
-   HypreAMS ams(*A,&fespace);
-   ams.SetPrintLevel(0);
-   ams.SetSingularProblem();
-
-   HypreAME ame(MPI_COMM_WORLD);
-   ame.SetNumModes(nev);
-   ame.SetPreconditioner(ams);
-   ame.SetMaxIter(100);
-   ame.SetTol(1e-8);
-   ame.SetPrintLevel(1);
-   ame.SetMassMatrix(*M);
-   ame.SetOperator(*A);
-   ame.Solve();
-
-   x = ame.GetEigenvector(mode);
-
-   delete A;
-   delete M;
-}
-
-/**
-   Solves the eigenvalue problem -Div(Grad x) = lambda x with homogeneous
-   Neumann boundary conditions on the boundary of the domain. Returns mode
-   number "mode" (counting from zero) in the ParGridFunction "x_l2". Note that
-   mode 0 is a constant field so higher mode numbers are often more
-   interesting. The eigenmode is solved using continuous H1 basis of the
-   appropriate order and then projected onto the L2 basis and returned.
-*/
-void PseudoScalarWaveGuide(int mode, ParGridFunction &x_l2)
-{
-   int nev = std::max(mode + 2, 5);
-   int seed = 75;
-
-   ParFiniteElementSpace &fespace_l2 = *x_l2.ParFESpace();
-   ParMesh &pmesh = *fespace_l2.GetParMesh();
-   int order_l2 = fespace_l2.FEColl()->GetOrder();
-
-   H1_FECollection fec(order_l2+1, pmesh.Dimension());
-   ParFiniteElementSpace fespace(&pmesh, &fec);
-   ParGridFunction x(&fespace);
-   x = 0.0;
-
-   GridFunctionCoefficient xCoef(&x);
-
-   if (mode == 0)
-   {
-      x = 1.0;
-      x_l2.ProjectCoefficient(xCoef);
-      return;
-   }
-
-   ParBilinearForm a(&fespace);
-   a.AddDomainIntegrator(new DiffusionIntegrator);
-   a.AddDomainIntegrator(new MassIntegrator); // Shift eigenvalues by 1
-   a.Assemble();
-   a.Finalize();
-
-   ParBilinearForm m(&fespace);
-   m.AddDomainIntegrator(new MassIntegrator);
-   m.Assemble();
-   m.Finalize();
-
-   HypreParMatrix *A = a.ParallelAssemble();
-   HypreParMatrix *M = m.ParallelAssemble();
-
-   HypreBoomerAMG amg(*A);
-   amg.SetPrintLevel(0);
-
-   HypreLOBPCG lobpcg(MPI_COMM_WORLD);
-   lobpcg.SetNumModes(nev);
-   lobpcg.SetRandomSeed(seed);
-   lobpcg.SetPreconditioner(amg);
-   lobpcg.SetMaxIter(200);
-   lobpcg.SetTol(1e-8);
-   lobpcg.SetPrecondUsageMode(1);
-   lobpcg.SetPrintLevel(1);
-   lobpcg.SetMassMatrix(*M);
-   lobpcg.SetOperator(*A);
-   lobpcg.Solve();
-
-   x = lobpcg.GetEigenvector(mode);
-
-   x_l2.ProjectCoefficient(xCoef);
-
-   delete A;
-   delete M;
-}
-
-// Compute eigenmode "mode" of either a Dirichlet or Neumann Laplacian or of a
-// Dirichlet curl curl operator based on the problem type and dimension of the
-// domain.
-void SetPortBC(int prob, int dim, int mode, ParGridFunction &port_bc)
-{
-   switch (prob)
-   {
-      case 0:
-         ScalarWaveGuide(mode, port_bc);
-         break;
-      case 1:
-         if (dim == 3)
-         {
-            VectorWaveGuide(mode, port_bc);
-         }
-         else
-         {
-            PseudoScalarWaveGuide(mode, port_bc);
-         }
-         break;
-      case 2:
-         PseudoScalarWaveGuide(mode, port_bc);
-         break;
-   }
-}
@@ -1,459 +0,0 @@
-//                                MFEM Example 36
-//
-// Compile with: make ex36
-//
-// Sample runs: ex36 -o 2
-//              ex36 -o 2 -r 4
-//
-// Description: This example code demonstrates the use of MFEM to solve the
-//              bound-constrained energy minimization problem
-//
-//                      minimize ||∇u||² subject to u ≥ ϕ in H¹₀.
-//
-//              This is known as the obstacle problem, and it is a simple
-//              mathematical model for contact mechanics.
-//
-//              In this example, the obstacle ϕ is a half-sphere centered
-//              at the origin of a circular domain Ω. After solving to a
-//              specified tolerance, the numerical solution is compared to
-//              a closed-form exact solution to assess accuracy.
-//
-//              The problem is discretized and solved using the proximal
-//              Galerkin finite element method, introduced by Keith and
-//              Surowiec [1].
-//
-//              This example highlights the ability of MFEM to deliver high-
-//              order solutions to variation inequality problems and
-//              showcases how to set up and solve nonlinear mixed methods.
-//
-// [1] Keith, B. and Surowiec, T. (2023) Proximal Galerkin: A structure-
-//     preserving finite element method for pointwise bound constraints.
-//     arXiv:2307.12444 [math.NA]
-
-#include "mfem.hpp"
-#include <fstream>
-#include <iostream>
-
-using namespace std;
-using namespace mfem;
-
-double spherical_obstacle(const Vector &pt);
-double exact_solution_obstacle(const Vector &pt);
-void exact_solution_gradient_obstacle(const Vector &pt, Vector &grad);
-
-class LogarithmGridFunctionCoefficient : public Coefficient
-{
-protected:
-   GridFunction *u; // grid function
-   Coefficient *obstacle;
-   double min_val;
-
-public:
-   LogarithmGridFunctionCoefficient(GridFunction &u_, Coefficient &obst_,
-                                    double min_val_=-36)
-      : u(&u_), obstacle(&obst_), min_val(min_val_) { }
-
-   virtual double Eval(ElementTransformation &T, const IntegrationPoint &ip);
-};
-
-class ExponentialGridFunctionCoefficient : public Coefficient
-{
-protected:
-   GridFunction *u;
-   Coefficient *obstacle;
-   double min_val;
-   double max_val;
-
-public:
-   ExponentialGridFunctionCoefficient(GridFunction &u_, Coefficient &obst_,
-                                      double min_val_=0.0, double max_val_=1e6)
-      : u(&u_), obstacle(&obst_), min_val(min_val_), max_val(max_val_) { }
-
-   virtual double Eval(ElementTransformation &T, const IntegrationPoint &ip);
-};
-
-int main(int argc, char *argv[])
-{
-   // 1. Parse command-line options.
-   int order = 1;
-   int max_it = 10;
-   int ref_levels = 3;
-   double alpha = 1.0;
-   double tol = 1e-5;
-   bool visualization = true;
-
-   OptionsParser args(argc, argv);
-   args.AddOption(&order, "-o", "--order",
-                  "Finite element order (polynomial degree).");
-   args.AddOption(&ref_levels, "-r", "--refs",
-                  "Number of h-refinements.");
-   args.AddOption(&max_it, "-mi", "--max-it",
-                  "Maximum number of iterations");
-   args.AddOption(&tol, "-tol", "--tol",
-                  "Stopping criteria based on the difference between"
-                  "successive solution updates");
-   args.AddOption(&alpha, "-step", "--step",
-                  "Step size alpha");
-   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
-                  "--no-visualization",
-                  "Enable or disable GLVis visualization.");
-   args.Parse();
-   if (!args.Good())
-   {
-      args.PrintUsage(cout);
-      return 1;
-   }
-   args.PrintOptions(cout);
-
-   // 2. Read the mesh from the mesh file.
-   const char *mesh_file = "../data/disc-nurbs.mesh";
-   Mesh mesh(mesh_file, 1, 1);
-   int dim = mesh.Dimension();
-
-   // 3. Postprocess the mesh.
-   // 3A. Refine the mesh to increase the resolution.
-   for (int l = 0; l < ref_levels; l++)
-   {
-      mesh.UniformRefinement();
-   }
-
-   // 3B. Interpolate the geometry after refinement to control geometry error.
-   // NOTE: Minimum second-order interpolation is used to improve the accuracy.
-   int curvature_order = max(order,2);
-   mesh.SetCurvature(curvature_order);
-
-   // 3C. Rescale the domain to a unit circle (radius = 1).
-   GridFunction *nodes = mesh.GetNodes();
-   double scale = 2*sqrt(2);
-   *nodes /= scale;
-
-   // 4. Define the necessary finite element spaces on the mesh.
-   H1_FECollection H1fec(order+1, dim);
-   FiniteElementSpace H1fes(&mesh, &H1fec);
-
-   L2_FECollection L2fec(order-1, dim);
-   FiniteElementSpace L2fes(&mesh, &L2fec);
-
-   cout << "Number of H1 finite element unknowns: "
-        << H1fes.GetTrueVSize() << endl;
-   cout << "Number of L2 finite element unknowns: "
-        << L2fes.GetTrueVSize() << endl;
-
-   Array<int> offsets(3);
-   offsets[0] = 0;
-   offsets[1] = H1fes.GetVSize();
-   offsets[2] = L2fes.GetVSize();
-   offsets.PartialSum();
-
-   BlockVector x(offsets), rhs(offsets);
-   x = 0.0; rhs = 0.0;
-
-   // 5. Determine the list of true (i.e. conforming) essential boundary dofs.
-   Array<int> ess_bdr;
-   if (mesh.bdr_attributes.Size())
-   {
-      ess_bdr.SetSize(mesh.bdr_attributes.Max());
-      ess_bdr = 1;
-   }
-
-   // 6. Define an initial guess for the solution.
-   auto IC_func = [](const Vector &x)
-   {
-      double r0 = 1.0;
-      double rr = 0.0;
-      for (int i=0; i<x.Size(); i++)
-      {
-         rr += x(i)*x(i);
-      }
-      return r0*r0 - rr;
-   };
-   ConstantCoefficient one(1.0);
-   ConstantCoefficient zero(0.0);
-
-   // 7. Define the solution vectors as a finite element grid functions
-   //    corresponding to the fespaces.
-   GridFunction u_gf, delta_psi_gf;
-
-   u_gf.MakeRef(&H1fes,x,offsets[0]);
-   delta_psi_gf.MakeRef(&L2fes,x,offsets[1]);
-   delta_psi_gf = 0.0;
-
-   GridFunction u_old_gf(&H1fes);
-   GridFunction psi_old_gf(&L2fes);
-   GridFunction psi_gf(&L2fes);
-   u_old_gf = 0.0;
-   psi_old_gf = 0.0;
-
-   // 8. Define the function coefficients for the solution and use them to
-   //    initialize the initial guess
-   FunctionCoefficient exact_coef(exact_solution_obstacle);
-   VectorFunctionCoefficient exact_grad_coef(dim,exact_solution_gradient_obstacle);
-   FunctionCoefficient IC_coef(IC_func);
-   ConstantCoefficient f(0.0);
-   FunctionCoefficient obstacle(spherical_obstacle);
-   u_gf.ProjectCoefficient(IC_coef);
-   u_old_gf = u_gf;
-
-   // 9. Initialize the slack variable ψₕ = ln(uₕ)
-   LogarithmGridFunctionCoefficient ln_u(u_gf, obstacle);
-   psi_gf.ProjectCoefficient(ln_u);
-   psi_old_gf = psi_gf;
-
-   char vishost[] = "localhost";
-   int  visport   = 19916;
-   socketstream sol_sock;
-   if (visualization)
-   {
-      sol_sock.open(vishost,visport);
-      sol_sock.precision(8);
-   }
-
-   // 10. Iterate
-   int k;
-   int total_iterations = 0;
-   double increment_u = 0.1;
-   for (k = 0; k < max_it; k++)
-   {
-      GridFunction u_tmp(&H1fes);
-      u_tmp = u_old_gf;
-
-      mfem::out << "\nOUTER ITERATION " << k+1 << endl;
-
-      int j;
-      for ( j = 0; j < 10; j++)
-      {
-         total_iterations++;
-
-         ConstantCoefficient alpha_cf(alpha);
-
-         LinearForm b0,b1;
-         b0.Update(&H1fes,rhs.GetBlock(0),0);
-         b1.Update(&L2fes,rhs.GetBlock(1),0);
-
-         ExponentialGridFunctionCoefficient exp_psi(psi_gf, zero);
-         ProductCoefficient neg_exp_psi(-1.0,exp_psi);
-         GradientGridFunctionCoefficient grad_u_old(&u_old_gf);
-         ProductCoefficient alpha_f(alpha, f);
-         GridFunctionCoefficient psi_cf(&psi_gf);
-         GridFunctionCoefficient psi_old_cf(&psi_old_gf);
-         SumCoefficient psi_old_minus_psi(psi_old_cf, psi_cf, 1.0, -1.0);
-
-         b0.AddDomainIntegrator(new DomainLFIntegrator(alpha_f));
-         b0.AddDomainIntegrator(new DomainLFIntegrator(psi_old_minus_psi));
-         b0.Assemble();
-
-         b1.AddDomainIntegrator(new DomainLFIntegrator(exp_psi));
-         b1.AddDomainIntegrator(new DomainLFIntegrator(obstacle));
-         b1.Assemble();
-
-         BilinearForm a00(&H1fes);
-         a00.SetDiagonalPolicy(mfem::Operator::DIAG_ONE);
-         a00.AddDomainIntegrator(new DiffusionIntegrator(alpha_cf));
-         a00.Assemble();
-         a00.EliminateEssentialBC(ess_bdr,x.GetBlock(0),rhs.GetBlock(0),
-                                  mfem::Operator::DIAG_ONE);
-         a00.Finalize();
-         SparseMatrix &A00 = a00.SpMat();
-
-         MixedBilinearForm a10(&H1fes,&L2fes);
-         a10.AddDomainIntegrator(new MixedScalarMassIntegrator());
-         a10.Assemble();
-         a10.EliminateTrialDofs(ess_bdr, x.GetBlock(0), rhs.GetBlock(1));
-         a10.Finalize();
-         SparseMatrix &A10 = a10.SpMat();
-
-         SparseMatrix *A01 = Transpose(A10);
-
-         BilinearForm a11(&L2fes);
-         a11.AddDomainIntegrator(new MassIntegrator(neg_exp_psi));
-         // NOTE: Shift the spectrum of the Hessian matrix for additional
-         //       stability (Quasi-Newton).
-         ConstantCoefficient eps_cf(-1e-6);
-         if (order == 1)
-         {
-            // NOTE: ∇ₕuₕ = 0 for constant functions.
-            //       Therefore, we use the mass matrix to shift the spectrum
-            a11.AddDomainIntegrator(new MassIntegrator(eps_cf));
-         }
-         else
-         {
-            a11.AddDomainIntegrator(new DiffusionIntegrator(eps_cf));
-         }
-         a11.Assemble();
-         a11.Finalize();
-         SparseMatrix &A11 = a11.SpMat();
-
-         BlockOperator A(offsets);
-         A.SetBlock(0,0,&A00);
-         A.SetBlock(1,0,&A10);
-         A.SetBlock(0,1,A01);
-         A.SetBlock(1,1,&A11);
-
-         BlockDiagonalPreconditioner prec(offsets);
-         prec.SetDiagonalBlock(0,new GSSmoother(A00));
-         prec.SetDiagonalBlock(1,new GSSmoother(A11));
-         prec.owns_blocks = 1;
-
-         GMRES(A,prec,rhs,x,0,10000,500,1e-12,0.0);
-
-         u_gf.MakeRef(&H1fes, x.GetBlock(0), 0);
-         delta_psi_gf.MakeRef(&L2fes, x.GetBlock(1), 0);
-
-         u_tmp -= u_gf;
-         double Newton_update_size = u_tmp.ComputeL2Error(zero);
-         u_tmp = u_gf;
-
-         double gamma = 1.0;
-         delta_psi_gf *= gamma;
-         psi_gf += delta_psi_gf;
-
-         if (visualization)
-         {
-            sol_sock << "solution\n" << mesh << u_gf << "window_title 'Discrete solution'"
-                     << flush;
-            mfem::out << "Newton_update_size = " << Newton_update_size << endl;
-         }
-
-         delete A01;
-
-         if (Newton_update_size < increment_u)
-         {
-            break;
-         }
-      }
-
-      u_tmp = u_gf;
-      u_tmp -= u_old_gf;
-      increment_u = u_tmp.ComputeL2Error(zero);
-
-      mfem::out << "Number of Newton iterations = " << j+1 << endl;
-      mfem::out << "Increment (|| uₕ - uₕ_prvs||) = " << increment_u << endl;
-
-      u_old_gf = u_gf;
-      psi_old_gf = psi_gf;
-
-      if (increment_u < tol || k == max_it-1)
-      {
-         break;
-      }
-
-      double H1_error = u_gf.ComputeH1Error(&exact_coef,&exact_grad_coef);
-      mfem::out << "H1-error  (|| u - uₕᵏ||)       = " << H1_error << endl;
-
-   }
-
-   mfem::out << "\n Outer iterations: " << k+1
-             << "\n Total iterations: " << total_iterations
-             << "\n Total dofs:       " << H1fes.GetTrueVSize() + L2fes.GetTrueVSize()
-             << endl;
-
-   // 11. Exact solution.
-   if (visualization)
-   {
-      socketstream err_sock(vishost, visport);
-      err_sock.precision(8);
-
-      GridFunction error_gf(&H1fes);
-      error_gf.ProjectCoefficient(exact_coef);
-      error_gf -= u_gf;
-
-      err_sock << "solution\n" << mesh << error_gf << "window_title 'Error'"  <<
-               flush;
-   }
-
-   {
-      double L2_error = u_gf.ComputeL2Error(exact_coef);
-      double H1_error = u_gf.ComputeH1Error(&exact_coef,&exact_grad_coef);
-
-      ExponentialGridFunctionCoefficient u_alt_cf(psi_gf,obstacle);
-      GridFunction u_alt_gf(&L2fes);
-      u_alt_gf.ProjectCoefficient(u_alt_cf);
-      double L2_error_alt = u_alt_gf.ComputeL2Error(exact_coef);
-
-      mfem::out << "\n Final L2-error (|| u - uₕ||)          = " << L2_error <<
-                endl;
-      mfem::out << " Final H1-error (|| u - uₕ||)          = " << H1_error << endl;
-      mfem::out << " Final L2-error (|| u - ϕ - exp(ψₕ)||) = " << L2_error_alt <<
-                endl;
-   }
-
-   return 0;
-}
-
-double LogarithmGridFunctionCoefficient::Eval(ElementTransformation &T,
-                                              const IntegrationPoint &ip)
-{
-   MFEM_ASSERT(u != NULL, "grid function is not set");
-
-   double val = u->GetValue(T, ip) - obstacle->Eval(T, ip);
-   return max(min_val, log(val));
-}
-
-double ExponentialGridFunctionCoefficient::Eval(ElementTransformation &T,
-                                                const IntegrationPoint &ip)
-{
-   MFEM_ASSERT(u != NULL, "grid function is not set");
-
-   double val = u->GetValue(T, ip);
-   return min(max_val, max(min_val, exp(val) + obstacle->Eval(T, ip)));
-}
-
-double spherical_obstacle(const Vector &pt)
-{
-   double x = pt(0), y = pt(1);
-   double r = sqrt(x*x + y*y);
-   double r0 = 0.5;
-   double beta = 0.9;
-
-   double b = r0*beta;
-   double tmp = sqrt(r0*r0 - b*b);
-   double B = tmp + b*b/tmp;
-   double C = -b/tmp;
-
-   if (r > b)
-   {
-      return B + r * C;
-   }
-   else
-   {
-      return sqrt(r0*r0 - r*r);
-   }
-}
-
-double exact_solution_obstacle(const Vector &pt)
-{
-   double x = pt(0), y = pt(1);
-   double r = sqrt(x*x + y*y);
-   double r0 = 0.5;
-   double a =  0.348982574111686;
-   double A = -0.340129705945858;
-
-   if (r > a)
-   {
-      return A * log(r);
-   }
-   else
-   {
-      return sqrt(r0*r0-r*r);
-   }
-}
-
-void exact_solution_gradient_obstacle(const Vector &pt, Vector &grad)
-{
-   double x = pt(0), y = pt(1);
-   double r = sqrt(x*x + y*y);
-   double r0 = 0.5;
-   double a =  0.348982574111686;
-   double A = -0.340129705945858;
-
-   if (r > a)
-   {
-      grad(0) =  A * x / (r*r);
-      grad(1) =  A * y / (r*r);
-   }
-   else
-   {
-      grad(0) = - x / sqrt( r0*r0 - r*r );
-      grad(1) = - y / sqrt( r0*r0 - r*r );
-   }
-}
@@ -1,523 +0,0 @@
-//                       MFEM Example 36 - Parallel Version
-//
-// Compile with: make ex36p
-//
-// Sample runs: mpirun -np 4 ex36p -o 2
-//              mpirun -np 4 ex36p -o 2 -r 4
-//
-// Description: This example code demonstrates the use of MFEM to solve the
-//              bound-constrained energy minimization problem
-//
-//                      minimize ||∇u||² subject to u ≥ ϕ in H¹₀.
-//
-//              This is known as the obstacle problem, and it is a simple
-//              mathematical model for contact mechanics.
-//
-//              In this example, the obstacle ϕ is a half-sphere centered
-//              at the origin of a circular domain Ω. After solving to a
-//              specified tolerance, the numerical solution is compared to
-//              a closed-form exact solution to assess accuracy.
-//
-//              The problem is discretized and solved using the proximal
-//              Galerkin finite element method, introduced by Keith and
-//              Surowiec [1].
-//
-//              This example highlights the ability of MFEM to deliver high-
-//              order solutions to variation inequality problems and
-//              showcases how to set up and solve nonlinear mixed methods.
-//
-// [1] Keith, B. and Surowiec, T. (2023) Proximal Galerkin: A structure-
-//     preserving finite element method for pointwise bound constraints.
-//     arXiv:2307.12444 [math.NA]
-
-#include "mfem.hpp"
-#include <fstream>
-#include <iostream>
-
-using namespace std;
-using namespace mfem;
-
-double spherical_obstacle(const Vector &pt);
-double exact_solution_obstacle(const Vector &pt);
-void exact_solution_gradient_obstacle(const Vector &pt, Vector &grad);
-
-class LogarithmGridFunctionCoefficient : public Coefficient
-{
-protected:
-   GridFunction *u; // grid function
-   Coefficient *obstacle;
-   double min_val;
-
-public:
-   LogarithmGridFunctionCoefficient(GridFunction &u_, Coefficient &obst_,
-                                    double min_val_=-36)
-      : u(&u_), obstacle(&obst_), min_val(min_val_) { }
-
-   virtual double Eval(ElementTransformation &T, const IntegrationPoint &ip);
-};
-
-class ExponentialGridFunctionCoefficient : public Coefficient
-{
-protected:
-   GridFunction *u;
-   Coefficient *obstacle;
-   double min_val;
-   double max_val;
-
-public:
-   ExponentialGridFunctionCoefficient(GridFunction &u_, Coefficient &obst_,
-                                      double min_val_=0.0, double max_val_=1e6)
-      : u(&u_), obstacle(&obst_), min_val(min_val_), max_val(max_val_) { }
-
-   virtual double Eval(ElementTransformation &T, const IntegrationPoint &ip);
-};
-
-int main(int argc, char *argv[])
-{
-   // 0. Initialize MPI and HYPRE.
-   Mpi::Init();
-   int num_procs = Mpi::WorldSize();
-   int myid = Mpi::WorldRank();
-   Hypre::Init();
-
-   // 1. Parse command-line options.
-   int order = 1;
-   int max_it = 10;
-   int ref_levels = 3;
-   double alpha = 1.0;
-   double tol = 1e-5;
-   bool visualization = true;
-
-   OptionsParser args(argc, argv);
-   args.AddOption(&order, "-o", "--order",
-                  "Finite element order (polynomial degree).");
-   args.AddOption(&ref_levels, "-r", "--refs",
-                  "Number of h-refinements.");
-   args.AddOption(&max_it, "-mi", "--max-it",
-                  "Maximum number of iterations");
-   args.AddOption(&tol, "-tol", "--tol",
-                  "Stopping criteria based on the difference between"
-                  "successive solution updates");
-   args.AddOption(&alpha, "-step", "--step",
-                  "Step size alpha");
-   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
-                  "--no-visualization",
-                  "Enable or disable GLVis visualization.");
-   args.Parse();
-   if (!args.Good())
-   {
-      if (myid == 0)
-      {
-         args.PrintUsage(cout);
-      }
-      return 1;
-   }
-   if (myid == 0)
-   {
-      args.PrintOptions(cout);
-   }
-
-   // 2. Read the mesh from the mesh file.
-   const char *mesh_file = "../data/disc-nurbs.mesh";
-   Mesh mesh(mesh_file, 1, 1);
-   int dim = mesh.Dimension();
-
-   // 3. Postprocess the mesh.
-   // 3A. Refine the mesh to increase the resolution.
-   for (int l = 0; l < ref_levels; l++)
-   {
-      mesh.UniformRefinement();
-   }
-
-   // 3B. Interpolate the geometry after refinement to control geometry error.
-   // NOTE: Minimum second-order interpolation is used to improve the accuracy.
-   int curvature_order = max(order,2);
-   mesh.SetCurvature(curvature_order);
-
-   // 3C. Rescale the domain to a unit circle (radius = 1).
-   GridFunction *nodes = mesh.GetNodes();
-   double scale = 2*sqrt(2);
-   *nodes /= scale;
-
-   ParMesh pmesh(MPI_COMM_WORLD, mesh);
-   mesh.Clear();
-
-   // 4. Define the necessary finite element spaces on the mesh.
-   H1_FECollection H1fec(order+1, dim);
-   ParFiniteElementSpace H1fes(&pmesh, &H1fec);
-
-   L2_FECollection L2fec(order-1, dim);
-   ParFiniteElementSpace L2fes(&pmesh, &L2fec);
-
-   int num_dofs_H1 = H1fes.GetTrueVSize();
-   MPI_Allreduce(MPI_IN_PLACE, &num_dofs_H1, 1, MPI_INT, MPI_SUM, MPI_COMM_WORLD);
-   int num_dofs_L2 = L2fes.GetTrueVSize();
-   MPI_Allreduce(MPI_IN_PLACE, &num_dofs_L2, 1, MPI_INT, MPI_SUM, MPI_COMM_WORLD);
-   if (myid == 0)
-   {
-      cout << "Number of H1 finite element unknowns: "
-           << num_dofs_H1 << endl;
-      cout << "Number of L2 finite element unknowns: "
-           << num_dofs_L2 << endl;
-   }
-
-   Array<int> offsets(3);
-   offsets[0] = 0;
-   offsets[1] = H1fes.GetVSize();
-   offsets[2] = L2fes.GetVSize();
-   offsets.PartialSum();
-
-   Array<int> toffsets(3);
-   toffsets[0] = 0;
-   toffsets[1] = H1fes.GetTrueVSize();
-   toffsets[2] = L2fes.GetTrueVSize();
-   toffsets.PartialSum();
-
-   BlockVector x(offsets), rhs(offsets);
-   x = 0.0; rhs = 0.0;
-
-   BlockVector tx(toffsets), trhs(toffsets);
-   tx = 0.0; trhs = 0.0;
-
-   // 5. Determine the list of true (i.e. conforming) essential boundary dofs.
-   Array<int> empty;
-   Array<int> ess_tdof_list;
-   if (pmesh.bdr_attributes.Size())
-   {
-      Array<int> ess_bdr(pmesh.bdr_attributes.Max());
-      ess_bdr = 1;
-      H1fes.GetEssentialTrueDofs(ess_bdr, ess_tdof_list);
-   }
-
-   // 6. Define an initial guess for the solution.
-   auto IC_func = [](const Vector &x)
-   {
-      double r0 = 1.0;
-      double rr = 0.0;
-      for (int i=0; i<x.Size(); i++)
-      {
-         rr += x(i)*x(i);
-      }
-      return r0*r0 - rr;
-   };
-   ConstantCoefficient one(1.0);
-   ConstantCoefficient zero(0.0);
-
-   // 7. Define the solution vectors as a finite element grid functions
-   //    corresponding to the fespaces.
-   ParGridFunction u_gf, delta_psi_gf;
-   u_gf.MakeRef(&H1fes,x,offsets[0]);
-   delta_psi_gf.MakeRef(&L2fes,x,offsets[1]);
-   delta_psi_gf = 0.0;
-
-   ParGridFunction u_old_gf(&H1fes);
-   ParGridFunction psi_old_gf(&L2fes);
-   ParGridFunction psi_gf(&L2fes);
-   u_old_gf = 0.0;
-   psi_old_gf = 0.0;
-
-   // 8. Define the function coefficients for the solution and use them to
-   //    initialize the initial guess
-   FunctionCoefficient exact_coef(exact_solution_obstacle);
-   VectorFunctionCoefficient exact_grad_coef(dim,exact_solution_gradient_obstacle);
-   FunctionCoefficient IC_coef(IC_func);
-   ConstantCoefficient f(0.0);
-   FunctionCoefficient obstacle(spherical_obstacle);
-   u_gf.ProjectCoefficient(IC_coef);
-   u_old_gf = u_gf;
-
-   // 9. Initialize the slack variable ψₕ = ln(uₕ)
-   LogarithmGridFunctionCoefficient ln_u(u_gf, obstacle);
-   psi_gf.ProjectCoefficient(ln_u);
-   psi_old_gf = psi_gf;
-
-   char vishost[] = "localhost";
-   int  visport   = 19916;
-   socketstream sol_sock;
-   if (visualization)
-   {
-      sol_sock.open(vishost,visport);
-      sol_sock.precision(8);
-   }
-
-   // 10. Iterate
-   int k;
-   int total_iterations = 0;
-   double increment_u = 0.1;
-   for (k = 0; k < max_it; k++)
-   {
-      ParGridFunction u_tmp(&H1fes);
-      u_tmp = u_old_gf;
-
-      if (myid == 0)
-      {
-         mfem::out << "\nOUTER ITERATION " << k+1 << endl;
-      }
-
-      int j;
-      for ( j = 0; j < 10; j++)
-      {
-         total_iterations++;
-
-         ConstantCoefficient alpha_cf(alpha);
-
-         ParLinearForm b0,b1;
-         b0.Update(&H1fes,rhs.GetBlock(0),0);
-         b1.Update(&L2fes,rhs.GetBlock(1),0);
-
-         ExponentialGridFunctionCoefficient exp_psi(psi_gf, zero);
-         ProductCoefficient neg_exp_psi(-1.0,exp_psi);
-         GradientGridFunctionCoefficient grad_u_old(&u_old_gf);
-         ProductCoefficient alpha_f(alpha, f);
-         GridFunctionCoefficient psi_cf(&psi_gf);
-         GridFunctionCoefficient psi_old_cf(&psi_old_gf);
-         SumCoefficient psi_old_minus_psi(psi_old_cf, psi_cf, 1.0, -1.0);
-
-         b0.AddDomainIntegrator(new DomainLFIntegrator(alpha_f));
-         b0.AddDomainIntegrator(new DomainLFIntegrator(psi_old_minus_psi));
-         b0.Assemble();
-
-         b1.AddDomainIntegrator(new DomainLFIntegrator(exp_psi));
-         b1.AddDomainIntegrator(new DomainLFIntegrator(obstacle));
-         b1.Assemble();
-
-         ParBilinearForm a00(&H1fes);
-         a00.SetDiagonalPolicy(mfem::Operator::DIAG_ONE);
-         a00.AddDomainIntegrator(new DiffusionIntegrator(alpha_cf));
-         a00.Assemble();
-         HypreParMatrix A00;
-         a00.FormLinearSystem(ess_tdof_list, x.GetBlock(0), rhs.GetBlock(0),
-                              A00, tx.GetBlock(0), trhs.GetBlock(0));
-
-
-         ParMixedBilinearForm a10(&H1fes,&L2fes);
-         a10.AddDomainIntegrator(new MixedScalarMassIntegrator());
-         a10.Assemble();
-         HypreParMatrix A10;
-         a10.FormRectangularLinearSystem(ess_tdof_list, empty, x.GetBlock(0),
-                                         rhs.GetBlock(1),
-                                         A10, tx.GetBlock(0), trhs.GetBlock(1));
-
-         HypreParMatrix *A01 = A10.Transpose();
-
-         ParBilinearForm a11(&L2fes);
-         a11.AddDomainIntegrator(new MassIntegrator(neg_exp_psi));
-         // NOTE: Shift the spectrum of the Hessian matrix for additional
-         //       stability (Quasi-Newton).
-         ConstantCoefficient eps_cf(-1e-6);
-         if (order == 1)
-         {
-            // NOTE: ∇ₕuₕ = 0 for constant functions.
-            //       Therefore, we use the mass matrix to shift the spectrum
-            a11.AddDomainIntegrator(new MassIntegrator(eps_cf));
-         }
-         else
-         {
-            a11.AddDomainIntegrator(new DiffusionIntegrator(eps_cf));
-         }
-         a11.Assemble();
-         a11.Finalize();
-         HypreParMatrix A11;
-         a11.FormSystemMatrix(empty, A11);
-
-         BlockOperator A(toffsets);
-         A.SetBlock(0,0,&A00);
-         A.SetBlock(1,0,&A10);
-         A.SetBlock(0,1,A01);
-         A.SetBlock(1,1,&A11);
-
-         BlockDiagonalPreconditioner prec(toffsets);
-         HypreBoomerAMG P00(A00);
-         P00.SetPrintLevel(0);
-         HypreSmoother P11(A11);
-         prec.SetDiagonalBlock(0,&P00);
-         prec.SetDiagonalBlock(1,&P11);
-
-         GMRESSolver gmres(MPI_COMM_WORLD);
-         gmres.SetPrintLevel(-1);
-         gmres.SetRelTol(1e-8);
-         gmres.SetMaxIter(20000);
-         gmres.SetKDim(500);
-         gmres.SetOperator(A);
-         gmres.SetPreconditioner(prec);
-         gmres.Mult(trhs,tx);
-
-         u_gf.SetFromTrueDofs(tx.GetBlock(0));
-         delta_psi_gf.SetFromTrueDofs(tx.GetBlock(1));
-
-         u_tmp -= u_gf;
-         double Newton_update_size = u_tmp.ComputeL2Error(zero);
-         u_tmp = u_gf;
-
-         double gamma = 1.0;
-         delta_psi_gf *= gamma;
-         psi_gf += delta_psi_gf;
-
-         if (visualization)
-         {
-            sol_sock << "parallel " << num_procs << " " << myid << "\n";
-            sol_sock << "solution\n" << pmesh << u_gf << "window_title 'Discrete solution'"
-                     << flush;
-         }
-
-         if (myid == 0)
-         {
-            mfem::out << "Newton_update_size = " << Newton_update_size << endl;
-         }
-
-         delete A01;
-
-         if (Newton_update_size < increment_u)
-         {
-            break;
-         }
-      }
-
-      u_tmp = u_gf;
-      u_tmp -= u_old_gf;
-      increment_u = u_tmp.ComputeL2Error(zero);
-
-      if (myid == 0)
-      {
-         mfem::out << "Number of Newton iterations = " << j+1 << endl;
-         mfem::out << "Increment (|| uₕ - uₕ_prvs||) = " << increment_u << endl;
-      }
-
-      u_old_gf = u_gf;
-      psi_old_gf = psi_gf;
-
-      if (increment_u < tol || k == max_it-1)
-      {
-         break;
-      }
-
-      double H1_error = u_gf.ComputeH1Error(&exact_coef,&exact_grad_coef);
-      if (myid == 0)
-      {
-         mfem::out << "H1-error  (|| u - uₕᵏ||)       = " << H1_error << endl;
-      }
-
-   }
-
-   if (myid == 0)
-   {
-      mfem::out << "\n Outer iterations: " << k+1
-                << "\n Total iterations: " << total_iterations
-                << "\n Total dofs:       " << num_dofs_H1 + num_dofs_L2
-                << endl;
-   }
-
-   // 11. Exact solution.
-   if (visualization)
-   {
-      socketstream err_sock(vishost, visport);
-      err_sock.precision(8);
-
-      ParGridFunction error_gf(&H1fes);
-      error_gf.ProjectCoefficient(exact_coef);
-      error_gf -= u_gf;
-
-      err_sock << "parallel " << num_procs << " " << myid << "\n";
-      err_sock << "solution\n" << pmesh << error_gf << "window_title 'Error'"  <<
-               flush;
-   }
-
-   {
-      double L2_error = u_gf.ComputeL2Error(exact_coef);
-      double H1_error = u_gf.ComputeH1Error(&exact_coef,&exact_grad_coef);
-
-      ExponentialGridFunctionCoefficient u_alt_cf(psi_gf,obstacle);
-      ParGridFunction u_alt_gf(&L2fes);
-      u_alt_gf.ProjectCoefficient(u_alt_cf);
-      double L2_error_alt = u_alt_gf.ComputeL2Error(exact_coef);
-
-      if (myid == 0)
-      {
-         mfem::out << "\n Final L2-error (|| u - uₕ||)          = " << L2_error <<
-                   endl;
-         mfem::out << " Final H1-error (|| u - uₕ||)          = " << H1_error << endl;
-         mfem::out << " Final L2-error (|| u - ϕ - exp(ψₕ)||) = " << L2_error_alt <<
-                   endl;
-      }
-   }
-
-   return 0;
-}
-
-double LogarithmGridFunctionCoefficient::Eval(ElementTransformation &T,
-                                              const IntegrationPoint &ip)
-{
-   MFEM_ASSERT(u != NULL, "grid function is not set");
-
-   double val = u->GetValue(T, ip) - obstacle->Eval(T, ip);
-   return max(min_val, log(val));
-}
-
-double ExponentialGridFunctionCoefficient::Eval(ElementTransformation &T,
-                                                const IntegrationPoint &ip)
-{
-   MFEM_ASSERT(u != NULL, "grid function is not set");
-
-   double val = u->GetValue(T, ip);
-   return min(max_val, max(min_val, exp(val) + obstacle->Eval(T, ip)));
-}
-
-double spherical_obstacle(const Vector &pt)
-{
-   double x = pt(0), y = pt(1);
-   double r = sqrt(x*x + y*y);
-   double r0 = 0.5;
-   double beta = 0.9;
-
-   double b = r0*beta;
-   double tmp = sqrt(r0*r0 - b*b);
-   double B = tmp + b*b/tmp;
-   double C = -b/tmp;
-
-   if (r > b)
-   {
-      return B + r * C;
-   }
-   else
-   {
-      return sqrt(r0*r0 - r*r);
-   }
-}
-
-double exact_solution_obstacle(const Vector &pt)
-{
-   double x = pt(0), y = pt(1);
-   double r = sqrt(x*x + y*y);
-   double r0 = 0.5;
-   double a =  0.348982574111686;
-   double A = -0.340129705945858;
-
-   if (r > a)
-   {
-      return A * log(r);
-   }
-   else
-   {
-      return sqrt(r0*r0-r*r);
-   }
-}
-
-void exact_solution_gradient_obstacle(const Vector &pt, Vector &grad)
-{
-   double x = pt(0), y = pt(1);
-   double r = sqrt(x*x + y*y);
-   double r0 = 0.5;
-   double a =  0.348982574111686;
-   double A = -0.340129705945858;
-
-   if (r > a)
-   {
-      grad(0) =  A * x / (r*r);
-      grad(1) =  A * y / (r*r);
-   }
-   else
-   {
-      grad(0) = - x / sqrt( r0*r0 - r*r );
-      grad(1) = - y / sqrt( r0*r0 - r*r );
-   }
-}
@@ -1,466 +0,0 @@
-//                                MFEM Example 37
-//
-// Compile with: make ex37
-//
-// Sample runs:
-//     ex37 -alpha 10
-//     ex37 -alpha 10 -pv
-//     ex37 -lambda 0.1 -mu 0.1
-//     ex37 -o 2 -alpha 5.0 -mi 50 -vf 0.4 -ntol 1e-5
-//     ex37 -r 6 -o 1 -alpha 25.0 -epsilon 0.02 -mi 50 -ntol 1e-5
-//
-// Description: This example code demonstrates the use of MFEM to solve a
-//              density-filtered [3] topology optimization problem. The
-//              objective is to minimize the compliance
-//
-//                  minimize ∫_Ω f⋅u dx over u ∈ [H¹(Ω)]² and ρ ∈ L¹(Ω)
-//
-//                  subject to
-//
-//                    -Div(r(ρ̃)Cε(u)) = f       in Ω + BCs
-//                    -ϵ²Δρ̃ + ρ̃ = ρ             in Ω + Neumann BCs
-//                    0 ≤ ρ ≤ 1                 in Ω
-//                    ∫_Ω ρ dx = θ vol(Ω)
-//
-//              Here, r(ρ̃) = ρ₀ + ρ̃³ (1-ρ₀) is the solid isotropic material
-//              penalization (SIMP) law, C is the elasticity tensor for an
-//              isotropic linearly elastic material, ϵ > 0 is the design
-//              length scale, and 0 < θ < 1 is the volume fraction.
-//
-//              The problem is discretized and gradients are computing using
-//              finite elements [1]. The design is optimized using an entropic
-//              mirror descent algorithm introduced by Keith and Surowiec [2]
-//              that is tailored to the bound constraint 0 ≤ ρ ≤ 1.
-//
-//              This example highlights the ability of MFEM to deliver high-
-//              order solutions to inverse design problems and showcases how
-//              to set up and solve PDE-constrained optimization problems
-//              using the so-called reduced space approach.
-//
-// [1] Andreassen, E., Clausen, A., Schevenels, M., Lazarov, B. S., & Sigmund, O.
-//     (2011). Efficient topology optimization in MATLAB using 88 lines of
-//     code. Structural and Multidisciplinary Optimization, 43(1), 1-16.
-// [2] Keith, B. and Surowiec, T. (2023) Proximal Galerkin: A structure-
-//     preserving finite element method for pointwise bound constraints.
-//     arXiv:2307.12444 [math.NA]
-// [3] Lazarov, B. S., & Sigmund, O. (2011). Filters in topology optimization
-//     based on Helmholtz‐type differential equations. International Journal
-//     for Numerical Methods in Engineering, 86(6), 765-781.
-
-#include "mfem.hpp"
-#include <iostream>
-#include <fstream>
-#include "ex37.hpp"
-
-using namespace std;
-using namespace mfem;
-
-/**
- * @brief Bregman projection of ρ = sigmoid(ψ) onto the subspace
- *        ∫_Ω ρ dx = θ vol(Ω) as follows:
- *
- *        1. Compute the root of the R → R function
- *            f(c) = ∫_Ω sigmoid(ψ + c) dx - θ vol(Ω)
- *        2. Set ψ ← ψ + c.
- *
- * @param psi a GridFunction to be updated
- * @param target_volume θ vol(Ω)
- * @param tol Newton iteration tolerance
- * @param max_its Newton maximum iteration number
- * @return double Final volume, ∫_Ω sigmoid(ψ)
- */
-double proj(GridFunction &psi, double target_volume, double tol=1e-12,
-            int max_its=10)
-{
-   MappedGridFunctionCoefficient sigmoid_psi(&psi, sigmoid);
-   MappedGridFunctionCoefficient der_sigmoid_psi(&psi, der_sigmoid);
-
-   LinearForm int_sigmoid_psi(psi.FESpace());
-   int_sigmoid_psi.AddDomainIntegrator(new DomainLFIntegrator(sigmoid_psi));
-   LinearForm int_der_sigmoid_psi(psi.FESpace());
-   int_der_sigmoid_psi.AddDomainIntegrator(new DomainLFIntegrator(
-                                              der_sigmoid_psi));
-   bool done = false;
-   for (int k=0; k<max_its; k++) // Newton iteration
-   {
-      int_sigmoid_psi.Assemble(); // Recompute f(c) with updated ψ
-      const double f = int_sigmoid_psi.Sum() - target_volume;
-
-      int_der_sigmoid_psi.Assemble(); // Recompute df(c) with updated ψ
-      const double df = int_der_sigmoid_psi.Sum();
-
-      const double dc = -f/df;
-      psi += dc;
-      if (abs(dc) < tol) { done = true; break; }
-   }
-   if (!done)
-   {
-      mfem_warning("Projection reached maximum iteration without converging. "
-                   "Result may not be accurate.");
-   }
-   int_sigmoid_psi.Assemble();
-   return int_sigmoid_psi.Sum();
-}
-
-/**
- * ---------------------------------------------------------------
- *                      ALGORITHM PREAMBLE
- * ---------------------------------------------------------------
- *
- *  The Lagrangian for this problem is
- *
- *          L(u,ρ,ρ̃,w,w̃) = (f,u) - (r(ρ̃) C ε(u),ε(w)) + (f,w)
- *                       - (ϵ² ∇ρ̃,∇w̃) - (ρ̃,w̃) + (ρ,w̃)
- *
- *  where
- *
- *    r(ρ̃) = ρ₀ + ρ̃³ (1 - ρ₀)       (SIMP rule)
- *
- *    ε(u) = (∇u + ∇uᵀ)/2           (symmetric gradient)
- *
- *    C e = λtr(e)I + 2μe           (isotropic material)
- *
- *  NOTE: The Lame parameters can be computed from Young's modulus E
- *        and Poisson's ratio ν as follows:
- *
- *             λ = E ν/((1+ν)(1-2ν)),      μ = E/(2(1+ν))
- *
- * ---------------------------------------------------------------
- *
- *  Discretization choices:
- *
- *     u ∈ V ⊂ (H¹)ᵈ (order p)
- *     ψ ∈ L² (order p - 1), ρ = sigmoid(ψ)
- *     ρ̃ ∈ H¹ (order p)
- *     w ∈ V  (order p)
- *     w̃ ∈ H¹ (order p)
- *
- * ---------------------------------------------------------------
- *                          ALGORITHM
- * ---------------------------------------------------------------
- *
- *  Update ρ with projected mirror descent via the following algorithm.
- *
- *  1. Initialize ψ = inv_sigmoid(vol_fraction) so that ∫ sigmoid(ψ) = θ vol(Ω)
- *
- *  While not converged:
- *
- *     2. Solve filter equation ∂_w̃ L = 0; i.e.,
- *
- *           (ϵ² ∇ ρ̃, ∇ v ) + (ρ̃,v) = (ρ,v)   ∀ v ∈ H¹.
- *
- *     3. Solve primal problem ∂_w L = 0; i.e.,
- *
- *      (λ r(ρ̃) ∇⋅u, ∇⋅v) + (2 μ r(ρ̃) ε(u), ε(v)) = (f,v)   ∀ v ∈ V.
- *
- *     NB. The dual problem ∂_u L = 0 is the negative of the primal problem due to symmetry.
- *
- *     4. Solve for filtered gradient ∂_ρ̃ L = 0; i.e.,
- *
- *      (ϵ² ∇ w̃ , ∇ v ) + (w̃ ,v) = (-r'(ρ̃) ( λ |∇⋅u|² + 2 μ |ε(u)|²),v)   ∀ v ∈ H¹.
- *
- *     5. Project the gradient onto the discrete latent space; i.e., solve
- *
- *                         (G,v) = (w̃,v)   ∀ v ∈ L².
- *
- *     6. Bregman proximal gradient update; i.e.,
- *
- *                            ψ ← ψ - αG + c,
- *
- *     where α > 0 is a step size parameter and c ∈ R is a constant ensuring
- *
- *                     ∫_Ω sigmoid(ψ - αG + c) dx = θ vol(Ω).
- *
- *  end
- */
-
-int main(int argc, char *argv[])
-{
-   // 1. Parse command-line options.
-   int ref_levels = 5;
-   int order = 2;
-   double alpha = 1.0;
-   double epsilon = 0.01;
-   double vol_fraction = 0.5;
-   int max_it = 1e3;
-   double itol = 1e-1;
-   double ntol = 1e-4;
-   double rho_min = 1e-6;
-   double lambda = 1.0;
-   double mu = 1.0;
-   bool glvis_visualization = true;
-   bool paraview_output = false;
-
-   OptionsParser args(argc, argv);
-   args.AddOption(&ref_levels, "-r", "--refine",
-                  "Number of times to refine the mesh uniformly.");
-   args.AddOption(&order, "-o", "--order",
-                  "Order (degree) of the finite elements.");
-   args.AddOption(&alpha, "-alpha", "--alpha-step-length",
-                  "Step length for gradient descent.");
-   args.AddOption(&epsilon, "-epsilon", "--epsilon-thickness",
-                  "Length scale for ρ.");
-   args.AddOption(&max_it, "-mi", "--max-it",
-                  "Maximum number of gradient descent iterations.");
-   args.AddOption(&ntol, "-ntol", "--rel-tol",
-                  "Normalized exit tolerance.");
-   args.AddOption(&itol, "-itol", "--abs-tol",
-                  "Increment exit tolerance.");
-   args.AddOption(&vol_fraction, "-vf", "--volume-fraction",
-                  "Volume fraction for the material density.");
-   args.AddOption(&lambda, "-lambda", "--lambda",
-                  "Lamé constant λ.");
-   args.AddOption(&mu, "-mu", "--mu",
-                  "Lamé constant μ.");
-   args.AddOption(&rho_min, "-rmin", "--psi-min",
-                  "Minimum of density coefficient.");
-   args.AddOption(&glvis_visualization, "-vis", "--visualization", "-no-vis",
-                  "--no-visualization",
-                  "Enable or disable GLVis visualization.");
-   args.AddOption(&paraview_output, "-pv", "--paraview", "-no-pv",
-                  "--no-paraview",
-                  "Enable or disable ParaView output.");
-   args.Parse();
-   if (!args.Good())
-   {
-      args.PrintUsage(mfem::out);
-      return 1;
-   }
-   args.PrintOptions(mfem::out);
-
-   Mesh mesh = Mesh::MakeCartesian2D(3, 1, mfem::Element::Type::QUADRILATERAL,
-                                     true, 3.0, 1.0);
-   int dim = mesh.Dimension();
-
-   // 2. Set BCs.
-   for (int i = 0; i<mesh.GetNBE(); i++)
-   {
-      Element * be = mesh.GetBdrElement(i);
-      Array<int> vertices;
-      be->GetVertices(vertices);
-
-      double * coords1 = mesh.GetVertex(vertices[0]);
-      double * coords2 = mesh.GetVertex(vertices[1]);
-
-      Vector center(2);
-      center(0) = 0.5*(coords1[0] + coords2[0]);
-      center(1) = 0.5*(coords1[1] + coords2[1]);
-
-      if (abs(center(0) - 0.0) < 1e-10)
-      {
-         // the left edge
-         be->SetAttribute(1);
-      }
-      else
-      {
-         // all other boundaries
-         be->SetAttribute(2);
-      }
-   }
-   mesh.SetAttributes();
-
-   // 3. Refine the mesh.
-   for (int lev = 0; lev < ref_levels; lev++)
-   {
-      mesh.UniformRefinement();
-   }
-
-   // 4. Define the necessary finite element spaces on the mesh.
-   H1_FECollection state_fec(order, dim); // space for u
-   H1_FECollection filter_fec(order, dim); // space for ρ̃
-   L2_FECollection control_fec(order-1, dim,
-                               BasisType::GaussLobatto); // space for ψ
-   FiniteElementSpace state_fes(&mesh, &state_fec,dim);
-   FiniteElementSpace filter_fes(&mesh, &filter_fec);
-   FiniteElementSpace control_fes(&mesh, &control_fec);
-
-   int state_size = state_fes.GetTrueVSize();
-   int control_size = control_fes.GetTrueVSize();
-   int filter_size = filter_fes.GetTrueVSize();
-   mfem::out << "Number of state unknowns: " << state_size << std::endl;
-   mfem::out << "Number of filter unknowns: " << filter_size << std::endl;
-   mfem::out << "Number of control unknowns: " << control_size << std::endl;
-
-   // 5. Set the initial guess for ρ.
-   GridFunction u(&state_fes);
-   GridFunction psi(&control_fes);
-   GridFunction psi_old(&control_fes);
-   GridFunction rho_filter(&filter_fes);
-   u = 0.0;
-   rho_filter = vol_fraction;
-   psi = inv_sigmoid(vol_fraction);
-   psi_old = inv_sigmoid(vol_fraction);
-
-   // ρ = sigmoid(ψ)
-   MappedGridFunctionCoefficient rho(&psi, sigmoid);
-   // Interpolation of ρ = sigmoid(ψ) in control fes (for ParaView output)
-   GridFunction rho_gf(&control_fes);
-   // ρ - ρ_old = sigmoid(ψ) - sigmoid(ψ_old)
-   DiffMappedGridFunctionCoefficient succ_diff_rho(&psi, &psi_old, sigmoid);
-
-   // 6. Set-up the physics solver.
-   int maxat = mesh.bdr_attributes.Max();
-   Array<int> ess_bdr(maxat);
-   ess_bdr = 0;
-   ess_bdr[0] = 1;
-   ConstantCoefficient one(1.0);
-   ConstantCoefficient lambda_cf(lambda);
-   ConstantCoefficient mu_cf(mu);
-   LinearElasticitySolver * ElasticitySolver = new LinearElasticitySolver();
-   ElasticitySolver->SetMesh(&mesh);
-   ElasticitySolver->SetOrder(state_fec.GetOrder());
-   ElasticitySolver->SetupFEM();
-   Vector center(2); center(0) = 2.9; center(1) = 0.5;
-   Vector force(2); force(0) = 0.0; force(1) = -1.0;
-   double r = 0.05;
-   VolumeForceCoefficient vforce_cf(r,center,force);
-   ElasticitySolver->SetRHSCoefficient(&vforce_cf);
-   ElasticitySolver->SetEssentialBoundary(ess_bdr);
-
-   // 7. Set-up the filter solver.
-   ConstantCoefficient eps2_cf(epsilon*epsilon);
-   DiffusionSolver * FilterSolver = new DiffusionSolver();
-   FilterSolver->SetMesh(&mesh);
-   FilterSolver->SetOrder(filter_fec.GetOrder());
-   FilterSolver->SetDiffusionCoefficient(&eps2_cf);
-   FilterSolver->SetMassCoefficient(&one);
-   Array<int> ess_bdr_filter;
-   if (mesh.bdr_attributes.Size())
-   {
-      ess_bdr_filter.SetSize(mesh.bdr_attributes.Max());
-      ess_bdr_filter = 0;
-   }
-   FilterSolver->SetEssentialBoundary(ess_bdr_filter);
-   FilterSolver->SetupFEM();
-
-   BilinearForm mass(&control_fes);
-   mass.AddDomainIntegrator(new InverseIntegrator(new MassIntegrator(one)));
-   mass.Assemble();
-   SparseMatrix M;
-   Array<int> empty;
-   mass.FormSystemMatrix(empty,M);
-
-   // 8. Define the Lagrange multiplier and gradient functions.
-   GridFunction grad(&control_fes);
-   GridFunction w_filter(&filter_fes);
-
-   // 9. Define some tools for later.
-   ConstantCoefficient zero(0.0);
-   GridFunction onegf(&control_fes);
-   onegf = 1.0;
-   GridFunction zerogf(&control_fes);
-   zerogf = 0.0;
-   LinearForm vol_form(&control_fes);
-   vol_form.AddDomainIntegrator(new DomainLFIntegrator(one));
-   vol_form.Assemble();
-   double domain_volume = vol_form(onegf);
-   const double target_volume = domain_volume * vol_fraction;
-
-   // 10. Connect to GLVis. Prepare for VisIt output.
-   char vishost[] = "localhost";
-   int  visport   = 19916;
-   socketstream sout_r;
-   if (glvis_visualization)
-   {
-      sout_r.open(vishost, visport);
-      sout_r.precision(8);
-   }
-
-   mfem::ParaViewDataCollection paraview_dc("ex37", &mesh);
-   if (paraview_output)
-   {
-      rho_gf.ProjectCoefficient(rho);
-      paraview_dc.SetPrefixPath("ParaView");
-      paraview_dc.SetLevelsOfDetail(order);
-      paraview_dc.SetDataFormat(VTKFormat::BINARY);
-      paraview_dc.SetHighOrderOutput(true);
-      paraview_dc.SetCycle(0);
-      paraview_dc.SetTime(0.0);
-      paraview_dc.RegisterField("displacement",&u);
-      paraview_dc.RegisterField("density",&rho_gf);
-      paraview_dc.RegisterField("filtered_density",&rho_filter);
-      paraview_dc.Save();
-   }
-
-   // 11. Iterate:
-   for (int k = 1; k <= max_it; k++)
-   {
-      if (k > 1) { alpha *= ((double) k) / ((double) k-1); }
-
-      mfem::out << "\nStep = " << k << std::endl;
-
-      // Step 1 - Filter solve
-      // Solve (ϵ^2 ∇ ρ̃, ∇ v ) + (ρ̃,v) = (ρ,v)
-      FilterSolver->SetRHSCoefficient(&rho);
-      FilterSolver->Solve();
-      rho_filter = *FilterSolver->GetFEMSolution();
-
-      // Step 2 - State solve
-      // Solve (λ r(ρ̃) ∇⋅u, ∇⋅v) + (2 μ r(ρ̃) ε(u), ε(v)) = (f,v)
-      SIMPInterpolationCoefficient SIMP_cf(&rho_filter,rho_min, 1.0);
-      ProductCoefficient lambda_SIMP_cf(lambda_cf,SIMP_cf);
-      ProductCoefficient mu_SIMP_cf(mu_cf,SIMP_cf);
-      ElasticitySolver->SetLameCoefficients(&lambda_SIMP_cf,&mu_SIMP_cf);
-      ElasticitySolver->Solve();
-      u = *ElasticitySolver->GetFEMSolution();
-
-      // Step 3 - Adjoint filter solve
-      // Solve (ϵ² ∇ w̃, ∇ v) + (w̃ ,v) = (-r'(ρ̃) ( λ |∇⋅u|² + 2 μ |ε(u)|²),v)
-      StrainEnergyDensityCoefficient rhs_cf(&lambda_cf,&mu_cf,&u, &rho_filter,
-                                            rho_min);
-      FilterSolver->SetRHSCoefficient(&rhs_cf);
-      FilterSolver->Solve();
-      w_filter = *FilterSolver->GetFEMSolution();
-
-      // Step 4 - Compute gradient
-      // Solve G = M⁻¹w̃
-      GridFunctionCoefficient w_cf(&w_filter);
-      LinearForm w_rhs(&control_fes);
-      w_rhs.AddDomainIntegrator(new DomainLFIntegrator(w_cf));
-      w_rhs.Assemble();
-      M.Mult(w_rhs,grad);
-
-      // Step 5 - Update design variable ψ ← proj(ψ - αG)
-      psi.Add(-alpha, grad);
-      const double material_volume = proj(psi, target_volume);
-
-      // Compute ||ρ - ρ_old|| in control fes.
-      double norm_increment = zerogf.ComputeL1Error(succ_diff_rho);
-      double norm_reduced_gradient = norm_increment/alpha;
-      psi_old = psi;
-
-      double compliance = (*(ElasticitySolver->GetLinearForm()))(u);
-      mfem::out << "norm of the reduced gradient = " << norm_reduced_gradient <<
-                std::endl;
-      mfem::out << "norm of the increment = " << norm_increment << endl;
-      mfem::out << "compliance = " << compliance << std::endl;
-      mfem::out << "volume fraction = " << material_volume / domain_volume <<
-                std::endl;
-
-      if (glvis_visualization)
-      {
-         GridFunction r_gf(&filter_fes);
-         r_gf.ProjectCoefficient(SIMP_cf);
-         sout_r << "solution\n" << mesh << r_gf
-                << "window_title 'Design density r(ρ̃)'" << flush;
-      }
-
-      if (paraview_output)
-      {
-         rho_gf.ProjectCoefficient(rho);
-         paraview_dc.SetCycle(k);
-         paraview_dc.SetTime((double)k);
-         paraview_dc.Save();
-      }
-
-      if (norm_reduced_gradient < ntol && norm_increment < itol)
-      {
-         break;
-      }
-   }
-
-   delete ElasticitySolver;
-   delete FilterSolver;
-
-   return 0;
-}
@@ -1,748 +0,0 @@
-//                  MFEM Example 37 - Serial/Parallel Shared Code
-
-#include "mfem.hpp"
-#include <fstream>
-#include <iostream>
-#include <functional>
-
-namespace mfem
-{
-
-/// @brief Inverse sigmoid function
-double inv_sigmoid(double x)
-{
-   double tol = 1e-12;
-   x = std::min(std::max(tol,x),1.0-tol);
-   return std::log(x/(1.0-x));
-}
-
-/// @brief Sigmoid function
-double sigmoid(double x)
-{
-   if (x >= 0)
-   {
-      return 1.0/(1.0+std::exp(-x));
-   }
-   else
-   {
-      return std::exp(x)/(1.0+std::exp(x));
-   }
-}
-
-/// @brief Derivative of sigmoid function
-double der_sigmoid(double x)
-{
-   double tmp = sigmoid(-x);
-   return tmp - std::pow(tmp,2);
-}
-
-/// @brief Returns f(u(x)) where u is a scalar GridFunction and f:R → R
-class MappedGridFunctionCoefficient : public GridFunctionCoefficient
-{
-protected:
-   std::function<double(const double)> fun; // f:R → R
-public:
-   MappedGridFunctionCoefficient()
-      :GridFunctionCoefficient(),
-       fun([](double x) {return x;}) {}
-   MappedGridFunctionCoefficient(const GridFunction *gf,
-                                 std::function<double(const double)> fun_,
-                                 int comp=1)
-      :GridFunctionCoefficient(gf, comp),
-       fun(fun_) {}
-
-
-   virtual double Eval(ElementTransformation &T,
-                       const IntegrationPoint &ip)
-   {
-      return fun(GridFunctionCoefficient::Eval(T, ip));
-   }
-   void SetFunction(std::function<double(const double)> fun_) { fun = fun_; }
-};
-
-
-/// @brief Returns f(u(x)) - f(v(x)) where u, v are scalar GridFunctions and f:R → R
-class DiffMappedGridFunctionCoefficient : public GridFunctionCoefficient
-{
-protected:
-   const GridFunction *OtherGridF;
-   GridFunctionCoefficient OtherGridF_cf;
-   std::function<double(const double)> fun; // f:R → R
-public:
-   DiffMappedGridFunctionCoefficient()
-      :GridFunctionCoefficient(),
-       OtherGridF(nullptr),
-       OtherGridF_cf(),
-       fun([](double x) {return x;}) {}
-   DiffMappedGridFunctionCoefficient(const GridFunction *gf,
-                                     const GridFunction *other_gf,
-                                     std::function<double(const double)> fun_,
-                                     int comp=1)
-      :GridFunctionCoefficient(gf, comp),
-       OtherGridF(other_gf),
-       OtherGridF_cf(OtherGridF),
-       fun(fun_) {}
-
-   virtual double Eval(ElementTransformation &T,
-                       const IntegrationPoint &ip)
-   {
-      const double value1 = fun(GridFunctionCoefficient::Eval(T, ip));
-      const double value2 = fun(OtherGridF_cf.Eval(T, ip));
-      return value1 - value2;
-   }
-   void SetFunction(std::function<double(const double)> fun_) { fun = fun_; }
-};
-
-/// @brief Solid isotropic material penalization (SIMP) coefficient
-class SIMPInterpolationCoefficient : public Coefficient
-{
-protected:
-   GridFunction *rho_filter;
-   double min_val;
-   double max_val;
-   double exponent;
-
-public:
-   SIMPInterpolationCoefficient(GridFunction *rho_filter_, double min_val_= 1e-6,
-                                double max_val_ = 1.0, double exponent_ = 3)
-      : rho_filter(rho_filter_), min_val(min_val_), max_val(max_val_),
-        exponent(exponent_) { }
-
-   virtual double Eval(ElementTransformation &T, const IntegrationPoint &ip)
-   {
-      double val = rho_filter->GetValue(T, ip);
-      double coeff = min_val + pow(val,exponent)*(max_val-min_val);
-      return coeff;
-   }
-};
-
-
-/// @brief Strain energy density coefficient
-class StrainEnergyDensityCoefficient : public Coefficient
-{
-protected:
-   Coefficient * lambda=nullptr;
-   Coefficient * mu=nullptr;
-   GridFunction *u = nullptr; // displacement
-   GridFunction *rho_filter = nullptr; // filter density
-   DenseMatrix grad; // auxiliary matrix, used in Eval
-   double exponent;
-   double rho_min;
-
-public:
-   StrainEnergyDensityCoefficient(Coefficient *lambda_, Coefficient *mu_,
-                                  GridFunction * u_, GridFunction * rho_filter_, double rho_min_=1e-6,
-                                  double exponent_ = 3.0)
-      : lambda(lambda_), mu(mu_),  u(u_), rho_filter(rho_filter_),
-        exponent(exponent_), rho_min(rho_min_)
-   {
-      MFEM_ASSERT(rho_min_ >= 0.0, "rho_min must be >= 0");
-      MFEM_ASSERT(rho_min_ < 1.0,  "rho_min must be > 1");
-      MFEM_ASSERT(u, "displacement field is not set");
-      MFEM_ASSERT(rho_filter, "density field is not set");
-   }
-
-   virtual double Eval(ElementTransformation &T, const IntegrationPoint &ip)
-   {
-      double L = lambda->Eval(T, ip);
-      double M = mu->Eval(T, ip);
-      u->GetVectorGradient(T, grad);
-      double div_u = grad.Trace();
-      double density = L*div_u*div_u;
-      int dim = T.GetSpaceDim();
-      for (int i=0; i<dim; i++)
-      {
-         for (int j=0; j<dim; j++)
-         {
-            density += M*grad(i,j)*(grad(i,j)+grad(j,i));
-         }
-      }
-      double val = rho_filter->GetValue(T,ip);
-
-      return -exponent * pow(val, exponent-1.0) * (1-rho_min) * density;
-   }
-};
-
-/// @brief Volumetric force for linear elasticity
-class VolumeForceCoefficient : public VectorCoefficient
-{
-private:
-   double r;
-   Vector center;
-   Vector force;
-public:
-   VolumeForceCoefficient(double r_,Vector &  center_, Vector & force_) :
-      VectorCoefficient(center_.Size()), r(r_), center(center_), force(force_) { }
-
-   using VectorCoefficient::Eval;
-
-   virtual void Eval(Vector &V, ElementTransformation &T,
-                     const IntegrationPoint &ip)
-   {
-      Vector xx; xx.SetSize(T.GetDimension());
-      T.Transform(ip,xx);
-      for (int i=0; i<xx.Size(); i++)
-      {
-         xx[i]=xx[i]-center[i];
-      }
-
-      double cr=xx.Norml2();
-      V.SetSize(T.GetDimension());
-      if (cr <= r)
-      {
-         V = force;
-      }
-      else
-      {
-         V = 0.0;
-      }
-   }
-
-   void Set(double r_,Vector & center_, Vector & force_)
-   {
-      r=r_;
-      center = center_;
-      force = force_;
-   }
-};
-
-/**
- * @brief Class for solving Poisson's equation:
- *
- *       - ∇ ⋅(κ ∇ u) = f  in Ω
- *
- */
-class DiffusionSolver
-{
-private:
-   Mesh * mesh = nullptr;
-   int order = 1;
-   // diffusion coefficient
-   Coefficient * diffcf = nullptr;
-   // mass coefficient
-   Coefficient * masscf = nullptr;
-   Coefficient * rhscf = nullptr;
-   Coefficient * essbdr_cf = nullptr;
-   Coefficient * neumann_cf = nullptr;
-   VectorCoefficient * gradient_cf = nullptr;
-
-   // FEM solver
-   int dim;
-   FiniteElementCollection * fec = nullptr;
-   FiniteElementSpace * fes = nullptr;
-   Array<int> ess_bdr;
-   Array<int> neumann_bdr;
-   GridFunction * u = nullptr;
-   LinearForm * b = nullptr;
-   bool parallel;
-#ifdef MFEM_USE_MPI
-   ParMesh * pmesh = nullptr;
-   ParFiniteElementSpace * pfes = nullptr;
-#endif
-
-public:
-   DiffusionSolver() { }
-   DiffusionSolver(Mesh * mesh_, int order_, Coefficient * diffcf_,
-                   Coefficient * cf_);
-
-   void SetMesh(Mesh * mesh_)
-   {
-      mesh = mesh_;
-      parallel = false;
-#ifdef MFEM_USE_MPI
-      pmesh = dynamic_cast<ParMesh *>(mesh);
-      if (pmesh) { parallel = true; }
-#endif
-   }
-   void SetOrder(int order_) { order = order_ ; }
-   void SetDiffusionCoefficient(Coefficient * diffcf_) { diffcf = diffcf_; }
-   void SetMassCoefficient(Coefficient * masscf_) { masscf = masscf_; }
-   void SetRHSCoefficient(Coefficient * rhscf_) { rhscf = rhscf_; }
-   void SetEssentialBoundary(const Array<int> & ess_bdr_) { ess_bdr = ess_bdr_;};
-   void SetNeumannBoundary(const Array<int> & neumann_bdr_) { neumann_bdr = neumann_bdr_;};
-   void SetNeumannData(Coefficient * neumann_cf_) {neumann_cf = neumann_cf_;}
-   void SetEssBdrData(Coefficient * essbdr_cf_) {essbdr_cf = essbdr_cf_;}
-   void SetGradientData(VectorCoefficient * gradient_cf_) {gradient_cf = gradient_cf_;}
-
-   void ResetFEM();
-   void SetupFEM();
-
-   void Solve();
-   GridFunction * GetFEMSolution();
-   LinearForm * GetLinearForm() {return b;}
-#ifdef MFEM_USE_MPI
-   ParGridFunction * GetParFEMSolution();
-   ParLinearForm * GetParLinearForm()
-   {
-      if (parallel)
-      {
-         return dynamic_cast<ParLinearForm *>(b);
-      }
-      else
-      {
-         MFEM_ABORT("Wrong code path. Call GetLinearForm");
-         return nullptr;
-      }
-   }
-#endif
-
-   ~DiffusionSolver();
-
-};
-
-/**
- * @brief Class for solving linear elasticity:
- *
- *        -∇ ⋅ σ(u) = f  in Ω  + BCs
- *
- *  where
- *
- *        σ(u) = λ ∇⋅u I + μ (∇ u + ∇uᵀ)
- *
- */
-class LinearElasticitySolver
-{
-private:
-   Mesh * mesh = nullptr;
-   int order = 1;
-   Coefficient * lambda_cf = nullptr;
-   Coefficient * mu_cf = nullptr;
-   VectorCoefficient * essbdr_cf = nullptr;
-   VectorCoefficient * rhs_cf = nullptr;
-
-   // FEM solver
-   int dim;
-   FiniteElementCollection * fec = nullptr;
-   FiniteElementSpace * fes = nullptr;
-   Array<int> ess_bdr;
-   Array<int> neumann_bdr;
-   GridFunction * u = nullptr;
-   LinearForm * b = nullptr;
-   bool parallel;
-#ifdef MFEM_USE_MPI
-   ParMesh * pmesh = nullptr;
-   ParFiniteElementSpace * pfes = nullptr;
-#endif
-
-public:
-   LinearElasticitySolver() { }
-   LinearElasticitySolver(Mesh * mesh_, int order_,
-                          Coefficient * lambda_cf_, Coefficient * mu_cf_);
-
-   void SetMesh(Mesh * mesh_)
-   {
-      mesh = mesh_;
-      parallel = false;
-#ifdef MFEM_USE_MPI
-      pmesh = dynamic_cast<ParMesh *>(mesh);
-      if (pmesh) { parallel = true; }
-#endif
-   }
-   void SetOrder(int order_) { order = order_ ; }
-   void SetLameCoefficients(Coefficient * lambda_cf_, Coefficient * mu_cf_) { lambda_cf = lambda_cf_; mu_cf = mu_cf_;  }
-   void SetRHSCoefficient(VectorCoefficient * rhs_cf_) { rhs_cf = rhs_cf_; }
-   void SetEssentialBoundary(const Array<int> & ess_bdr_) { ess_bdr = ess_bdr_;};
-   void SetNeumannBoundary(const Array<int> & neumann_bdr_) { neumann_bdr = neumann_bdr_;};
-   void SetEssBdrData(VectorCoefficient * essbdr_cf_) {essbdr_cf = essbdr_cf_;}
-
-   void ResetFEM();
-   void SetupFEM();
-
-   void Solve();
-   GridFunction * GetFEMSolution();
-   LinearForm * GetLinearForm() {return b;}
-#ifdef MFEM_USE_MPI
-   ParGridFunction * GetParFEMSolution();
-   ParLinearForm * GetParLinearForm()
-   {
-      if (parallel)
-      {
-         return dynamic_cast<ParLinearForm *>(b);
-      }
-      else
-      {
-         MFEM_ABORT("Wrong code path. Call GetLinearForm");
-         return nullptr;
-      }
-   }
-#endif
-
-   ~LinearElasticitySolver();
-
-};
-
-
-// Poisson solver
-
-DiffusionSolver::DiffusionSolver(Mesh * mesh_, int order_,
-                                 Coefficient * diffcf_, Coefficient * rhscf_)
-   : mesh(mesh_), order(order_), diffcf(diffcf_), rhscf(rhscf_)
-{
-
-#ifdef MFEM_USE_MPI
-   pmesh = dynamic_cast<ParMesh *>(mesh);
-   if (pmesh) { parallel = true; }
-#endif
-
-   SetupFEM();
-}
-
-void DiffusionSolver::SetupFEM()
-{
-   dim = mesh->Dimension();
-   fec = new H1_FECollection(order, dim);
-
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      pfes = new ParFiniteElementSpace(pmesh, fec);
-      u = new ParGridFunction(pfes);
-      b = new ParLinearForm(pfes);
-   }
-   else
-   {
-      fes = new FiniteElementSpace(mesh, fec);
-      u = new GridFunction(fes);
-      b = new LinearForm(fes);
-   }
-#else
-   fes = new FiniteElementSpace(mesh, fec);
-   u = new GridFunction(fes);
-   b = new LinearForm(fes);
-#endif
-   *u=0.0;
-
-   if (!ess_bdr.Size())
-   {
-      if (mesh->bdr_attributes.Size())
-      {
-         ess_bdr.SetSize(mesh->bdr_attributes.Max());
-         ess_bdr = 1;
-      }
-   }
-}
-
-void DiffusionSolver::Solve()
-{
-   OperatorPtr A;
-   Vector B, X;
-   Array<int> ess_tdof_list;
-
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      pfes->GetEssentialTrueDofs(ess_bdr,ess_tdof_list);
-   }
-   else
-   {
-      fes->GetEssentialTrueDofs(ess_bdr,ess_tdof_list);
-   }
-#else
-   fes->GetEssentialTrueDofs(ess_bdr,ess_tdof_list);
-#endif
-   *u=0.0;
-   if (b)
-   {
-      delete b;
-#ifdef MFEM_USE_MPI
-      if (parallel)
-      {
-         b = new ParLinearForm(pfes);
-      }
-      else
-      {
-         b = new LinearForm(fes);
-      }
-#else
-      b = new LinearForm(fes);
-#endif
-   }
-   if (rhscf)
-   {
-      b->AddDomainIntegrator(new DomainLFIntegrator(*rhscf));
-   }
-   if (neumann_cf)
-   {
-      MFEM_VERIFY(neumann_bdr.Size(), "neumann_bdr attributes not provided");
-      b->AddBoundaryIntegrator(new BoundaryLFIntegrator(*neumann_cf),neumann_bdr);
-   }
-   else if (gradient_cf)
-   {
-      MFEM_VERIFY(neumann_bdr.Size(), "neumann_bdr attributes not provided");
-      b->AddBoundaryIntegrator(new BoundaryNormalLFIntegrator(*gradient_cf),
-                               neumann_bdr);
-   }
-
-   b->Assemble();
-
-   BilinearForm * a = nullptr;
-
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      a = new ParBilinearForm(pfes);
-   }
-   else
-   {
-      a = new BilinearForm(fes);
-   }
-#else
-   a = new BilinearForm(fes);
-#endif
-   a->AddDomainIntegrator(new DiffusionIntegrator(*diffcf));
-   if (masscf)
-   {
-      a->AddDomainIntegrator(new MassIntegrator(*masscf));
-   }
-   a->Assemble();
-   if (essbdr_cf)
-   {
-      u->ProjectBdrCoefficient(*essbdr_cf,ess_bdr);
-   }
-   a->FormLinearSystem(ess_tdof_list, *u, *b, A, X, B);
-
-   CGSolver * cg = nullptr;
-   Solver * M = nullptr;
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      M = new HypreBoomerAMG;
-      dynamic_cast<HypreBoomerAMG*>(M)->SetPrintLevel(0);
-      cg = new CGSolver(pmesh->GetComm());
-   }
-   else
-   {
-      M = new GSSmoother((SparseMatrix&)(*A));
-      cg = new CGSolver;
-   }
-#else
-   M = new GSSmoother((SparseMatrix&)(*A));
-   cg = new CGSolver;
-#endif
-   cg->SetRelTol(1e-12);
-   cg->SetMaxIter(10000);
-   cg->SetPrintLevel(0);
-   cg->SetPreconditioner(*M);
-   cg->SetOperator(*A);
-   cg->Mult(B, X);
-   delete M;
-   delete cg;
-   a->RecoverFEMSolution(X, *b, *u);
-   delete a;
-}
-
-GridFunction * DiffusionSolver::GetFEMSolution()
-{
-   return u;
-}
-
-#ifdef MFEM_USE_MPI
-ParGridFunction * DiffusionSolver::GetParFEMSolution()
-{
-   if (parallel)
-   {
-      return dynamic_cast<ParGridFunction*>(u);
-   }
-   else
-   {
-      MFEM_ABORT("Wrong code path. Call GetFEMSolution");
-      return nullptr;
-   }
-}
-#endif
-
-DiffusionSolver::~DiffusionSolver()
-{
-   delete u; u = nullptr;
-   delete fes; fes = nullptr;
-#ifdef MFEM_USE_MPI
-   delete pfes; pfes=nullptr;
-#endif
-   delete fec; fec = nullptr;
-   delete b;
-}
-
-
-// Elasticity solver
-
-LinearElasticitySolver::LinearElasticitySolver(Mesh * mesh_, int order_,
-                                               Coefficient * lambda_cf_, Coefficient * mu_cf_)
-   : mesh(mesh_), order(order_), lambda_cf(lambda_cf_), mu_cf(mu_cf_)
-{
-#ifdef MFEM_USE_MPI
-   pmesh = dynamic_cast<ParMesh *>(mesh);
-   if (pmesh) { parallel = true; }
-#endif
-   SetupFEM();
-}
-
-void LinearElasticitySolver::SetupFEM()
-{
-   dim = mesh->Dimension();
-   fec = new H1_FECollection(order, dim,BasisType::Positive);
-
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      pfes = new ParFiniteElementSpace(pmesh, fec, dim);
-      u = new ParGridFunction(pfes);
-      b = new ParLinearForm(pfes);
-   }
-   else
-   {
-      fes = new FiniteElementSpace(mesh, fec,dim);
-      u = new GridFunction(fes);
-      b = new LinearForm(fes);
-   }
-#else
-   fes = new FiniteElementSpace(mesh, fec, dim);
-   u = new GridFunction(fes);
-   b = new LinearForm(fes);
-#endif
-   *u=0.0;
-
-   if (!ess_bdr.Size())
-   {
-      if (mesh->bdr_attributes.Size())
-      {
-         ess_bdr.SetSize(mesh->bdr_attributes.Max());
-         ess_bdr = 1;
-      }
-   }
-}
-
-void LinearElasticitySolver::Solve()
-{
-   GridFunction * x = nullptr;
-   OperatorPtr A;
-   Vector B, X;
-   Array<int> ess_tdof_list;
-
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      x = new ParGridFunction(pfes);
-      pfes->GetEssentialTrueDofs(ess_bdr,ess_tdof_list);
-   }
-   else
-   {
-      x = new GridFunction(fes);
-      fes->GetEssentialTrueDofs(ess_bdr,ess_tdof_list);
-   }
-#else
-   x = new GridFunction(fes);
-   fes->GetEssentialTrueDofs(ess_bdr,ess_tdof_list);
-#endif
-   *u=0.0;
-   if (b)
-   {
-      delete b;
-#ifdef MFEM_USE_MPI
-      if (parallel)
-      {
-         b = new ParLinearForm(pfes);
-      }
-      else
-      {
-         b = new LinearForm(fes);
-      }
-#else
-      b = new LinearForm(fes);
-#endif
-   }
-   if (rhs_cf)
-   {
-      b->AddDomainIntegrator(new VectorDomainLFIntegrator(*rhs_cf));
-   }
-
-   b->Assemble();
-
-   *x = 0.0;
-
-   BilinearForm * a = nullptr;
-
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      a = new ParBilinearForm(pfes);
-   }
-   else
-   {
-      a = new BilinearForm(fes);
-   }
-#else
-   a = new BilinearForm(fes);
-#endif
-   a->AddDomainIntegrator(new ElasticityIntegrator(*lambda_cf, *mu_cf));
-   a->Assemble();
-   if (essbdr_cf)
-   {
-      u->ProjectBdrCoefficient(*essbdr_cf,ess_bdr);
-   }
-   a->FormLinearSystem(ess_tdof_list, *x, *b, A, X, B);
-
-   CGSolver * cg = nullptr;
-   Solver * M = nullptr;
-#ifdef MFEM_USE_MPI
-   if (parallel)
-   {
-      M = new HypreBoomerAMG;
-      dynamic_cast<HypreBoomerAMG*>(M)->SetPrintLevel(0);
-      cg = new CGSolver(pmesh->GetComm());
-   }
-   else
-   {
-      M = new GSSmoother((SparseMatrix&)(*A));
-      cg = new CGSolver;
-   }
-#else
-   M = new GSSmoother((SparseMatrix&)(*A));
-   cg = new CGSolver;
-#endif
-   cg->SetRelTol(1e-10);
-   cg->SetMaxIter(10000);
-   cg->SetPrintLevel(0);
-   cg->SetPreconditioner(*M);
-   cg->SetOperator(*A);
-   cg->Mult(B, X);
-   delete M;
-   delete cg;
-   a->RecoverFEMSolution(X, *b, *x);
-   *u+=*x;
-   delete a;
-   delete x;
-}
-
-GridFunction * LinearElasticitySolver::GetFEMSolution()
-{
-   return u;
-}
-
-#ifdef MFEM_USE_MPI
-ParGridFunction * LinearElasticitySolver::GetParFEMSolution()
-{
-   if (parallel)
-   {
-      return dynamic_cast<ParGridFunction*>(u);
-   }
-   else
-   {
-      MFEM_ABORT("Wrong code path. Call GetFEMSolution");
-      return nullptr;
-   }
-}
-#endif
-
-LinearElasticitySolver::~LinearElasticitySolver()
-{
-   delete u; u = nullptr;
-   delete fes; fes = nullptr;
-#ifdef MFEM_USE_MPI
-   delete pfes; pfes=nullptr;
-#endif
-   delete fec; fec = nullptr;
-   delete b;
-}
-
-} // namespace mfem
-
@@ -1,497 +0,0 @@
-//                       MFEM Example 37 - Parallel Version
-//
-// Compile with: make ex37p
-//
-// Sample runs:
-//    mpirun -np 4 ex37p -alpha 10 -pv
-//    mpirun -np 4 ex37p -lambda 0.1 -mu 0.1
-//    mpirun -np 4 ex37p -o 2 -alpha 5.0 -mi 50 -vf 0.4 -ntol 1e-5
-//    mpirun -np 4 ex37p -r 6 -o 2 -alpha 10.0 -epsilon 0.02 -mi 50 -ntol 1e-5
-//
-// Description: This example code demonstrates the use of MFEM to solve a
-//              density-filtered [3] topology optimization problem. The
-//              objective is to minimize the compliance
-//
-//                  minimize ∫_Ω f⋅u dx over u ∈ [H¹(Ω)]² and ρ ∈ L¹(Ω)
-//
-//                  subject to
-//
-//                    -Div(r(ρ̃)Cε(u)) = f       in Ω + BCs
-//                    -ϵ²Δρ̃ + ρ̃ = ρ             in Ω + Neumann BCs
-//                    0 ≤ ρ ≤ 1                 in Ω
-//                    ∫_Ω ρ dx = θ vol(Ω)
-//
-//              Here, r(ρ̃) = ρ₀ + ρ̃³ (1-ρ₀) is the solid isotropic material
-//              penalization (SIMP) law, C is the elasticity tensor for an
-//              isotropic linearly elastic material, ϵ > 0 is the design
-//              length scale, and 0 < θ < 1 is the volume fraction.
-//
-//              The problem is discretized and gradients are computing using
-//              finite elements [1]. The design is optimized using an entropic
-//              mirror descent algorithm introduced by Keith and Surowiec [2]
-//              that is tailored to the bound constraint 0 ≤ ρ ≤ 1.
-//
-//              This example highlights the ability of MFEM to deliver high-
-//              order solutions to inverse design problems and showcases how
-//              to set up and solve PDE-constrained optimization problems
-//              using the so-called reduced space approach.
-//
-// [1] Andreassen, E., Clausen, A., Schevenels, M., Lazarov, B. S., & Sigmund, O.
-//     (2011). Efficient topology optimization in MATLAB using 88 lines of
-//     code. Structural and Multidisciplinary Optimization, 43(1), 1-16.
-// [2] Keith, B. and Surowiec, T. (2023) Proximal Galerkin: A structure-
-//     preserving finite element method for pointwise bound constraints.
-//     arXiv:2307.12444 [math.NA]
-// [3] Lazarov, B. S., & Sigmund, O. (2011). Filters in topology optimization
-//     based on Helmholtz‐type differential equations. International Journal
-//     for Numerical Methods in Engineering, 86(6), 765-781.
-
-#include "mfem.hpp"
-#include <iostream>
-#include <fstream>
-#include "ex37.hpp"
-
-using namespace std;
-using namespace mfem;
-
-/**
- * @brief Bregman projection of ρ = sigmoid(ψ) onto the subspace
- *        ∫_Ω ρ dx = θ vol(Ω) as follows:
- *
- *        1. Compute the root of the R → R function
- *            f(c) = ∫_Ω sigmoid(ψ + c) dx - θ vol(Ω)
- *        2. Set ψ ← ψ + c.
- *
- * @param psi a GridFunction to be updated
- * @param target_volume θ vol(Ω)
- * @param tol Newton iteration tolerance
- * @param max_its Newton maximum iteration number
- * @return double Final volume, ∫_Ω sigmoid(ψ)
- */
-double proj(ParGridFunction &psi, double target_volume, double tol=1e-12,
-            int max_its=10)
-{
-   MappedGridFunctionCoefficient sigmoid_psi(&psi, sigmoid);
-   MappedGridFunctionCoefficient der_sigmoid_psi(&psi, der_sigmoid);
-
-   ParLinearForm int_sigmoid_psi(psi.ParFESpace());
-   int_sigmoid_psi.AddDomainIntegrator(new DomainLFIntegrator(sigmoid_psi));
-   ParLinearForm int_der_sigmoid_psi(psi.ParFESpace());
-   int_der_sigmoid_psi.AddDomainIntegrator(new DomainLFIntegrator(
-                                              der_sigmoid_psi));
-   bool done = false;
-   for (int k=0; k<max_its; k++) // Newton iteration
-   {
-      int_sigmoid_psi.Assemble(); // Recompute f(c) with updated ψ
-      double f = int_sigmoid_psi.Sum();
-      MPI_Allreduce(MPI_IN_PLACE, &f, 1, MPI_DOUBLE, MPI_SUM, MPI_COMM_WORLD);
-      f -= target_volume;
-
-      int_der_sigmoid_psi.Assemble(); // Recompute df(c) with updated ψ
-      double df = int_der_sigmoid_psi.Sum();
-      MPI_Allreduce(MPI_IN_PLACE, &df, 1, MPI_DOUBLE, MPI_SUM, MPI_COMM_WORLD);
-
-      const double dc = -f/df;
-      psi += dc;
-      if (abs(dc) < tol) { done = true; break; }
-   }
-   if (!done)
-   {
-      mfem_warning("Projection reached maximum iteration without converging. "
-                   "Result may not be accurate.");
-   }
-   int_sigmoid_psi.Assemble();
-   double material_volume = int_sigmoid_psi.Sum();
-   MPI_Allreduce(MPI_IN_PLACE, &material_volume, 1, MPI_DOUBLE, MPI_SUM,
-                 MPI_COMM_WORLD);
-   return material_volume;
-}
-
-/**
- * ---------------------------------------------------------------
- *                      ALGORITHM PREAMBLE
- * ---------------------------------------------------------------
- *
- *  The Lagrangian for this problem is
- *
- *          L(u,ρ,ρ̃,w,w̃) = (f,u) - (r(ρ̃) C ε(u),ε(w)) + (f,w)
- *                       - (ϵ² ∇ρ̃,∇w̃) - (ρ̃,w̃) + (ρ,w̃)
- *
- *  where
- *
- *    r(ρ̃) = ρ₀ + ρ̃³ (1 - ρ₀)       (SIMP rule)
- *
- *    ε(u) = (∇u + ∇uᵀ)/2           (symmetric gradient)
- *
- *    C e = λtr(e)I + 2μe           (isotropic material)
- *
- *  NOTE: The Lame parameters can be computed from Young's modulus E
- *        and Poisson's ratio ν as follows:
- *
- *             λ = E ν/((1+ν)(1-2ν)),      μ = E/(2(1+ν))
- *
- * ---------------------------------------------------------------
- *
- *  Discretization choices:
- *
- *     u ∈ V ⊂ (H¹)ᵈ (order p)
- *     ψ ∈ L² (order p - 1), ρ = sigmoid(ψ)
- *     ρ̃ ∈ H¹ (order p)
- *     w ∈ V  (order p)
- *     w̃ ∈ H¹ (order p)
- *
- * ---------------------------------------------------------------
- *                          ALGORITHM
- * ---------------------------------------------------------------
- *
- *  Update ρ with projected mirror descent via the following algorithm.
- *
- *  1. Initialize ψ = inv_sigmoid(vol_fraction) so that ∫ sigmoid(ψ) = θ vol(Ω)
- *
- *  While not converged:
- *
- *     2. Solve filter equation ∂_w̃ L = 0; i.e.,
- *
- *           (ϵ² ∇ ρ̃, ∇ v ) + (ρ̃,v) = (ρ,v)   ∀ v ∈ H¹.
- *
- *     3. Solve primal problem ∂_w L = 0; i.e.,
- *
- *      (λ r(ρ̃) ∇⋅u, ∇⋅v) + (2 μ r(ρ̃) ε(u), ε(v)) = (f,v)   ∀ v ∈ V.
- *
- *     NB. The dual problem ∂_u L = 0 is the negative of the primal problem due to symmetry.
- *
- *     4. Solve for filtered gradient ∂_ρ̃ L = 0; i.e.,
- *
- *      (ϵ² ∇ w̃ , ∇ v ) + (w̃ ,v) = (-r'(ρ̃) ( λ |∇⋅u|² + 2 μ |ε(u)|²),v)   ∀ v ∈ H¹.
- *
- *     5. Project the gradient onto the discrete latent space; i.e., solve
- *
- *                         (G,v) = (w̃,v)   ∀ v ∈ L².
- *
- *     6. Bregman proximal gradient update; i.e.,
- *
- *                            ψ ← ψ - αG + c,
- *
- *     where α > 0 is a step size parameter and c ∈ R is a constant ensuring
- *
- *                     ∫_Ω sigmoid(ψ - αG + c) dx = θ vol(Ω).
- *
- *  end
- */
-
-int main(int argc, char *argv[])
-{
-   // 0. Initialize MPI and HYPRE.
-   Mpi::Init();
-   int num_procs = Mpi::WorldSize();
-   int myid = Mpi::WorldRank();
-   Hypre::Init();
-
-   // 1. Parse command-line options.
-   int ref_levels = 5;
-   int order = 2;
-   double alpha = 1.0;
-   double epsilon = 0.01;
-   double vol_fraction = 0.5;
-   int max_it = 1e3;
-   double itol = 1e-1;
-   double ntol = 1e-4;
-   double rho_min = 1e-6;
-   double lambda = 1.0;
-   double mu = 1.0;
-   bool glvis_visualization = true;
-   bool paraview_output = false;
-
-   OptionsParser args(argc, argv);
-   args.AddOption(&ref_levels, "-r", "--refine",
-                  "Number of times to refine the mesh uniformly.");
-   args.AddOption(&order, "-o", "--order",
-                  "Order (degree) of the finite elements.");
-   args.AddOption(&alpha, "-alpha", "--alpha-step-length",
-                  "Step length for gradient descent.");
-   args.AddOption(&epsilon, "-epsilon", "--epsilon-thickness",
-                  "Length scale for ρ.");
-   args.AddOption(&max_it, "-mi", "--max-it",
-                  "Maximum number of gradient descent iterations.");
-   args.AddOption(&ntol, "-ntol", "--rel-tol",
-                  "Normalized exit tolerance.");
-   args.AddOption(&itol, "-itol", "--abs-tol",
-                  "Increment exit tolerance.");
-   args.AddOption(&vol_fraction, "-vf", "--volume-fraction",
-                  "Volume fraction for the material density.");
-   args.AddOption(&lambda, "-lambda", "--lambda",
-                  "Lamé constant λ.");
-   args.AddOption(&mu, "-mu", "--mu",
-                  "Lamé constant μ.");
-   args.AddOption(&rho_min, "-rmin", "--psi-min",
-                  "Minimum of density coefficient.");
-   args.AddOption(&glvis_visualization, "-vis", "--visualization", "-no-vis",
-                  "--no-visualization",
-                  "Enable or disable GLVis visualization.");
-   args.AddOption(&paraview_output, "-pv", "--paraview", "-no-pv",
-                  "--no-paraview",
-                  "Enable or disable ParaView output.");
-   args.Parse();
-   if (!args.Good())
-   {
-      if (myid == 0)
-      {
-         args.PrintUsage(cout);
-      }
-      MPI_Finalize();
-      return 1;
-   }
-   if (myid == 0)
-   {
-      mfem::out << num_procs << " number of process created.\n";
-      args.PrintOptions(cout);
-   }
-
-   Mesh mesh = Mesh::MakeCartesian2D(3, 1, mfem::Element::Type::QUADRILATERAL,
-                                     true, 3.0, 1.0);
-   int dim = mesh.Dimension();
-
-   // 2. Set BCs.
-   for (int i = 0; i<mesh.GetNBE(); i++)
-   {
-      Element * be = mesh.GetBdrElement(i);
-      Array<int> vertices;
-      be->GetVertices(vertices);
-
-      double * coords1 = mesh.GetVertex(vertices[0]);
-      double * coords2 = mesh.GetVertex(vertices[1]);
-
-      Vector center(2);
-      center(0) = 0.5*(coords1[0] + coords2[0]);
-      center(1) = 0.5*(coords1[1] + coords2[1]);
-
-      if (abs(center(0) - 0.0) < 1e-10)
-      {
-         // the left edge
-         be->SetAttribute(1);
-      }
-      else
-      {
-         // all other boundaries
-         be->SetAttribute(2);
-      }
-   }
-   mesh.SetAttributes();
-
-   // 3. Refine the mesh.
-   for (int lev = 0; lev < ref_levels; lev++)
-   {
-      mesh.UniformRefinement();
-   }
-
-   ParMesh pmesh(MPI_COMM_WORLD, mesh);
-   mesh.Clear();
-
-   // 4. Define the necessary finite element spaces on the mesh.
-   H1_FECollection state_fec(order, dim); // space for u
-   H1_FECollection filter_fec(order, dim); // space for ρ̃
-   L2_FECollection control_fec(order-1, dim,
-                               BasisType::GaussLobatto); // space for ψ
-   ParFiniteElementSpace state_fes(&pmesh, &state_fec,dim);
-   ParFiniteElementSpace filter_fes(&pmesh, &filter_fec);
-   ParFiniteElementSpace control_fes(&pmesh, &control_fec);
-
-   HYPRE_BigInt state_size = state_fes.GlobalTrueVSize();
-   HYPRE_BigInt control_size = control_fes.GlobalTrueVSize();
-   HYPRE_BigInt filter_size = filter_fes.GlobalTrueVSize();
-   if (myid==0)
-   {
-      cout << "Number of state unknowns: " << state_size << endl;
-      cout << "Number of filter unknowns: " << filter_size << endl;
-      cout << "Number of control unknowns: " << control_size << endl;
-   }
-
-   // 5. Set the initial guess for ρ.
-   ParGridFunction u(&state_fes);
-   ParGridFunction psi(&control_fes);
-   ParGridFunction psi_old(&control_fes);
-   ParGridFunction rho_filter(&filter_fes);
-   u = 0.0;
-   rho_filter = vol_fraction;
-   psi = inv_sigmoid(vol_fraction);
-   psi_old = inv_sigmoid(vol_fraction);
-
-   // ρ = sigmoid(ψ)
-   MappedGridFunctionCoefficient rho(&psi, sigmoid);
-   // Interpolation of ρ = sigmoid(ψ) in control fes (for ParaView output)
-   ParGridFunction rho_gf(&control_fes);
-   // ρ - ρ_old = sigmoid(ψ) - sigmoid(ψ_old)
-   DiffMappedGridFunctionCoefficient succ_diff_rho(&psi, &psi_old, sigmoid);
-
-   // 6. Set-up the physics solver.
-   int maxat = pmesh.bdr_attributes.Max();
-   Array<int> ess_bdr(maxat);
-   ess_bdr = 0;
-   ess_bdr[0] = 1;
-   ConstantCoefficient one(1.0);
-   ConstantCoefficient lambda_cf(lambda);
-   ConstantCoefficient mu_cf(mu);
-   LinearElasticitySolver * ElasticitySolver = new LinearElasticitySolver();
-   ElasticitySolver->SetMesh(&pmesh);
-   ElasticitySolver->SetOrder(state_fec.GetOrder());
-   ElasticitySolver->SetupFEM();
-   Vector center(2); center(0) = 2.9; center(1) = 0.5;
-   Vector force(2); force(0) = 0.0; force(1) = -1.0;
-   double r = 0.05;
-   VolumeForceCoefficient vforce_cf(r,center,force);
-   ElasticitySolver->SetRHSCoefficient(&vforce_cf);
-   ElasticitySolver->SetEssentialBoundary(ess_bdr);
-
-   // 7. Set-up the filter solver.
-   ConstantCoefficient eps2_cf(epsilon*epsilon);
-   DiffusionSolver * FilterSolver = new DiffusionSolver();
-   FilterSolver->SetMesh(&pmesh);
-   FilterSolver->SetOrder(filter_fec.GetOrder());
-   FilterSolver->SetDiffusionCoefficient(&eps2_cf);
-   FilterSolver->SetMassCoefficient(&one);
-   Array<int> ess_bdr_filter;
-   if (pmesh.bdr_attributes.Size())
-   {
-      ess_bdr_filter.SetSize(pmesh.bdr_attributes.Max());
-      ess_bdr_filter = 0;
-   }
-   FilterSolver->SetEssentialBoundary(ess_bdr_filter);
-   FilterSolver->SetupFEM();
-
-   ParBilinearForm mass(&control_fes);
-   mass.AddDomainIntegrator(new InverseIntegrator(new MassIntegrator(one)));
-   mass.Assemble();
-   HypreParMatrix M;
-   Array<int> empty;
-   mass.FormSystemMatrix(empty,M);
-
-   // 8. Define the Lagrange multiplier and gradient functions.
-   ParGridFunction grad(&control_fes);
-   ParGridFunction w_filter(&filter_fes);
-
-   // 9. Define some tools for later.
-   ConstantCoefficient zero(0.0);
-   ParGridFunction onegf(&control_fes);
-   onegf = 1.0;
-   ParGridFunction zerogf(&control_fes);
-   zerogf = 0.0;
-   ParLinearForm vol_form(&control_fes);
-   vol_form.AddDomainIntegrator(new DomainLFIntegrator(one));
-   vol_form.Assemble();
-   double domain_volume = vol_form(onegf);
-   const double target_volume = domain_volume * vol_fraction;
-
-   // 10. Connect to GLVis. Prepare for VisIt output.
-   char vishost[] = "localhost";
-   int  visport   = 19916;
-   socketstream sout_r;
-   if (glvis_visualization)
-   {
-      sout_r.open(vishost, visport);
-      sout_r.precision(8);
-   }
-
-   mfem::ParaViewDataCollection paraview_dc("ex37p", &pmesh);
-   if (paraview_output)
-   {
-      rho_gf.ProjectCoefficient(rho);
-      paraview_dc.SetPrefixPath("ParaView");
-      paraview_dc.SetLevelsOfDetail(order);
-      paraview_dc.SetDataFormat(VTKFormat::BINARY);
-      paraview_dc.SetHighOrderOutput(true);
-      paraview_dc.SetCycle(0);
-      paraview_dc.SetTime(0.0);
-      paraview_dc.RegisterField("displacement",&u);
-      paraview_dc.RegisterField("density",&rho_gf);
-      paraview_dc.RegisterField("filtered_density",&rho_filter);
-      paraview_dc.Save();
-   }
-
-   // 11. Iterate:
-   for (int k = 1; k <= max_it; k++)
-   {
-      if (k > 1) { alpha *= ((double) k) / ((double) k-1); }
-
-      if (myid == 0)
-      {
-         cout << "\nStep = " << k << endl;
-      }
-
-      // Step 1 - Filter solve
-      // Solve (ϵ^2 ∇ ρ̃, ∇ v ) + (ρ̃,v) = (ρ,v)
-      FilterSolver->SetRHSCoefficient(&rho);
-      FilterSolver->Solve();
-      rho_filter = *FilterSolver->GetFEMSolution();
-
-      // Step 2 - State solve
-      // Solve (λ r(ρ̃) ∇⋅u, ∇⋅v) + (2 μ r(ρ̃) ε(u), ε(v)) = (f,v)
-      SIMPInterpolationCoefficient SIMP_cf(&rho_filter,rho_min, 1.0);
-      ProductCoefficient lambda_SIMP_cf(lambda_cf,SIMP_cf);
-      ProductCoefficient mu_SIMP_cf(mu_cf,SIMP_cf);
-      ElasticitySolver->SetLameCoefficients(&lambda_SIMP_cf,&mu_SIMP_cf);
-      ElasticitySolver->Solve();
-      u = *ElasticitySolver->GetFEMSolution();
-
-      // Step 3 - Adjoint filter solve
-      // Solve (ϵ² ∇ w̃, ∇ v) + (w̃ ,v) = (-r'(ρ̃) ( λ |∇⋅u|² + 2 μ |ε(u)|²),v)
-      StrainEnergyDensityCoefficient rhs_cf(&lambda_cf,&mu_cf,&u, &rho_filter,
-                                            rho_min);
-      FilterSolver->SetRHSCoefficient(&rhs_cf);
-      FilterSolver->Solve();
-      w_filter = *FilterSolver->GetFEMSolution();
-
-      // Step 4 - Compute gradient
-      // Solve G = M⁻¹w̃
-      GridFunctionCoefficient w_cf(&w_filter);
-      ParLinearForm w_rhs(&control_fes);
-      w_rhs.AddDomainIntegrator(new DomainLFIntegrator(w_cf));
-      w_rhs.Assemble();
-      M.Mult(w_rhs,grad);
-
-      // Step 5 - Update design variable ψ ← proj(ψ - αG)
-      psi.Add(-alpha, grad);
-      const double material_volume = proj(psi, target_volume);
-
-      // Compute ||ρ - ρ_old|| in control fes.
-      double norm_increment = zerogf.ComputeL1Error(succ_diff_rho);
-      double norm_reduced_gradient = norm_increment/alpha;
-      psi_old = psi;
-
-      double compliance = (*(ElasticitySolver->GetLinearForm()))(u);
-      MPI_Allreduce(MPI_IN_PLACE,&compliance,1,MPI_DOUBLE,MPI_SUM,MPI_COMM_WORLD);
-      if (myid == 0)
-      {
-         mfem::out << "norm of the reduced gradient = " << norm_reduced_gradient << endl;
-         mfem::out << "norm of the increment = " << norm_increment << endl;
-         mfem::out << "compliance = " << compliance << endl;
-         mfem::out << "volume fraction = " << material_volume / domain_volume << endl;
-      }
-
-      if (glvis_visualization)
-      {
-         ParGridFunction r_gf(&filter_fes);
-         r_gf.ProjectCoefficient(SIMP_cf);
-         sout_r << "parallel " << num_procs << " " << myid << "\n";
-         sout_r << "solution\n" << pmesh << r_gf
-                << "window_title 'Design density r(ρ̃)'" << flush;
-      }
-
-      if (paraview_output)
-      {
-         rho_gf.ProjectCoefficient(rho);
-         paraview_dc.SetCycle(k);
-         paraview_dc.SetTime((double)k);
-         paraview_dc.Save();
-      }
-
-      if (norm_reduced_gradient < ntol && norm_increment < itol)
-      {
-         break;
-      }
-   }
-
-   delete ElasticitySolver;
-   delete FilterSolver;
-
-   return 0;
-}
@@ -5,7 +5,6 @@
 // Sample runs:  mpirun -np 4 ex3p -m ../data/star.mesh
 //               mpirun -np 4 ex3p -m ../data/square-disc.mesh -o 2
 //               mpirun -np 4 ex3p -m ../data/beam-tet.mesh
-//               mpirun -np 4 ex3p -m ../data/beam-tet.mesh -nc -o 2
 //               mpirun -np 4 ex3p -m ../data/beam-hex.mesh
 //               mpirun -np 4 ex3p -m ../data/beam-hex.mesh -o 2 -pa
 //               mpirun -np 4 ex3p -m ../data/escher.mesh
@@ -71,7 +70,6 @@ int main(int argc, char *argv[])
   int order = 1;
   bool static_cond = false;
   bool pa = false;
-   bool nc = false;
   const char *device_config = "cpu";
   bool visualization = true;
 #ifdef MFEM_USE_AMGX
@@ -89,9 +87,6 @@ int main(int argc, char *argv[])
                  "--no-static-condensation", "Enable static condensation.");
   args.AddOption(&pa, "-pa", "--partial-assembly", "-no-pa",
                  "--no-partial-assembly", "Enable Partial Assembly.");
-   args.AddOption(&nc, "-nc", "--non-conforming", "-c",
-                  "--conforming",
-                  "Mark the mesh as nonconforming before partitioning.");
   args.AddOption(&device_config, "-d", "--device",
                  "Device configuration string, see Device::Configure().");
   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
@@ -129,11 +124,6 @@ int main(int argc, char *argv[])
   Mesh *mesh = new Mesh(mesh_file, 1, 1);
   dim = mesh->Dimension();
   int sdim = mesh->SpaceDimension();
-   if (nc)
-   {
-      // Can set to false to use conformal refinement for simplices.
-      mesh->EnsureNCMesh(true);
-   }

   // 5. Refine the serial mesh on all processors to increase the resolution. In
   //    this example we do 'ref_levels' of uniform refinement. We choose
@@ -450,8 +450,7 @@ int main(int argc, char *argv[])

 // Implementation of class FE_Evolution
 FE_Evolution::FE_Evolution(BilinearForm &M_, BilinearForm &K_, const Vector &b_)
-   : TimeDependentOperator(M_.FESpace()->GetTrueVSize()),
-     M(M_), K(K_), b(b_), z(height)
+   : TimeDependentOperator(M_.Height()), M(M_), K(K_), b(b_), z(M_.Height())
 {
   Array<int> ess_tdof_list;
   if (M.GetAssemblyLevel() == AssemblyLevel::LEGACY)
@@ -536,10 +536,8 @@ int main(int argc, char *argv[])
      if (!sout)
      {
         if (Mpi::Root())
-         {
            cout << "Unable to connect to GLVis server at "
                 << vishost << ':' << visport << endl;
-         }
         visualization = false;
         if (Mpi::Root())
         {
@@ -554,10 +552,8 @@ int main(int argc, char *argv[])
         sout << "pause\n";
         sout << flush;
         if (Mpi::Root())
-         {
            cout << "GLVis visualization paused."
                 << " Press space (in the GLVis window) to resume it.\n";
-         }
      }
   }

@@ -659,9 +655,9 @@ int main(int argc, char *argv[])
 // Implementation of class FE_Evolution
 FE_Evolution::FE_Evolution(ParBilinearForm &M_, ParBilinearForm &K_,
                           const Vector &b_, PrecType prec_type)
-   : TimeDependentOperator(M_.ParFESpace()->GetTrueVSize()), b(b_),
+   : TimeDependentOperator(M_.Height()), b(b_),
     M_solver(M_.ParFESpace()->GetComm()),
-     z(height)
+     z(M_.Height())
 {
   if (M_.GetAssemblyLevel()==AssemblyLevel::LEGACY)
   {
@@ -23,21 +23,20 @@ MFEM_LIB_FILE = mfem_is_not_built

 SEQ_EXAMPLES = ex0 ex1 ex2 ex3 ex4 ex5 ex6 ex7 ex8 ex9 ex10 ex14 ex15 ex16 \
  ex17 ex18 ex19 ex20 ex21 ex22 ex23 ex24 ex25 ex26 ex27 ex28 ex29 ex30 \
-  ex31 ex33 ex34 ex36 ex37
+  ex31 ex33
 PAR_EXAMPLES = ex0p ex1p ex2p ex3p ex4p ex5p ex6p ex7p ex8p ex9p ex10p ex11p \
  ex12p ex13p ex14p ex15p ex16p ex17p ex18p ex19p ex20p ex21p ex22p ex24p \
-  ex25p ex26p ex27p ex28p ex29p ex30p ex31p ex32p ex33p ex34p ex35p ex36p \
-	ex37p
-SEQ_DEVICE_EXAMPLES = ex1 ex3 ex4 ex5 ex6 ex9 ex22 ex24 ex25 ex26 ex34
+  ex25p ex26p ex27p ex28p ex29p ex30p ex31p ex32p ex33p
+SEQ_DEVICE_EXAMPLES = ex1 ex3 ex4 ex5 ex6 ex9 ex22 ex24 ex25 ex26
 PAR_DEVICE_EXAMPLES = ex1p ex2p ex3p ex4p ex5p ex6p ex7p ex9p ex13p ex22p \
-  ex24p ex25p ex26p ex34p ex35p
+  ex24p ex25p ex26p

 ifeq ($(MFEM_USE_MPI),NO)
   EXAMPLES = $(SEQ_EXAMPLES)
 else
   EXAMPLES = $(PAR_EXAMPLES) $(SEQ_EXAMPLES)
 endif
-SUBDIRS =
+SUBDIRS = contact 
 ifeq ($(MFEM_USE_AMGX),YES)
   SUBDIRS += amgx
 endif
@@ -93,12 +92,10 @@ $(SUBDIRS_TPRINT):
 # Additional dependencies
 ex18: $(SRC)ex18.hpp
 ex33: $(SRC)ex33.hpp
-ex37: $(SRC)ex37.hpp

 ifeq ($(MFEM_USE_MPI),YES)
 ex18p: $(SRC)ex18.hpp
 ex33p: $(SRC)ex33.hpp
-ex37p: $(SRC)ex37.hpp
 endif

 MFEM_TESTS = EXAMPLES
@@ -142,10 +139,6 @@ ex27-test-seq: ex27
 	@$(call mfem-test,$<,, Serial example,-dg)
 ex27p-test-par: ex27p
 	@$(call mfem-test,$<, $(RUN_MPI), Parallel example,-dg)
-ex37-test-seq: ex37
-	@$(call mfem-test,$<,, Serial example,-mi 3)
-ex37p-test-par: ex37p
-	@$(call mfem-test,$<, $(RUN_MPI), Parallel example,-mi 3)
 # Testing: optional tests
 ifeq ($(MFEM_USE_STRUMPACK),YES)
 ex11p-test-strumpack: ex11p
@@ -190,4 +183,3 @@ clean-exec:
 	@rm -f ex23.mesh ex23-*.gf
 	@rm -f ex25.mesh ex25-*.gf ex25p-*.*
 	@rm -rf ex28_* ex28p_*
-	@rm -rf cond.* cond_mesh.* cond_j.* dsol.* port_mesh.* port_mode.*
@@ -24,13 +24,13 @@ if (MFEM_USE_MPI)
    ex10p.cpp
    )
  list(APPEND PETSC_RC_FILES
-    rc_ex1p rc_ex1p_device rc_ex1p_deviceamg
-    rc_ex2p rc_ex2p_bddc rc_ex2p_asm
+    rc_ex1p
+    rc_ex2p
    rc_ex3p rc_ex3p_bddc
    rc_ex4p rc_ex4p_bddc
    rc_ex5p_bddc rc_ex5p_fieldsplit
-    rc_ex9p_expl rc_ex9p_expl_device rc_ex9p_impl
-    rc_ex10p rc_ex10p_mf rc_ex10p_mfop rc_ex10p_jfnk
+    rc_ex9p_expl rc_ex9p_impl
+    rc_ex10p
    )
 endif()

@@ -39,7 +39,7 @@ if (MFEM_USE_SLEPC)
    ex11p.cpp
    )
  list(APPEND PETSC_RC_FILES
-    rc_ex11p_lobpcg rc_ex11p_lobpcg_device rc_ex11p_gd
+    rc_ex11p_lobpcg rc_ex11p_gd
    )
 endif()

@@ -74,13 +74,7 @@ add_mfem_examples(PETSC_EXAMPLES_SRCS ${PFX} copy_petsc_rc_files test_petsc)
 # Command line options for the tests.
 set(EX1_ARGS_W       -m ../../data/amr-quad.mesh --usepetsc)
 set(EX1_ARGS_P       -m ../../data/amr-quad.mesh --usepetsc --petscopts rc_ex1p)
-set(EX1_ARGS_CUDA    -m ../../data/star.mesh --usepetsc --partial-assembly --device cuda --petscopts rc_ex1p_device)
-set(EX1_ARGS_CUDAAMG -m ../../data/star.mesh --usepetsc --device cuda --petscopts rc_ex1p_deviceamg)
-set(EX1_ARGS_HIP     -m ../../data/star.mesh --usepetsc --partial-assembly --device hip --petscopts rc_ex1p_device)
-set(EX1_ARGS_HIPAMG  -m ../../data/star.mesh --usepetsc --device hip --petscopts rc_ex1p_deviceamg)
 set(EX2_ARGS         -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex2p)
-set(EX2_ARGS_BDDC    -m ../../data/beam-tri.mesh --usepetsc --nonoverlapping --petscopts rc_ex2p_bddc)
-set(EX2_ARGS_ASM     -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex2p_asm)
 set(EX3_ARGS         -m ../../data/klein-bottle.mesh -o 2 -f 0.1 --usepetsc --petscopts rc_ex3p_bddc --nonoverlapping)
 set(EX4_ARGS         -m ../../data/klein-bottle.mesh -o 2 --usepetsc --petscopts rc_ex4p_bddc --nonoverlapping)
 set(EX4_HYB_ARGS     -m ../../data/klein-bottle.mesh -o 2 --usepetsc --petscopts rc_ex4p_bddc --nonoverlapping --hybridization)
@@ -91,46 +85,22 @@ set(EX6_ARGS         -m ../../data/amr-quad.mesh --usepetsc)
 set(EX6_NONOVL_ARGS  -m ../../data/amr-quad.mesh --usepetsc --nonoverlapping)
 set(EX9_E_ARGS       -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl -dt 0.1)
 set(EX9_ES_ARGS      -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl --no-step)
-set(EX9_ES_ARGS_CUDA -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl_device --no-step --partial-assembly --device cuda)
-set(EX9_ES_ARGS_HIP  -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl_device --no-step --partial-assembly --device hip)
 set(EX9_IS_ARGS      -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_impl --implicit -tf 0.5)
 set(EX10_ARGS        -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex10p -tf 30 -s 3 -rs 2 -dt 3)
-set(EX10_MF_ARGS     -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex10p_mf -tf 6 -s 3 -rs 0 -dt 3)
-set(EX10_MFOP_ARGS   -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex10p_mfop -tf 6 -s 3 -rs 0 -dt 3)
-set(EX10_JFNK_ARGS   -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex10p_jfnk --jfnk -tf 6 -s 3 -rs 0 -dt 3)
 if (MFEM_USE_SLEPC)
  set(EX11_ARGS_SINV   -m ../../data/star.mesh --useslepc)
  set(EX11_ARGS_LOBPCG -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_lobpcg)
-  set(EX11_ARGS_LOBPCG_CUDA -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_lobpcg_device --device cuda)
-  set(EX11_ARGS_LOBPCG_HIP  -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_lobpcg_device --device hip)
  set(EX11_ARGS_GD -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_gd)
 endif()

 # Add the tests: one test per command-line-variable.
 if (MFEM_ENABLE_TESTING)
  set(TEST_OPTIONS_VARS
-      EX1_ARGS_W EX1_ARGS_P EX2_ARGS EX2_ARGS_BDDC EX2_ARGS_ASM EX3_ARGS
-      EX4_ARGS EX4_HYB_ARGS EX5_BDDC_LB_ARGS EX5_BDDC_GB_ARGS EX5_FSPL_ARGS
-      EX6_ARGS EX6_NONOVL_ARGS EX9_E_ARGS EX9_ES_ARGS EX9_IS_ARGS EX10_ARGS
-      EX10_MF_ARGS EX10_MFOP_ARGS EX10_JFNK_ARGS)
+      EX1_ARGS_W EX1_ARGS_P EX2_ARGS EX3_ARGS EX4_ARGS EX4_HYB_ARGS
+      EX5_BDDC_LB_ARGS EX5_BDDC_GB_ARGS EX5_FSPL_ARGS EX6_ARGS EX6_NONOVL_ARGS
+      EX9_E_ARGS EX9_ES_ARGS EX9_IS_ARGS EX10_ARGS)
  if (MFEM_USE_SLEPC)
-    list(APPEND TEST_OPTIONS_VARS
-      EX11_ARGS_SINV EX11_ARGS_LOBPCG EX11_ARGS_GD)
-  endif()
-  # CUDA/HIP tests
-  if (MFEM_USE_CUDA)
-    list(APPEND TEST_OPTIONS_VARS
-      EX1_ARGS_CUDA EX1_ARGS_CUDAAMG EX9_ES_ARGS_CUDA)
-    if (MFEM_USE_SLEPC)
-      list(APPEND TEST_OPTIONS_VARS EX11_ARGS_LOBPCG_CUDA)
-    endif()
-  elseif (MFEM_USE_HIP)
-    list(APPEND TEST_OPTIONS_VARS
-      EX1_ARGS_HIP EX1_ARGS_HIPAMG EX9_ES_ARGS_HIP)
-    if (MFEM_USE_SLEPC)
-      # SLEPc does not support BVSVEC with HIP
-      # list(APPEND TEST_OPTIONS_VARS EX11_ARGS_LOBPCG_HIP)
-    endif()
+      list(APPEND TEST_OPTIONS_VARS EX11_ARGS_SINV EX11_ARGS_LOBPCG EX11_ARGS_GD)
  endif()

  foreach(TEST_OPTIONS_VAR ${TEST_OPTIONS_VARS})
@@ -145,7 +115,7 @@ if (MFEM_ENABLE_TESTING)

    # All PETSC tests are parallel.
    if (MFEM_USE_MPI)
-      add_test(NAME ${TEST_NAME_FULL}_np=${MFEM_MPI_NP}
+      add_test(NAME ${TEST_NAME_FULL}_np=4
        COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} ${MFEM_MPI_NP}
        ${MPIEXEC_PREFLAGS}
        $<TARGET_FILE:${TEST_NAME}> ${TEST_OPTIONS}
@@ -7,7 +7,7 @@
 //               mpirun -np 4 ex1p -m ../../data/amr-quad.mesh --petscopts rc_ex1p
 //
 // Device sample runs:
-//               mpirun -np 4 ex1p -pa -d cuda --petscopts rc_ex1p_device
+//               mpirun -np 4 ex1p -pa -d cuda --petscopts rc_ex1p_cuda
 //
 // Description:  This example code demonstrates the use of MFEM to define a
 //               simple finite element discretization of the Laplace problem
@@ -17,7 +17,7 @@
 //               finite elements (velocity u) and piecewise discontinuous
 //               polynomials (pressure p).
 //
-//               The example demonstrates the use of the BlockOperator class, as
+//               The example demonstrates the use of the BlockMatrix class, as
 //               well as the collective saving of several grid functions in a
 //               VisIt (visit.llnl.gov) visualization format.
 //
@@ -520,10 +520,10 @@ int main(int argc, char *argv[])
 // Implementation of class FE_Evolution
 FE_Evolution::FE_Evolution(ParBilinearForm &M_, ParBilinearForm &K_,
                           const Vector &b_,bool M_in_lhs)
-   : TimeDependentOperator(M_.ParFESpace()->GetTrueVSize(), 0.0,
+   : TimeDependentOperator(M_.Height(), 0.0,
                           M_in_lhs ? TimeDependentOperator::IMPLICIT
                           : TimeDependentOperator::EXPLICIT),
-     b(b_), comm(M_.ParFESpace()->GetComm()), M_solver(comm), z(height),
+     b(b_), comm(M_.ParFESpace()->GetComm()), M_solver(comm), z(M_.Height()),
     iJacobian(NULL), rJacobian(NULL)
 {
   MAlev = M_.GetAssemblyLevel();
@@ -66,9 +66,7 @@ include $(MFEM_TEST_MK)

 # Testing: Parallel runs
 RUN_MPI = $(MFEM_MPIEXEC) $(MFEM_MPIEXEC_NP) $(MFEM_MPI_NP)
-TESTNAME      = Parallel PETSc example
-TESTNAME_CUDA = Parallel CUDA PETSc example
-TESTNAME_HIP  = Parallel HIP PETSc example
+TESTNAME = Parallel PETSc example
 %-test-par: %
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME))

@@ -76,10 +74,8 @@ TESTNAME_HIP  = Parallel HIP PETSc example
 # Testing PETSc execution options.
 EX1_ARGS_W            := -m ../../data/amr-quad.mesh --usepetsc
 EX1_ARGS_P            := -m ../../data/amr-quad.mesh --usepetsc --petscopts rc_ex1p
-EX1_ARGS_CUDA         := -m ../../data/star.mesh --usepetsc --partial-assembly --device cuda --petscopts rc_ex1p_device
-EX1_ARGS_CUDAAMG      := -m ../../data/star.mesh --usepetsc --device cuda --petscopts rc_ex1p_deviceamg
-EX1_ARGS_HIP          := -m ../../data/star.mesh --usepetsc --partial-assembly --device hip --petscopts rc_ex1p_device
-EX1_ARGS_HIPAMG       := -m ../../data/star.mesh --usepetsc --device hip --petscopts rc_ex1p_deviceamg
+EX1_ARGS_CUDA         := -m ../../data/star.mesh --usepetsc --partial-assembly --device cuda --petscopts rc_ex1p_cuda
+EX1_ARGS_CUDAAMG      := -m ../../data/star.mesh --usepetsc --device cuda --petscopts rc_ex1p_cudaamg
 EX2_ARGS              := -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex2p
 EX2_ARGS_BDDC         := -m ../../data/beam-tri.mesh --usepetsc --nonoverlapping --petscopts rc_ex2p_bddc
 EX2_ARGS_ASM          := -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex2p_asm
@@ -93,8 +89,7 @@ EX6_ARGS              := -m ../../data/amr-quad.mesh --usepetsc
 EX6_NONOVL_ARGS       := -m ../../data/amr-quad.mesh --usepetsc --nonoverlapping
 EX9_E_ARGS            := -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl -dt 0.1
 EX9_ES_ARGS           := -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl --no-step
-EX9_ES_ARGS_CUDA      := -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl_device --no-step --partial-assembly --device cuda
-EX9_ES_ARGS_HIP       := -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl_device --no-step --partial-assembly --device hip
+EX9_ES_ARGS_CUDA      := -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_expl_cuda --no-step --partial-assembly --device cuda
 EX9_IS_ARGS           := -m ../../data/periodic-hexagon.mesh --usepetsc --petscopts rc_ex9p_impl --implicit -tf 0.5
 EX10_ARGS             := -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex10p -tf 30 -s 3 -rs 2 -dt 3
 EX10_MF_ARGS          := -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex10p_mf -tf 6 -s 3 -rs 0 -dt 3
@@ -102,20 +97,15 @@ EX10_MFOP_ARGS        := -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_
 EX10_JFNK_ARGS        := -m ../../data/beam-quad.mesh --usepetsc --petscopts rc_ex10p_jfnk --jfnk -tf 6 -s 3 -rs 0 -dt 3
 EX11_ARGS_SINV        := -m ../../data/star.mesh --useslepc
 EX11_ARGS_LOBPCG      := -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_lobpcg
-EX11_ARGS_LOBPCG_CUDA := -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_lobpcg_device --device cuda
-EX11_ARGS_LOBPCG_HIP  := -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_lobpcg_device --device hip
+EX11_ARGS_LOBPCG_CUDA := -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_lobpcg_cuda --device cuda
 EX11_ARGS_GD          := -m ../../data/star.mesh --useslepc --slepcopts rc_ex11p_gd

 ex1p-test-par: ex1p
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX1_ARGS_W))
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX1_ARGS_P))
 ifeq ($(MFEM_USE_CUDA),YES)
-	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_CUDA),$(EX1_ARGS_CUDA))
-	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_CUDA),$(EX1_ARGS_CUDAAMG))
-endif
-ifeq ($(MFEM_USE_HIP),YES)
-	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_HIP),$(EX1_ARGS_HIP))
-	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_HIP),$(EX1_ARGS_HIPAMG))
+	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX1_ARGS_CUDA))
+	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX1_ARGS_CUDAAMG))
 endif
 ex2p-test-par: ex2p
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX2_ARGS))
@@ -138,10 +128,7 @@ ex9p-test-par: ex9p
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX9_ES_ARGS))
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX9_IS_ARGS))
 ifeq ($(MFEM_USE_CUDA),YES)
-	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_CUDA),$(EX9_ES_ARGS_CUDA))
-endif
-ifeq ($(MFEM_USE_HIP),YES)
-	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_HIP),$(EX9_ES_ARGS_HIP))
+	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX9_ES_ARGS_CUDA))
 endif
 ex10p-test-par: ex10p
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX10_ARGS))
@@ -153,12 +140,8 @@ ex11p-test-par: ex11p
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX11_ARGS_SINV))
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX11_ARGS_LOBPCG))
 ifeq ($(MFEM_USE_CUDA),YES)
-	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_CUDA),$(EX11_ARGS_LOBPCG_CUDA))
+	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX11_ARGS_LOBPCG_CUDA))
 endif
-# SLEPc does not support BVSVEC with HIP
-#ifeq ($(MFEM_USE_HIP),YES)
-#	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME_HIP),$(EX11_ARGS_LOBPCG_HIP))
-#endif
 	@$(call mfem-test,$<, $(RUN_MPI), $(TESTNAME),$(EX11_ARGS_GD))
 endif

@@ -173,9 +156,6 @@ clean: clean-build clean-exec
 clean-build:
 	rm -f *.o *~ $(SEQ_EXAMPLES) $(PAR_EXAMPLES)
 	rm -rf *.dSYM *.TVD.*breakpoints
-ifneq ($(SRC),)
-	rm -f $(RC_FILES)
-endif

 clean-exec:
 	@rm -rf mesh.* sol.* sol_p.* sol_u.* Example5*
@@ -68,43 +68,11 @@ if (MFEM_ENABLE_TESTING)
      add_test(NAME ${TEST_NAME}_ser
        COMMAND ${TEST_NAME} ${THIS_TEST_OPTIONS})
    else()
-      add_test(NAME ${TEST_NAME}_np=${MFEM_MPI_NP}
+      add_test(NAME ${TEST_NAME}_np=4
        COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} ${MFEM_MPI_NP}
        ${MPIEXEC_PREFLAGS}
        $<TARGET_FILE:${TEST_NAME}> ${THIS_TEST_OPTIONS}
        ${MPIEXEC_POSTFLAGS})
    endif()
  endforeach()
-
-  # Add CUDA/HIP tests.
-  set(DEVICE_EXAMPLES
-      # serial examples with device support:
-      ex9
-      # parallel examples with device support:
-      ex9p)
-  set(MFEM_TEST_DEVICE)
-  if (MFEM_USE_CUDA)
-    set(MFEM_TEST_DEVICE "cuda")
-  elseif (MFEM_USE_HIP)
-    set(MFEM_TEST_DEVICE "hip")
-  endif()
-  if (MFEM_TEST_DEVICE)
-    foreach(TEST_NAME ${DEVICE_EXAMPLES})
-      string(TOUPPER ${TEST_NAME} UP_TEST_NAME)
-
-      set(THIS_TEST_OPTIONS "-no-vis" "-d" "${MFEM_TEST_DEVICE}")
-      list(APPEND THIS_TEST_OPTIONS ${${UP_TEST_NAME}_TEST_OPTS})
-
-      if (NOT (${TEST_NAME} MATCHES ".*p$"))
-        add_test(NAME ${PFX}${TEST_NAME}_${MFEM_TEST_DEVICE}_ser
-          COMMAND ${PFX}${TEST_NAME} ${THIS_TEST_OPTIONS})
-      else()
-        add_test(NAME ${PFX}${TEST_NAME}_${MFEM_TEST_DEVICE}_np=${MFEM_MPI_NP}
-          COMMAND ${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} ${MFEM_MPI_NP}
-          ${MPIEXEC_PREFLAGS}
-          $<TARGET_FILE:${PFX}${TEST_NAME}> ${THIS_TEST_OPTIONS}
-          ${MPIEXEC_POSTFLAGS})
-      endif()
-    endforeach()
-  endif(MFEM_TEST_DEVICE)
-endif(MFEM_ENABLE_TESTING)
+endif()
@@ -12,7 +12,8 @@ use of MFEM features based on the SUNDIALS suite of time integration and
 non-linear solvers.

 To build these examples, make sure that MFEM is configured with the option
-"MFEM_USE_SUNDIALS = YES", see the top-level INSTALL file for details.
+"MFEM_USE_SUNDIALS = YES", see the top-level INSTALL file for details (version
+2.7 or higher of SUNDIALS is required).

 We recommend comparing the original example codes with the corresponding files
 in the current directory.
@@ -280,16 +280,15 @@ int main(int argc, char *argv[])
      k.SetAssemblyLevel(AssemblyLevel::FULL);
   }
   m.AddDomainIntegrator(new MassIntegrator);
-   constexpr double alpha = -1.0;
-   k.AddDomainIntegrator(new ConvectionIntegrator(velocity, alpha));
+   k.AddDomainIntegrator(new ConvectionIntegrator(velocity, -1.0));
   k.AddInteriorFaceIntegrator(
-      new NonconservativeDGTraceIntegrator(velocity, alpha));
+      new TransposeIntegrator(new DGTraceIntegrator(velocity, 1.0, -0.5)));
   k.AddBdrFaceIntegrator(
-      new NonconservativeDGTraceIntegrator(velocity, alpha));
+      new TransposeIntegrator(new DGTraceIntegrator(velocity, 1.0, -0.5)));

   LinearForm b(&fes);
   b.AddBdrFaceIntegrator(
-      new BoundaryFlowIntegrator(inflow, velocity, alpha));
+      new BoundaryFlowIntegrator(inflow, velocity, -1.0, -0.5));

   m.Assemble();
   int skip_zeros = 0;
@@ -476,8 +475,7 @@ int main(int argc, char *argv[])

 // Implementation of class FE_Evolution
 FE_Evolution::FE_Evolution(BilinearForm &M_, BilinearForm &K_, const Vector &b_)
-   : TimeDependentOperator(M_.FESpace()->GetTrueVSize()),
-     M(M_), K(K_), b(b_), z(height)
+   : TimeDependentOperator(M_.Height()), M(M_), K(K_), b(b_), z(M_.Height())
 {
   Array<int> ess_tdof_list;
   if (M.GetAssemblyLevel() == AssemblyLevel::LEGACY)
@@ -63,66 +63,6 @@ double inflow_function(const Vector &x);
 // Mesh bounding box
 Vector bb_min, bb_max;

-// Type of preconditioner for implicit time integrator
-enum class PrecType : int
-{
-   ILU = 0,
-   AIR = 1
-};
-
-#if MFEM_HYPRE_VERSION >= 21800
-// Algebraic multigrid preconditioner for advective problems based on
-// approximate ideal restriction (AIR). Most effective when matrix is
-// first scaled by DG block inverse, and AIR applied to scaled matrix.
-// See https://doi.org/10.1137/17M1144350.
-class AIR_prec : public Solver
-{
-private:
-   const HypreParMatrix *A;
-   // Copy of A scaled by block-diagonal inverse
-   HypreParMatrix A_s;
-
-   HypreBoomerAMG *AIR_solver;
-   int blocksize;
-
-public:
-   AIR_prec(int blocksize_) : AIR_solver(NULL), blocksize(blocksize_) { }
-
-   void SetOperator(const Operator &op)
-   {
-      width = op.Width();
-      height = op.Height();
-
-      A = dynamic_cast<const HypreParMatrix *>(&op);
-      MFEM_VERIFY(A != NULL, "AIR_prec requires a HypreParMatrix.")
-
-      // Scale A by block-diagonal inverse
-      BlockInverseScale(A, &A_s, NULL, NULL, blocksize,
-                        BlockInverseScaleJob::MATRIX_ONLY);
-      delete AIR_solver;
-      AIR_solver = new HypreBoomerAMG(A_s);
-      AIR_solver->SetAdvectiveOptions(1, "", "FA");
-      AIR_solver->SetPrintLevel(0);
-      AIR_solver->SetMaxLevels(50);
-   }
-
-   virtual void Mult(const Vector &x, Vector &y) const
-   {
-      // Scale the rhs by block inverse and solve system
-      HypreParVector z_s;
-      BlockInverseScale(A, NULL, &x, &z_s, blocksize,
-                        BlockInverseScaleJob::RHS_ONLY);
-      AIR_solver->Mult(z_s, y);
-   }
-
-   ~AIR_prec()
-   {
-      delete AIR_solver;
-   }
-};
-#endif
-
-
 class DG_Solver : public Solver
 {
 private:
@@ -130,37 +70,24 @@ private:
   SparseMatrix M_diag;
   HypreParMatrix *A;
   GMRESSolver linear_solver;
-   Solver *prec;
+   BlockILU prec;
   double dt;
 public:
-   DG_Solver(HypreParMatrix &M_, HypreParMatrix &K_, const FiniteElementSpace &fes,
-             PrecType prec_type)
+   DG_Solver(HypreParMatrix &M_, HypreParMatrix &K_, const FiniteElementSpace &fes)
      : M(M_),
        K(K_),
        A(NULL),
        linear_solver(M.GetComm()),
+        prec(fes.GetFE(0)->GetDof(),
+             BlockILU::Reordering::MINIMUM_DISCARDED_FILL),
        dt(-1.0)
   {
-      int block_size = fes.GetFE(0)->GetDof();
-      if (prec_type == PrecType::ILU)
-      {
-         prec = new BlockILU(block_size,
-                             BlockILU::Reordering::MINIMUM_DISCARDED_FILL);
-      }
-      else if (prec_type == PrecType::AIR)
-      {
-#if MFEM_HYPRE_VERSION >= 21800
-         prec = new AIR_prec(block_size);
-#else
-         MFEM_ABORT("Must have MFEM_HYPRE_VERSION >= 21800 to use AIR.\n");
-#endif
-      }
      linear_solver.iterative_mode = false;
      linear_solver.SetRelTol(1e-9);
      linear_solver.SetAbsTol(0.0);
      linear_solver.SetMaxIter(100);
      linear_solver.SetPrintLevel(0);
-      linear_solver.SetPreconditioner(*prec);
+      linear_solver.SetPreconditioner(prec);

      M.GetDiag(M_diag);
   }
@@ -193,12 +120,10 @@ public:

   ~DG_Solver()
   {
-      delete prec;
      delete A;
   }
 };

-
 /** A time-dependent operator for the right-hand side of the ODE. The DG weak
    form of du/dt = -v.grad(u) is M du/dt = K u + b, where M and K are the mass
    and advection matrices, and b describes the flow on the boundary. This can
@@ -216,8 +141,7 @@ private:
   mutable Vector z;

 public:
-   FE_Evolution(ParBilinearForm &M_, ParBilinearForm &K_, const Vector &b_,
-                PrecType prec_type);
+   FE_Evolution(ParBilinearForm &M_, ParBilinearForm &K_, const Vector &b_);

   virtual void Mult(const Vector &x, Vector &y) const;
   virtual void ImplicitSolve(const double dt, const Vector &x, Vector &k);
@@ -254,11 +178,6 @@ int main(int argc, char *argv[])
   bool adios2 = false;
   bool binary = false;
   int vis_steps = 5;
-#if MFEM_HYPRE_VERSION >= 21800
-   PrecType prec_type = PrecType::AIR;
-#else
-   PrecType prec_type = PrecType::ILU;
-#endif

   // Relative and absolute tolerances for CVODE and ARKODE.
   const double reltol = 1e-2, abstol = 1e-2;
@@ -299,8 +218,6 @@ int main(int argc, char *argv[])
                  "Final time; start time is 0.");
   args.AddOption(&dt, "-dt", "--time-step",
                  "Time step.");
-   args.AddOption((int *)&prec_type, "-pt", "--prec-type", "Preconditioner for "
-                  "implicit solves. 0 for ILU, 1 for pAIR-AMG.");
   args.AddOption(&visualization, "-vis", "--visualization", "-no-vis",
                  "--no-visualization",
                  "Enable or disable GLVis visualization.");
@@ -321,13 +238,13 @@ int main(int argc, char *argv[])
   args.Parse();
   if (!args.Good())
   {
-      if (Mpi::Root())
+      if (myid == 0)
      {
         args.PrintUsage(cout);
      }
      return 1;
   }
-   if (Mpi::Root())
+   if (myid == 0)
   {
      args.PrintOptions(cout);
   }
@@ -335,7 +252,7 @@ int main(int argc, char *argv[])
   // check for valid ODE solver option
   if (ode_solver_type < 1 || ode_solver_type > 9)
   {
-      if (Mpi::Root())
+      if (myid == 0)
      {
         cout << "Unknown ODE solver type: " << ode_solver_type << '\n';
      }
@@ -343,7 +260,7 @@ int main(int argc, char *argv[])
   }

   Device device(device_config);
-   if (Mpi::Root()) { device.Print(); }
+   if (myid == 0) { device.Print(); }

   // 3. Read the serial mesh from the given mesh file on all processors. We can
   //    handle geometrically periodic meshes in this code.
@@ -380,7 +297,7 @@ int main(int argc, char *argv[])
   ParFiniteElementSpace *fes = new ParFiniteElementSpace(pmesh, &fec);

   HYPRE_BigInt global_vSize = fes->GlobalTrueVSize();
-   if (Mpi::Root())
+   if (myid == 0)
   {
      cout << "Number of unknowns: " << global_vSize << endl;
   }
@@ -411,16 +328,15 @@ int main(int argc, char *argv[])
   }

   m->AddDomainIntegrator(new MassIntegrator);
-   constexpr double alpha = -1.0;
-   k->AddDomainIntegrator(new ConvectionIntegrator(velocity, alpha));
+   k->AddDomainIntegrator(new ConvectionIntegrator(velocity, -1.0));
   k->AddInteriorFaceIntegrator(
-      new NonconservativeDGTraceIntegrator(velocity, alpha));
+      new TransposeIntegrator(new DGTraceIntegrator(velocity, 1.0, -0.5)));
   k->AddBdrFaceIntegrator(
-      new NonconservativeDGTraceIntegrator(velocity, alpha));
+      new TransposeIntegrator(new DGTraceIntegrator(velocity, 1.0, -0.5)));

   ParLinearForm *b = new ParLinearForm(fes);
   b->AddBdrFaceIntegrator(
-      new BoundaryFlowIntegrator(inflow, velocity, alpha));
+      new BoundaryFlowIntegrator(inflow, velocity, -1.0, -0.5));

   int skip_zeros = 0;
   m->Assemble();
@@ -519,13 +435,11 @@ int main(int argc, char *argv[])
      sout.open(vishost, visport);
      if (!sout)
      {
-         if (Mpi::Root())
-         {
+         if (myid == 0)
            cout << "Unable to connect to GLVis server at "
                 << vishost << ':' << visport << endl;
-         }
         visualization = false;
-         if (Mpi::Root())
+         if (myid == 0)
         {
            cout << "GLVis visualization disabled.\n";
         }
@@ -537,17 +451,15 @@ int main(int argc, char *argv[])
         sout << "solution\n" << *pmesh << *u;
         sout << "pause\n";
         sout << flush;
-         if (Mpi::Root())
-         {
+         if (myid == 0)
            cout << "GLVis visualization paused."
                 << " Press space (in the GLVis window) to resume it.\n";
-         }
      }
   }

   // 9. Define the time-dependent evolution operator describing the ODE
   //    right-hand side, and define the ODE solver used for time integration.
-   FE_Evolution adv(*m, *k, *B, prec_type);
+   FE_Evolution adv(*m, *k, *B);

   double t = 0.0;
   adv.SetTime(t);
@@ -599,7 +511,7 @@ int main(int argc, char *argv[])

      if (done || ti % vis_steps == 0)
      {
-         if (Mpi::Root())
+         if (myid == 0)
         {
            cout << "time step: " << ti << ", time: " << t << endl;
            if (cvode) { cvode->PrintInfo(); }
@@ -678,11 +590,11 @@ int main(int argc, char *argv[])

 // Implementation of class FE_Evolution
 FE_Evolution::FE_Evolution(ParBilinearForm &M_, ParBilinearForm &K_,
-                           const Vector &b_, PrecType prec_type)
-   : TimeDependentOperator(M_.ParFESpace()->GetTrueVSize()),
+                           const Vector &b_)
+   : TimeDependentOperator(M_.Height()),
     b(b_),
     M_solver(M_.ParFESpace()->GetComm()),
-     z(height)
+     z(M_.Height())
 {
   if (M_.GetAssemblyLevel()==AssemblyLevel::LEGACY)
   {
@@ -705,7 +617,7 @@ FE_Evolution::FE_Evolution(ParBilinearForm &M_, ParBilinearForm &K_,
      HypreSmoother *hypre_prec = new HypreSmoother(M_mat, HypreSmoother::Jacobi);
      M_prec = hypre_prec;

-      dg_solver = new DG_Solver(M_mat, K_mat, *M_.FESpace(), prec_type);
+      dg_solver = new DG_Solver(M_mat, K_mat, *M_.FESpace());
   }
   else
   {
@@ -721,10 +633,6 @@ FE_Evolution::FE_Evolution(ParBilinearForm &M_, ParBilinearForm &K_,
   M_solver.SetPrintLevel(0);
 }

-// Solve the equation:
-//    u_t = M^{-1}(Ku + b),
-// by solving associated linear system
-//    (M - dt*K) d = K*u + b
 void FE_Evolution::ImplicitSolve(const double dt, const Vector &x, Vector &k)
 {
   K->Mult(x, z);
@@ -23,8 +23,6 @@ MFEM_LIB_FILE = mfem_is_not_built

 SEQ_EXAMPLES = ex9 ex10 ex16
 PAR_EXAMPLES = ex9p ex10p ex16p
-SEQ_DEVICE_EXAMPLES = ex9
-PAR_DEVICE_EXAMPLES = ex9p
 ifeq ($(MFEM_USE_MPI),NO)
   EXAMPLES = $(SEQ_EXAMPLES)
 else
@@ -56,22 +54,10 @@ include $(MFEM_TEST_MK)
 RUN_MPI = $(MFEM_MPIEXEC) $(MFEM_MPIEXEC_NP) $(MFEM_MPI_NP)
 SERIAL_NAME := Serial SUNDIALS example
 PARALLEL_NAME := Parallel SUNDIALS example
-SERIAL_CUDA_NAME := Serial SUNDIALS CUDA example
-PARALLEL_CUDA_NAME := Parallel SUNDIALS CUDA example
-SERIAL_HIP_NAME := Serial SUNDIALS HIP example
-PARALLEL_HIP_NAME := Parallel SUNDIALS HIP example
 %-test-par: %
 	@$(call mfem-test,$<, $(RUN_MPI), $(PARALLEL_NAME))
 %-test-seq: %
 	@$(call mfem-test,$<,, $(SERIAL_NAME))
-%-test-par-cuda: %
-	@$(call mfem-test,$<, $(RUN_MPI), $(PARALLEL_CUDA_NAME),-d cuda)
-%-test-seq-cuda: %
-	@$(call mfem-test,$<,, $(SERIAL_CUDA_NAME),-d cuda)
-%-test-par-hip: %
-	@$(call mfem-test,$<, $(RUN_MPI), $(PARALLEL_HIP_NAME),-d hip)
-%-test-seq-hip: %
-	@$(call mfem-test,$<,, $(SERIAL_HIP_NAME),-d hip)

 # Testing: Specific execution options:
 # Example 9: test CVODE with CV_ADAMS (non-stiff implicit) time stepping
@@ -82,16 +68,6 @@ ex9-test-seq: ex9
 	@$(call mfem-test,$<,, $(SERIAL_NAME),$(EX9_ARGS))
 ex9p-test-par: ex9p
 	@$(call mfem-test,$<, $(RUN_MPI), $(PARALLEL_NAME),$(EX9P_ARGS))
-ex9-test-seq-cuda: ex9
-	@$(call mfem-test,$<,, $(SERIAL_CUDA_NAME),-d cuda $(EX9_ARGS))
-ex9p-test-par-cuda: ex9p
-	@$(call mfem-test,$<, $(RUN_MPI), $(PARALLEL_CUDA_NAME),-d cuda \
-	   $(EX9P_ARGS))
-ex9-test-seq-hip: ex9
-	@$(call mfem-test,$<,, $(SERIAL_HIP_NAME),-d hip $(EX9_ARGS))
-ex9p-test-par-hip: ex9p
-	@$(call mfem-test,$<, $(RUN_MPI), $(PARALLEL_HIP_NAME),-d hip \
-	   $(EX9P_ARGS))
 # Example 10: test CVODE with CV_BDF (stiff implicit) time stepping
 EX10_COMMON_ARGS := -m ../../data/beam-quad.mesh -o 2 -s 5 -dt 0.15 -tf 6 -vs 10
 EX10_ARGS  := $(EX10_COMMON_ARGS) -r  2
@@ -67,7 +67,6 @@ int main(int argc, char *argv[])
   int slu_colperm = 4;
   int slu_rowperm = 1;
   int slu_iterref = 2;
-   int slu_npdep = 1;

   OptionsParser args(argc, argv);
   args.AddOption(&mesh_file, "-m", "--mesh",
@@ -86,11 +85,9 @@ int main(int argc, char *argv[])
                  "6-ZOLTAN");
   args.AddOption(&slu_rowperm, "-rp", "--rowperm",
                  "SuperLU Row Permutation Method:  0-NOROWPERM, 1-LargeDiag");
-   args.AddOption(&slu_iterref, "-ir", "--iterref",
+   args.AddOption(&slu_iterref, "-rp", "--rowperm",
                  "SuperLU Iterative Refinement:  0-NOREFINE, 1-Single, "
                  "2-Double, 3-Extra");
-   args.AddOption(&slu_npdep, "-npdep", "--npdepth",
-                  "Depth of 3D parition for SuperLU (>= 7.2.0)");

   args.Parse();
   if (!args.Good())
@@ -217,7 +214,7 @@ int main(int argc, char *argv[])
   a.FormLinearSystem(ess_tdof_list, x, b, A, X, B);

   // 13. Solve the linear system A X = B utilizing SuperLU.
-   SuperLUSolver *superlu = new SuperLUSolver(MPI_COMM_WORLD, slu_npdep);
+   SuperLUSolver *superlu = new SuperLUSolver(MPI_COMM_WORLD);
   Operator *SLU_A = new SuperLURowLocMatrix(*A.As<HypreParMatrix>());
   superlu->SetPrintStatistics(true);
   superlu->SetSymmetricPattern(false);
@@ -284,9 +281,10 @@ int main(int argc, char *argv[])
   superlu->SetOperator(*SLU_A);
   superlu->SetPrintStatistics(true);
   superlu->Mult(B, X);
+   superlu->DismantleGrid();

-   delete superlu;
   delete SLU_A;
+   delete superlu;

   // 14. Recover the parallel grid function corresponding to X. This is the
   //     local finite element solution on each processor.
@@ -13,45 +13,28 @@ set(SRCS
  bilinearform.cpp
  bilinearform_ext.cpp
  bilininteg.cpp
-  integ/bilininteg_br2.cpp
-  integ/bilininteg_convection_mf.cpp
-  integ/bilininteg_convection_pa.cpp
-  integ/bilininteg_convection_ea.cpp
-  integ/bilininteg_curlcurl_pa.cpp
-  integ/bilininteg_dgtrace_pa.cpp
-  integ/bilininteg_dgtrace_ea.cpp
-  integ/bilininteg_diffusion_mf.cpp
-  integ/bilininteg_diffusion_pa.cpp
-  integ/bilininteg_diffusion_ea.cpp
-  integ/bilininteg_diffusion_patch.cpp
-  integ/bilininteg_divdiv_pa.cpp
-  integ/bilininteg_gradient_pa.cpp
-  integ/bilininteg_interp_pa.cpp
-  integ/bilininteg_mass_mf.cpp
-  integ/bilininteg_mass_pa.cpp
-  integ/bilininteg_mass_ea.cpp
-  integ/bilininteg_mixedcurl_pa.cpp
-  integ/bilininteg_mixedvecgrad_pa.cpp
-  integ/bilininteg_transpose_ea.cpp
-  integ/bilininteg_vecdiffusion_mf.cpp
-  integ/bilininteg_vecdiffusion_pa.cpp
-  integ/bilininteg_vecdiv_pa.cpp
-  integ/bilininteg_vecmass_mf.cpp
-  integ/bilininteg_vecmass_pa.cpp
-  integ/bilininteg_vectorfediv_pa.cpp
-  integ/bilininteg_vectorfemass_pa.cpp
-  integ/bilininteg_diffusion_kernels.cpp
-  integ/bilininteg_hcurl_kernels.cpp
-  integ/bilininteg_hdiv_kernels.cpp
-  integ/bilininteg_hcurlhdiv_kernels.cpp
-  integ/bilininteg_mass_kernels.cpp
-  integ/lininteg_boundary.cpp
-  integ/lininteg_boundary_flux.cpp
-  integ/lininteg_domain.cpp
-  integ/lininteg_domain_grad.cpp
-  integ/lininteg_domain_vectorfe.cpp
-  integ/nonlininteg_vecconvection_pa.cpp
-  integ/nonlininteg_vecconvection_mf.cpp
+  bilininteg_br2.cpp
+  bilininteg_convection_mf.cpp
+  bilininteg_convection_pa.cpp
+  bilininteg_convection_ea.cpp
+  bilininteg_dgtrace_pa.cpp
+  bilininteg_dgtrace_ea.cpp
+  bilininteg_diffusion_mf.cpp
+  bilininteg_diffusion_pa.cpp
+  bilininteg_diffusion_ea.cpp
+  bilininteg_divergence.cpp
+  bilininteg_hcurl.cpp
+  bilininteg_hdiv.cpp
+  bilininteg_vectorfe.cpp
+  bilininteg_gradient.cpp
+  bilininteg_mass_mf.cpp
+  bilininteg_mass_pa.cpp
+  bilininteg_mass_ea.cpp
+  bilininteg_transpose_ea.cpp
+  bilininteg_vecdiffusion.cpp
+  bilininteg_vecdiffusion_mf.cpp
+  bilininteg_vecmass.cpp
+  bilininteg_vecmass_mf.cpp
  coefficient.cpp
  complex_fem.cpp
  convergence.cpp
@@ -88,10 +71,14 @@ set(SRCS
  ceed/solvers/algebraic.cpp
  ceed/solvers/full-assembly.cpp
  ceed/solvers/solvers-atpmg.cpp
-  kdtree.cpp
  linearform.cpp
  linearform_ext.cpp
  lininteg.cpp
+  lininteg_boundary.cpp
+  lininteg_boundary_flux.cpp
+  lininteg_domain.cpp
+  lininteg_domain_grad.cpp
+  lininteg_vectorfe_domain.cpp
  lor/lor.cpp
  lor/lor_ads.cpp
  lor/lor_ams.cpp
@@ -104,6 +91,8 @@ set(SRCS
  nonlinearform_ext.cpp
  nonlininteg.cpp
  fespacehierarchy.cpp
+  nonlininteg_vectorconvection.cpp
+  nonlininteg_vectorconvection_mf.cpp
  qfunction.cpp
  qinterp/det.cpp
  qinterp/eval_by_nodes.cpp
@@ -154,11 +143,7 @@ set(HDRS
  bilinearform.hpp
  bilinearform_ext.hpp
  bilininteg.hpp
-  integ/bilininteg_diffusion_kernels.hpp
-  integ/bilininteg_hcurl_kernels.hpp
-  integ/bilininteg_hdiv_kernels.hpp
-  integ/bilininteg_hcurlhdiv_kernels.hpp
-  integ/bilininteg_mass_kernels.hpp
+  bilininteg_mass_pa.hpp
  coefficient.hpp
  complex_fem.hpp
  convergence.hpp
@@ -200,7 +185,6 @@ set(HDRS
  ceed/solvers/algebraic.hpp
  ceed/solvers/full-assembly.hpp
  ceed/solvers/solvers-atpmg.hpp
-  kdtree.hpp
  linearform.hpp
  linearform_ext.hpp
  lininteg.hpp
@@ -13,7 +13,6 @@

 #include "fem.hpp"
 #include "../general/device.hpp"
-#include "../mesh/nurbs.hpp"
 #include <cmath>

 namespace mfem
@@ -422,17 +421,11 @@ void BilinearForm::Assemble(int skip_zeros)
                        "invalid element marker for domain integrator #"
                        << k << ", counting from zero");
         }
-
-         if (domain_integs[k]->Patchwise())
-         {
-            MFEM_VERIFY(fes->GetNURBSext(), "Patchwise integration requires a "
-                        << "NURBS FE space");
-         }
      }

-      // Element-wise integration
      for (int i = 0; i < fes -> GetNE(); i++)
      {
+         int elem_attr = fes->GetMesh()->GetAttribute(i);
         doftrans = fes->GetElementVDofs(i, vdofs);
         if (element_matrices)
         {
@@ -440,13 +433,11 @@ void BilinearForm::Assemble(int skip_zeros)
         }
         else
         {
-            const int elem_attr = fes->GetMesh()->GetAttribute(i);
            elmat.SetSize(0);
            for (int k = 0; k < domain_integs.Size(); k++)
            {
-               if ((domain_integs_marker[k] == NULL ||
+               if ( domain_integs_marker[k] == NULL ||
                    (*(domain_integs_marker[k]))[elem_attr-1] == 1)
-                   && !domain_integs[k]->Patchwise())
               {
                  const FiniteElement &fe = *fes->GetFE(i);
                  eltrans = fes->GetElementTransformation(i);
@@ -488,43 +479,6 @@ void BilinearForm::Assemble(int skip_zeros)
            }
         }
      }
-
-      // Patch-wise integration
-      if (fes->GetNURBSext())
-      {
-         for (int p=0; p<mesh->NURBSext->GetNP(); ++p)
-         {
-            bool vdofsSet = false;
-            for (int k = 0; k < domain_integs.Size(); k++)
-            {
-               if (domain_integs[k]->Patchwise())
-               {
-                  if (!vdofsSet)
-                  {
-                     fes->GetPatchVDofs(p, vdofs);
-                     vdofsSet = true;
-                  }
-
-                  SparseMatrix* spmat = nullptr;
-                  domain_integs[k]->AssemblePatchMatrix(p, *fes, spmat);
-                  Array<int> cols;
-                  Vector srow;
-
-                  for (int r=0; r<spmat->Height(); ++r)
-                  {
-                     spmat->GetRow(r, cols, srow);
-                     for (int i=0; i<cols.Size(); ++i)
-                     {
-                        cols[i] = vdofs[cols[i]];
-                     }
-                     mat->AddRow(vdofs[r], cols, srow);
-                  }
-
-                  delete spmat;
-               }
-            }
-         }
-      }
   }

   if (boundary_integs.Size())
@@ -254,12 +254,6 @@ public:
   /// Access all the integrators added with AddDomainIntegrator().
   Array<BilinearFormIntegrator*> *GetDBFI() { return &domain_integs; }

-   /// @brief Access all boundary markers added with AddDomainIntegrator().
-   ///
-   /// If no marker was specified when the integrator was added, the
-   /// corresponding pointer (to Array<int>) will be NULL. */
-   Array<Array<int>*> *GetDBFI_Marker() { return &domain_integs_marker; }
-
   /// Access all the integrators added with AddBoundaryIntegrator().
   Array<BilinearFormIntegrator*> *GetBBFI() { return &boundary_integs; }
   /** @brief Access all boundary markers added with AddBoundaryIntegrator().
@@ -458,7 +452,7 @@ public:
       practice it is convenient to have it in transposed form for
       construction of RAP operators in matrix-free methods. */
   virtual const Operator *GetOutputRestrictionTranspose() const
-   { return fes->GetRestrictionTransposeOperator(); }
+   { return GetOutputProlongation(); }
   /// Get the output finite element space restriction matrix
   virtual const Operator *GetOutputRestriction() const
   { return GetRestriction(); }
@@ -56,9 +56,6 @@ void MFBilinearFormExtension::Assemble()
   {
      integrators[i]->AssembleMF(*a->FESpace());
   }
-
-   MFEM_VERIFY(a->GetBBFI()->Size() == 0, "AddBoundaryIntegrator is not "
-               "currently supported in MFBilinearFormExtension");
 }

 void MFBilinearFormExtension::AssembleDiagonal(Vector &y) const
@@ -264,14 +261,6 @@ void PABilinearFormExtension::SetupRestrictionOperators(const L2FaceValues m)
      localX.SetSize(elem_restrict->Height(), Device::GetDeviceMemoryType());
      localY.SetSize(elem_restrict->Height(), Device::GetDeviceMemoryType());
      localY.UseDevice(true); // ensure 'localY = 0.0' is done on device
-
-      // Gather the attributes on the host from all the elements
-      const Mesh &mesh = *trial_fes->GetMesh();
-      elem_attributes.SetSize(mesh.GetNE());
-      for (int i = 0; i < mesh.GetNE(); ++i)
-      {
-         elem_attributes[i] = mesh.GetAttribute(i);
-      }
   }

   // Construct face restriction operators only if the bilinear form has
@@ -286,9 +275,7 @@ void PABilinearFormExtension::SetupRestrictionOperators(const L2FaceValues m)
      int_face_Y.UseDevice(true); // ensure 'int_face_Y = 0.0' is done on device
   }

-   const bool has_bdr_integs = (a->GetBFBFI()->Size() > 0 ||
-                                a->GetBBFI()->Size() > 0);
-   if (bdr_face_restrict_lex == NULL && has_bdr_integs)
+   if (bdr_face_restrict_lex == NULL && a->GetBFBFI()->Size() > 0)
   {
      bdr_face_restrict_lex = trial_fes->GetFaceRestriction(
                                 ElementDofOrdering::LEXICOGRAPHIC,
@@ -297,46 +284,6 @@ void PABilinearFormExtension::SetupRestrictionOperators(const L2FaceValues m)
      bdr_face_X.SetSize(bdr_face_restrict_lex->Height(), Device::GetMemoryType());
      bdr_face_Y.SetSize(bdr_face_restrict_lex->Height(), Device::GetMemoryType());
      bdr_face_Y.UseDevice(true); // ensure 'faceBoundY = 0.0' is done on device
-
-      const Mesh &mesh = *trial_fes->GetMesh();
-      // See LinearFormExtension::Update for explanation of f_to_be logic.
-      std::unordered_map<int,int> f_to_be;
-      for (int i = 0; i < mesh.GetNBE(); ++i)
-      {
-         const int f = mesh.GetBdrElementEdgeIndex(i);
-         f_to_be[f] = i;
-      }
-      const int nf_bdr = trial_fes->GetNFbyType(FaceType::Boundary);
-      bdr_attributes.SetSize(nf_bdr);
-      int f_ind = 0;
-      int missing_bdr_elems = 0;
-      for (int f = 0; f < mesh.GetNumFaces(); ++f)
-      {
-         if (!mesh.GetFaceInformation(f).IsOfFaceType(FaceType::Boundary))
-         {
-            continue;
-         }
-         int attribute = 1; // default value
-         if (f_to_be.find(f) != f_to_be.end())
-         {
-            const int be = f_to_be[f];
-            attribute = mesh.GetBdrAttribute(be);
-         }
-         else
-         {
-            // If a boundary face does not correspond to the a boundary element,
-            // we assign it the default attribute of 1. We also generate a
-            // warning at runtime with the number of such missing elements.
-            ++missing_bdr_elems;
-         }
-         bdr_attributes[f_ind] = attribute;
-         ++f_ind;
-      }
-      if (missing_bdr_elems)
-      {
-         MFEM_WARNING("Missing " << missing_bdr_elems << " boundary elements "
-                      "for boundary faces.");
-      }
   }
 }

@@ -345,36 +292,27 @@ void PABilinearFormExtension::Assemble()
   SetupRestrictionOperators(L2FaceValues::DoubleValued);

   Array<BilinearFormIntegrator*> &integrators = *a->GetDBFI();
-   for (BilinearFormIntegrator *integ : integrators)
+   const int integratorCount = integrators.Size();
+   for (int i = 0; i < integratorCount; ++i)
   {
-      if (integ->Patchwise())
-      {
-         MFEM_VERIFY(a->FESpace()->GetNURBSext(),
-                     "Patchwise integration requires a NURBS FE space");
-         integ->AssembleNURBSPA(*a->FESpace());
-      }
-      else
-      {
-         integ->AssemblePA(*a->FESpace());
-      }
+      integrators[i]->AssemblePA(*a->FESpace());
   }

-   Array<BilinearFormIntegrator*> &bdr_integrators = *a->GetBBFI();
-   for (BilinearFormIntegrator *integ : bdr_integrators)
-   {
-      integ->AssemblePABoundary(*a->FESpace());
-   }
+   MFEM_VERIFY(a->GetBBFI()->Size() == 0,
+               "Partial assembly does not support AddBoundaryIntegrator yet.");

   Array<BilinearFormIntegrator*> &intFaceIntegrators = *a->GetFBFI();
-   for (BilinearFormIntegrator *integ : intFaceIntegrators)
+   const int intFaceIntegratorCount = intFaceIntegrators.Size();
+   for (int i = 0; i < intFaceIntegratorCount; ++i)
   {
-      integ->AssemblePAInteriorFaces(*a->FESpace());
+      intFaceIntegrators[i]->AssemblePAInteriorFaces(*a->FESpace());
   }

   Array<BilinearFormIntegrator*> &bdrFaceIntegrators = *a->GetBFBFI();
-   for (BilinearFormIntegrator *integ : bdrFaceIntegrators)
+   const int boundFaceIntegratorCount = bdrFaceIntegrators.Size();
+   for (int i = 0; i < boundFaceIntegratorCount; ++i)
   {
-      integ->AssemblePABoundaryFaces(*a->FESpace());
+      bdrFaceIntegrators[i]->AssemblePABoundaryFaces(*a->FESpace());
   }
 }

@@ -385,27 +323,20 @@ void PABilinearFormExtension::AssembleDiagonal(Vector &y) const
   const int iSz = integrators.Size();
   if (elem_restrict && !DeviceCanUseCeed())
   {
-      if (iSz > 0)
+      localY = 0.0;
+      for (int i = 0; i < iSz; ++i)
      {
-         localY = 0.0;
-         for (int i = 0; i < iSz; ++i)
-         {
-            integrators[i]->AssembleDiagonalPA(localY);
-         }
-         const ElementRestriction* H1elem_restrict =
-            dynamic_cast<const ElementRestriction*>(elem_restrict);
-         if (H1elem_restrict)
-         {
-            H1elem_restrict->MultTransposeUnsigned(localY, y);
-         }
-         else
-         {
-            elem_restrict->MultTranspose(localY, y);
-         }
+         integrators[i]->AssembleDiagonalPA(localY);
+      }
+      const ElementRestriction* H1elem_restrict =
+         dynamic_cast<const ElementRestriction*>(elem_restrict);
+      if (H1elem_restrict)
+      {
+         H1elem_restrict->MultTransposeUnsigned(localY, y);
      }
      else
      {
-         y = 0.0;
+         elem_restrict->MultTranspose(localY, y);
      }
   }
   else
@@ -417,18 +348,6 @@ void PABilinearFormExtension::AssembleDiagonal(Vector &y) const
         integrators[i]->AssembleDiagonalPA(y);
      }
   }
-
-   Array<BilinearFormIntegrator*> &bdr_integs = *a->GetBBFI();
-   const int n_bdr_integs = bdr_integs.Size();
-   if (bdr_face_restrict_lex && n_bdr_integs > 0)
-   {
-      bdr_face_Y = 0.0;
-      for (int i = 0; i < n_bdr_integs; ++i)
-      {
-         bdr_integs[i]->AssembleDiagonalPA(bdr_face_Y);
-      }
-      bdr_face_restrict_lex->AddMultTransposeUnsigned(bdr_face_Y, y);
-   }
 }

 void PABilinearFormExtension::Update()
@@ -467,59 +386,24 @@ void PABilinearFormExtension::Mult(const Vector &x, Vector &y) const
   Array<BilinearFormIntegrator*> &integrators = *a->GetDBFI();

   const int iSz = integrators.Size();
-
-   bool allPatchwise = true;
-   bool somePatchwise = false;
-
-   for (int i = 0; i < iSz; ++i)
-   {
-      if (integrators[i]->Patchwise())
-      {
-         somePatchwise = true;
-      }
-      else
-      {
-         allPatchwise = false;
-      }
-   }
-
-   MFEM_VERIFY(!(somePatchwise && !allPatchwise),
-               "All or none of the integrators should be patchwise");
-
-   if (DeviceCanUseCeed() || !elem_restrict || allPatchwise)
+   if (DeviceCanUseCeed() || !elem_restrict)
   {
      y.UseDevice(true); // typically this is a large vector, so store on device
      y = 0.0;
      for (int i = 0; i < iSz; ++i)
      {
-         if (integrators[i]->Patchwise())
-         {
-            integrators[i]->AddMultNURBSPA(x, y);
-         }
-         else
-         {
-            integrators[i]->AddMultPA(x, y);
-         }
+         integrators[i]->AddMultPA(x, y);
      }
   }
   else
   {
-      if (iSz)
+      elem_restrict->Mult(x, localX);
+      localY = 0.0;
+      for (int i = 0; i < iSz; ++i)
      {
-         Array<Array<int>*> &elem_markers = *a->GetDBFI_Marker();
-         elem_restrict->Mult(x, localX);
-         localY = 0.0;
-         for (int i = 0; i < iSz; ++i)
-         {
-            AddMultWithMarkers(*integrators[i], localX, elem_markers[i], elem_attributes,
-                               false, localY);
-         }
-         elem_restrict->MultTranspose(localY, y);
-      }
-      else
-      {
-         y = 0.0;
+         integrators[i]->AddMultPA(localX, localY);
      }
+      elem_restrict->MultTranspose(localY, y);
   }

   Array<BilinearFormIntegrator*> &intFaceIntegrators = *a->GetFBFI();
@@ -538,28 +422,17 @@ void PABilinearFormExtension::Mult(const Vector &x, Vector &y) const
      }
   }

-   Array<BilinearFormIntegrator*> &bdr_integs = *a->GetBBFI();
-   Array<BilinearFormIntegrator*> &bdr_face_integs = *a->GetBFBFI();
-   const int n_bdr_integs = bdr_integs.Size();
-   const int n_bdr_face_integs = bdr_face_integs.Size();
-   const bool has_bdr_integs = (n_bdr_face_integs > 0 || n_bdr_integs > 0);
-   if (bdr_face_restrict_lex && has_bdr_integs)
+   Array<BilinearFormIntegrator*> &bdrFaceIntegrators = *a->GetBFBFI();
+   const int bFISz = bdrFaceIntegrators.Size();
+   if (bdr_face_restrict_lex && bFISz>0)
   {
-      Array<Array<int>*> &bdr_markers = *a->GetBBFI_Marker();
-      Array<Array<int>*> &bdr_face_markers = *a->GetBFBFI_Marker();
      bdr_face_restrict_lex->Mult(x, bdr_face_X);
      if (bdr_face_X.Size()>0)
      {
         bdr_face_Y = 0.0;
-         for (int i = 0; i < n_bdr_integs; ++i)
+         for (int i = 0; i < bFISz; ++i)
         {
-            AddMultWithMarkers(*bdr_integs[i], bdr_face_X, bdr_markers[i], bdr_attributes,
-                               false, bdr_face_Y);
-         }
-         for (int i = 0; i < n_bdr_face_integs; ++i)
-         {
-            AddMultWithMarkers(*bdr_face_integs[i], bdr_face_X, bdr_face_markers[i],
-                               bdr_attributes, false, bdr_face_Y);
+            bdrFaceIntegrators[i]->AddMultPA(bdr_face_X, bdr_face_Y);
         }
         bdr_face_restrict_lex->AddMultTransposeInPlace(bdr_face_Y, y);
      }
@@ -572,13 +445,11 @@ void PABilinearFormExtension::MultTranspose(const Vector &x, Vector &y) const
   const int iSz = integrators.Size();
   if (elem_restrict)
   {
-      Array<Array<int>*> &elem_markers = *a->GetDBFI_Marker();
      elem_restrict->Mult(x, localX);
      localY = 0.0;
      for (int i = 0; i < iSz; ++i)
      {
-         AddMultWithMarkers(*integrators[i], localX, elem_markers[i], elem_attributes,
-                            true, localY);
+         integrators[i]->AddMultTransposePA(localX, localY);
      }
      elem_restrict->MultTranspose(localY, y);
   }
@@ -608,85 +479,23 @@ void PABilinearFormExtension::MultTranspose(const Vector &x, Vector &y) const
      }
   }

-   Array<BilinearFormIntegrator*> &bdr_integs = *a->GetBBFI();
-   Array<BilinearFormIntegrator*> &bdr_face_integs = *a->GetBFBFI();
-   const int n_bdr_integs = bdr_integs.Size();
-   const int n_bdr_face_integs = bdr_face_integs.Size();
-   const bool has_bdr_integs = (n_bdr_face_integs > 0 || n_bdr_integs > 0);
-   if (bdr_face_restrict_lex && has_bdr_integs)
+   Array<BilinearFormIntegrator*> &bdrFaceIntegrators = *a->GetBFBFI();
+   const int bFISz = bdrFaceIntegrators.Size();
+   if (bdr_face_restrict_lex && bFISz>0)
   {
-      Array<Array<int>*> &bdr_markers = *a->GetBBFI_Marker();
-      Array<Array<int>*> &bdr_face_markers = *a->GetBFBFI_Marker();
-
      bdr_face_restrict_lex->Mult(x, bdr_face_X);
-      if (bdr_face_X.Size() > 0)
+      if (bdr_face_X.Size()>0)
      {
         bdr_face_Y = 0.0;
-         for (int i = 0; i < n_bdr_integs; ++i)
+         for (int i = 0; i < bFISz; ++i)
         {
-            AddMultWithMarkers(*bdr_integs[i], bdr_face_X, bdr_markers[i], bdr_attributes,
-                               true, bdr_face_Y);
-         }
-         for (int i = 0; i < n_bdr_face_integs; ++i)
-         {
-            AddMultWithMarkers(*bdr_face_integs[i], bdr_face_X, bdr_face_markers[i],
-                               bdr_attributes, true, bdr_face_Y);
+            bdrFaceIntegrators[i]->AddMultTransposePA(bdr_face_X, bdr_face_Y);
         }
         bdr_face_restrict_lex->AddMultTransposeInPlace(bdr_face_Y, y);
      }
   }
 }

-// Compute kernels for PABilinearFormExtension::AddMultWithMarkers.
-// Cannot be in member function with non-public visibility.
-static void AddWithMarkers_(
-   const int ne,
-   const int nd,
-   const Vector &x,
-   const Array<int> &markers,
-   const Array<int> &attributes,
-   Vector &y)
-{
-   const auto d_x = Reshape(x.Read(), nd, ne);
-   const auto d_m = Reshape(markers.Read(), markers.Size());
-   const auto d_attr = Reshape(attributes.Read(), ne);
-   auto d_y = Reshape(y.ReadWrite(), nd, ne);
-   mfem::forall(ne, [=] MFEM_HOST_DEVICE (int e)
-   {
-      const int attr = d_attr[e];
-      if (d_m[attr - 1] == 0) { return; }
-      for (int i = 0; i < nd; ++i)
-      {
-         d_y(i, e) += d_x(i, e);
-      }
-   });
-}
-
-void PABilinearFormExtension::AddMultWithMarkers(
-   const BilinearFormIntegrator &integ,
-   const Vector &x,
-   const Array<int> *markers,
-   const Array<int> &attributes,
-   const bool transpose,
-   Vector &y) const
-{
-   if (markers)
-   {
-      tmp_evec.SetSize(y.Size());
-      tmp_evec = 0.0;
-      if (transpose) { integ.AddMultTransposePA(x, tmp_evec); }
-      else { integ.AddMultPA(x, tmp_evec); }
-      const int ne = attributes.Size();
-      const int nd = x.Size() / ne;
-      AddWithMarkers_(ne, nd, tmp_evec, *markers, attributes, y);
-   }
-   else
-   {
-      if (transpose) { integ.AddMultTransposePA(x, y); }
-      else { integ.AddMultPA(x, y); }
-   }
-}
-
 // Data and methods for element-assembled bilinear forms
 EABilinearFormExtension::EABilinearFormExtension(BilinearForm *form)
   : PABilinearFormExtension(form),
@@ -787,7 +596,7 @@ void EABilinearFormExtension::Mult(const Vector &x, Vector &y) const
      auto X = Reshape(useRestrict?localX.Read():x.Read(), NDOFS, ne);
      auto Y = Reshape(useRestrict?localY.ReadWrite():y.ReadWrite(), NDOFS, ne);
      auto A = Reshape(ea_data.Read(), NDOFS, NDOFS, ne);
-      mfem::forall(ne*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+      MFEM_FORALL(glob_j, ne*NDOFS,
      {
         const int e = glob_j/NDOFS;
         const int j = glob_j%NDOFS;
@@ -822,7 +631,7 @@ void EABilinearFormExtension::Mult(const Vector &x, Vector &y) const
         if (!factorize_face_terms)
         {
            auto A_int = Reshape(ea_data_int.Read(), NDOFS, NDOFS, 2, nf_int);
-            mfem::forall(nf_int*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+            MFEM_FORALL(glob_j, nf_int*NDOFS,
            {
               const int f = glob_j/NDOFS;
               const int j = glob_j%NDOFS;
@@ -841,7 +650,7 @@ void EABilinearFormExtension::Mult(const Vector &x, Vector &y) const
            });
         }
         auto A_ext = Reshape(ea_data_ext.Read(), NDOFS, NDOFS, 2, nf_int);
-         mfem::forall(nf_int*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+         MFEM_FORALL(glob_j, nf_int*NDOFS,
         {
            const int f = glob_j/NDOFS;
            const int j = glob_j%NDOFS;
@@ -878,7 +687,7 @@ void EABilinearFormExtension::Mult(const Vector &x, Vector &y) const
         auto X = Reshape(bdr_face_X.Read(), NDOFS, nf_bdr);
         auto Y = Reshape(bdr_face_Y.ReadWrite(), NDOFS, nf_bdr);
         auto A = Reshape(ea_data_bdr.Read(), NDOFS, NDOFS, nf_bdr);
-         mfem::forall(nf_bdr*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+         MFEM_FORALL(glob_j, nf_bdr*NDOFS,
         {
            const int f = glob_j/NDOFS;
            const int j = glob_j%NDOFS;
@@ -915,7 +724,7 @@ void EABilinearFormExtension::MultTranspose(const Vector &x, Vector &y) const
      auto X = Reshape(useRestrict?localX.Read():x.Read(), NDOFS, ne);
      auto Y = Reshape(useRestrict?localY.ReadWrite():y.ReadWrite(), NDOFS, ne);
      auto A = Reshape(ea_data.Read(), NDOFS, NDOFS, ne);
-      mfem::forall(ne*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+      MFEM_FORALL(glob_j, ne*NDOFS,
      {
         const int e = glob_j/NDOFS;
         const int j = glob_j%NDOFS;
@@ -950,7 +759,7 @@ void EABilinearFormExtension::MultTranspose(const Vector &x, Vector &y) const
         if (!factorize_face_terms)
         {
            auto A_int = Reshape(ea_data_int.Read(), NDOFS, NDOFS, 2, nf_int);
-            mfem::forall(nf_int*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+            MFEM_FORALL(glob_j, nf_int*NDOFS,
            {
               const int f = glob_j/NDOFS;
               const int j = glob_j%NDOFS;
@@ -969,7 +778,7 @@ void EABilinearFormExtension::MultTranspose(const Vector &x, Vector &y) const
            });
         }
         auto A_ext = Reshape(ea_data_ext.Read(), NDOFS, NDOFS, 2, nf_int);
-         mfem::forall(nf_int*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+         MFEM_FORALL(glob_j, nf_int*NDOFS,
         {
            const int f = glob_j/NDOFS;
            const int j = glob_j%NDOFS;
@@ -1006,7 +815,7 @@ void EABilinearFormExtension::MultTranspose(const Vector &x, Vector &y) const
         auto X = Reshape(bdr_face_X.Read(), NDOFS, nf_bdr);
         auto Y = Reshape(bdr_face_Y.ReadWrite(), NDOFS, nf_bdr);
         auto A = Reshape(ea_data_bdr.Read(), NDOFS, NDOFS, nf_bdr);
-         mfem::forall(nf_bdr*NDOFS, [=] MFEM_HOST_DEVICE (int glob_j)
+         MFEM_FORALL(glob_j, nf_bdr*NDOFS,
         {
            const int f = glob_j/NDOFS;
            const int j = glob_j%NDOFS;
@@ -1221,13 +1030,13 @@ void FABilinearFormExtension::DGMult(const Vector &x, Vector &y) const
      const int local_size = a->FESpace()->GetVSize();
      auto dg_x_ptr = dg_x.Write();
      auto x_ptr = x.Read();
-      mfem::forall(local_size, [=] MFEM_HOST_DEVICE (int i)
+      MFEM_FORALL(i,local_size,
      {
         dg_x_ptr[i] = x_ptr[i];
      });
      const int shared_size = shared_x.Size();
      auto shared_x_ptr = shared_x.Read();
-      mfem::forall(shared_size, [=] MFEM_HOST_DEVICE (int i)
+      MFEM_FORALL(i,shared_size,
      {
         dg_x_ptr[local_size+i] = shared_x_ptr[i];
      });
@@ -1238,7 +1047,7 @@ void FABilinearFormExtension::DGMult(const Vector &x, Vector &y) const
         // DG Restriction
         auto dg_y_ptr = dg_y.Read();
         auto y_ptr = y.ReadWrite();
-         mfem::forall(local_size, [=] MFEM_HOST_DEVICE (int i)
+         MFEM_FORALL(i,local_size,
         {
            y_ptr[i] += dg_y_ptr[i];
         });
@@ -1282,13 +1091,13 @@ void FABilinearFormExtension::DGMultTranspose(const Vector &x, Vector &y) const
      const int local_size = a->FESpace()->GetVSize();
      auto dg_x_ptr = dg_x.Write();
      auto x_ptr = x.Read();
-      mfem::forall(local_size, [=] MFEM_HOST_DEVICE (int i)
+      MFEM_FORALL(i,local_size,
      {
         dg_x_ptr[i] = x_ptr[i];
      });
      const int shared_size = shared_x.Size();
      auto shared_x_ptr = shared_x.Read();
-      mfem::forall(shared_size, [=] MFEM_HOST_DEVICE (int i)
+      MFEM_FORALL(i,shared_size,
      {
         dg_x_ptr[local_size+i] = shared_x_ptr[i];
      });
@@ -1299,7 +1108,7 @@ void FABilinearFormExtension::DGMultTranspose(const Vector &x, Vector &y) const
         // DG Restriction
         auto dg_y_ptr = dg_y.Read();
         auto y_ptr = y.ReadWrite();
-         mfem::forall(local_size, [=] MFEM_HOST_DEVICE (int i)
+         MFEM_FORALL(i,local_size,
         {
            y_ptr[i] += dg_y_ptr[i];
         });
@@ -1637,7 +1446,7 @@ void PADiscreteLinearOperatorExtension::Assemble()
   }

   auto tm = test_multiplicity.ReadWrite();
-   mfem::forall(test_multiplicity.Size(), [=] MFEM_HOST_DEVICE (int i)
+   MFEM_FORALL(i, test_multiplicity.Size(),
   {
      tm[i] = 1.0 / tm[i];
   });
@@ -1689,7 +1498,7 @@ void PADiscreteLinearOperatorExtension::AddMultTranspose(
   MFEM_VERIFY(x.Size() == test_multiplicity.Size(), "Input vector of wrong size");
   auto xs = xscaled.ReadWrite();
   auto tm = test_multiplicity.Read();
-   mfem::forall(x.Size(), [=] MFEM_HOST_DEVICE (int i)
+   MFEM_FORALL(i, x.Size(),
   {
      xs[i] *= tm[i];
   });
@@ -68,9 +68,6 @@ class PABilinearFormExtension : public BilinearFormExtension
 {
 protected:
   const FiniteElementSpace *trial_fes, *test_fes; // Not owned
-   /// Attributes of all mesh elements.
-   Array<int> elem_attributes, bdr_attributes;
-   mutable Vector tmp_evec; // Work array
   mutable Vector localX, localY;
   mutable Vector int_face_X, int_face_Y;
   mutable Vector bdr_face_X, bdr_face_Y;
@@ -94,25 +91,6 @@ public:

 protected:
   void SetupRestrictionOperators(const L2FaceValues m);
-
-   /// @brief Accumulate the action (or transpose) of the integrator on @a x
-   /// into @a y, taking into account the (possibly null) @a markers array.
-   ///
-   /// If @a markers is non-null, then only those elements or boundary elements
-   /// whose attribute is marked in the markers array will be added to @a y.
-   ///
-   /// @param integ The integrator (domain, boundary, or boundary face).
-   /// @param x Input E-vector.
-   /// @param markers Marked attributes (possibly null, meaning all attributes).
-   /// @param attributes Array of element or boundary element attributes.
-   /// @param transpose Compute the action or transpose of the integrator .
-   /// @param y Output E-vector
-   void AddMultWithMarkers(const BilinearFormIntegrator &integ,
-                           const Vector &x,
-                           const Array<int> *markers,
-                           const Array<int> &attributes,
-                           const bool transpose,
-                           Vector &y) const;
 };

 /// Data and methods for element-assembled bilinear forms
@@ -22,53 +22,41 @@ namespace mfem

 void BilinearFormIntegrator::AssemblePA(const FiniteElementSpace&)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssemblePA(fes)\n"
-              "   is not implemented for this class.");
-}
-
-void BilinearFormIntegrator::AssembleNURBSPA(const FiniteElementSpace&)
-{
-   mfem_error ("BilinearFormIntegrator::AssembleNURBSPA(fes)\n"
+   mfem_error ("BilinearFormIntegrator::AssemblePA(fes)\n"
               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssemblePA(const FiniteElementSpace&,
                                        const FiniteElementSpace&)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssemblePA(fes, fes)\n"
-              "   is not implemented for this class.");
-}
-
-void BilinearFormIntegrator::AssemblePABoundary(const FiniteElementSpace&)
-{
-   MFEM_ABORT("BilinearFormIntegrator::AssemblePABoundary(fes)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssemblePA(fes, fes)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssemblePAInteriorFaces(const FiniteElementSpace&)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssemblePAInteriorFaces(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssemblePAInteriorFaces(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssemblePABoundaryFaces(const FiniteElementSpace&)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssemblePABoundaryFaces(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssemblePABoundaryFaces(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleDiagonalPA(Vector &)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssembleDiagonalPA(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssembleDiagonalPA(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleEA(const FiniteElementSpace &fes,
                                        Vector &emat,
                                        const bool add)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssembleEA(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssembleEA(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleEAInteriorFaces(const FiniteElementSpace
@@ -77,8 +65,8 @@ void BilinearFormIntegrator::AssembleEAInteriorFaces(const FiniteElementSpace
                                                     Vector &ea_data_ext,
                                                     const bool add)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssembleEAInteriorFaces(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssembleEAInteriorFaces(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleEABoundaryFaces(const FiniteElementSpace
@@ -86,8 +74,8 @@ void BilinearFormIntegrator::AssembleEABoundaryFaces(const FiniteElementSpace
                                                     Vector &ea_data_bdr,
                                                     const bool add)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssembleEABoundaryFaces(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssembleEABoundaryFaces(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleDiagonalPA_ADAt(const Vector &, Vector &)
@@ -98,75 +86,62 @@ void BilinearFormIntegrator::AssembleDiagonalPA_ADAt(const Vector &, Vector &)

 void BilinearFormIntegrator::AddMultPA(const Vector &, Vector &) const
 {
-   MFEM_ABORT("BilinearFormIntegrator:AddMultPA:(...)\n"
-              "   is not implemented for this class.");
-}
-
-void BilinearFormIntegrator::AddMultNURBSPA(const Vector &, Vector &) const
-{
-   MFEM_ABORT("BilinearFormIntegrator::AddMultNURBSPA(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::MultAssembled(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AddMultTransposePA(const Vector &, Vector &) const
 {
-   MFEM_ABORT("BilinearFormIntegrator::AddMultTransposePA(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AddMultTransposePA(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleMF(const FiniteElementSpace &fes)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssembleMF(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssembleMF(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AddMultMF(const Vector &, Vector &) const
 {
-   MFEM_ABORT("BilinearFormIntegrator::AddMultMF(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AddMultMF(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AddMultTransposeMF(const Vector &, Vector &) const
 {
-   MFEM_ABORT("BilinearFormIntegrator::AddMultTransposeMF(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AddMultTransposeMF(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleDiagonalMF(Vector &)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssembleDiagonalMF(...)\n"
-              "   is not implemented for this class.");
-}
-
-void BilinearFormIntegrator::AssembleElementMatrix(
-   const FiniteElement &el, ElementTransformation &Trans,
-   DenseMatrix &elmat)
-{
-   MFEM_ABORT("BilinearFormIntegrator::AssembleElementMatrix(...)\n"
-              "   is not implemented for this class.");
-}
-
-void BilinearFormIntegrator::AssembleElementMatrix2(
-   const FiniteElement &el1, const FiniteElement &el2,
-   ElementTransformation &Trans, DenseMatrix &elmat)
-{
-   MFEM_ABORT("BilinearFormIntegrator::AssembleElementMatrix2(...)\n"
-              "   is not implemented for this class.");
-}
-
-void BilinearFormIntegrator::AssemblePatchMatrix(
-   const int patch, const FiniteElementSpace &fes, SparseMatrix*& smat)
-{
-   mfem_error ("BilinearFormIntegrator::AssemblePatchMatrix(...)\n"
+   mfem_error ("BilinearFormIntegrator::AssembleDiagonalMF(...)\n"
               "   is not implemented for this class.");
 }

-void BilinearFormIntegrator::AssembleFaceMatrix(
+void BilinearFormIntegrator::AssembleElementMatrix (
+   const FiniteElement &el, ElementTransformation &Trans,
+   DenseMatrix &elmat )
+{
+   mfem_error ("BilinearFormIntegrator::AssembleElementMatrix(...)\n"
+               "   is not implemented for this class.");
+}
+
+void BilinearFormIntegrator::AssembleElementMatrix2 (
+   const FiniteElement &el1, const FiniteElement &el2,
+   ElementTransformation &Trans, DenseMatrix &elmat )
+{
+   mfem_error ("BilinearFormIntegrator::AssembleElementMatrix2(...)\n"
+               "   is not implemented for this class.");
+}
+
+void BilinearFormIntegrator::AssembleFaceMatrix (
   const FiniteElement &el1, const FiniteElement &el2,
   FaceElementTransformations &Trans, DenseMatrix &elmat)
 {
-   MFEM_ABORT("BilinearFormIntegrator::AssembleFaceMatrix(...)\n"
-              "   is not implemented for this class.");
+   mfem_error ("BilinearFormIntegrator::AssembleFaceMatrix(...)\n"
+               "   is not implemented for this class.");
 }

 void BilinearFormIntegrator::AssembleFaceMatrix(
@@ -178,16 +153,6 @@ void BilinearFormIntegrator::AssembleFaceMatrix(
              " Integrator class.");
 }

-void BilinearFormIntegrator::AssembleTraceFaceMatrix (int elem,
-                                                      const FiniteElement &trial_face_fe,
-                                                      const FiniteElement &test_fe1,
-                                                      FaceElementTransformations &Trans,
-                                                      DenseMatrix &elmat)
-{
-   MFEM_ABORT("AssembleTraceFaceMatrix (DPG form) is not implemented for this"
-              " Integrator class.");
-}
-
 void BilinearFormIntegrator::AssembleElementVector(
   const FiniteElement &el, ElementTransformation &Tr, const Vector &elfun,
   Vector &elvect)
@@ -867,19 +832,6 @@ void DiffusionIntegrator::AssembleElementMatrix

   const IntegrationRule *ir = IntRule ? IntRule : &GetRule(el, el);

-   const NURBSFiniteElement *NURBSFE =
-      dynamic_cast<const NURBSFiniteElement *>(&el);
-
-   bool deleteRule = false;
-   if (NURBSFE && patchRules)
-   {
-      const int patch = NURBSFE->GetPatch();
-      const int* ijk = NURBSFE->GetIJK();
-      Array<const KnotVector*>& kv = NURBSFE->KnotVectors();
-      ir = &patchRules->GetElementRule(NURBSFE->GetElement(), patch, ijk, kv,
-                                       deleteRule);
-   }
-
   elmat = 0.0;
   for (int i = 0; i < ir->GetNPoints(); i++)
   {
@@ -914,11 +866,6 @@ void DiffusionIntegrator::AssembleElementMatrix
         AddMult_a_AAt(w, dshapedxt, elmat);
      }
   }
-
-   if (deleteRule)
-   {
-      delete ir;
-   }
 }

 void DiffusionIntegrator::AssembleElementMatrix2(
@@ -2456,7 +2403,7 @@ void VectorFEMassIntegrator::AssembleElementMatrix(
 {
   int dof = el.GetDof();
   int spaceDim = Trans.GetSpaceDim();
-   int vdim = std::max(spaceDim, el.GetRangeDim());
+   int vdim = std::max(spaceDim, el.GetVDim());

   double w;

@@ -2524,7 +2471,7 @@ void VectorFEMassIntegrator::AssembleElementMatrix2(
   {
      // assume test_fe is scalar FE and trial_fe is vector FE
      int spaceDim = Trans.GetSpaceDim();
-      int vdim = std::max(spaceDim, trial_fe.GetRangeDim());
+      int vdim = std::max(spaceDim, trial_fe.GetVDim());
      int trial_dof = trial_fe.GetDof();
      int test_dof = test_fe.GetDof();
      double w;
@@ -2622,8 +2569,8 @@ void VectorFEMassIntegrator::AssembleElementMatrix2(
   {
      // assume both test_fe and trial_fe are vector FE
      int spaceDim = Trans.GetSpaceDim();
-      int trial_vdim = std::max(spaceDim, trial_fe.GetRangeDim());
-      int test_vdim = std::max(spaceDim, test_fe.GetRangeDim());
+      int trial_vdim = std::max(spaceDim, trial_fe.GetVDim());
+      int test_vdim = std::max(spaceDim, test_fe.GetVDim());
      int trial_dof = trial_fe.GetDof();
      int test_dof = test_fe.GetDof();
      double w;
@@ -2686,7 +2633,7 @@ void VectorFEMassIntegrator::AssembleElementMatrix2(
   }
   else
   {
-      MFEM_ABORT("VectorFEMassIntegrator::AssembleElementMatrix2(...)\n"
+      mfem_error("VectorFEMassIntegrator::AssembleElementMatrix2(...)\n"
                 "   is not implemented for given trial and test bases.");
   }
 }
@@ -4050,216 +3997,6 @@ void NormalTraceJumpIntegrator::AssembleFaceMatrix(
   }
 }

-void TraceIntegrator::AssembleTraceFaceMatrix(int elem,
-                                              const FiniteElement &trial_face_fe,
-                                              const FiniteElement &test_fe,
-                                              FaceElementTransformations & Trans,
-                                              DenseMatrix &elmat)
-{
-   MFEM_VERIFY(test_fe.GetMapType() == FiniteElement::VALUE,
-               "TraceIntegrator::AssembleTraceFaceMatrix: Test space should be H1");
-   MFEM_VERIFY(trial_face_fe.GetMapType() == FiniteElement::INTEGRAL,
-               "TraceIntegrator::AssembleTraceFaceMatrix: Trial space should be RT trace");
-
-   int i, j, face_ndof, ndof;
-   int order;
-
-   face_ndof = trial_face_fe.GetDof();
-   ndof = test_fe.GetDof();
-
-   face_shape.SetSize(face_ndof);
-   shape.SetSize(ndof);
-
-   elmat.SetSize(ndof, face_ndof);
-   elmat = 0.0;
-
-   const IntegrationRule *ir = IntRule;
-   if (ir == NULL)
-   {
-      order = test_fe.GetOrder();
-      order += trial_face_fe.GetOrder();
-      ir = &IntRules.Get(Trans.GetGeometryType(), order);
-   }
-
-   int iel = Trans.Elem1->ElementNo;
-   if (iel != elem)
-   {
-      MFEM_VERIFY(elem == Trans.Elem2->ElementNo, "Elem != Trans.Elem2->ElementNo");
-   }
-
-   double scale = 1.0;
-   if (iel != elem) { scale = -1.; }
-   for (int p = 0; p < ir->GetNPoints(); p++)
-   {
-      const IntegrationPoint &ip = ir->IntPoint(p);
-
-      // Set the integration point in the face and the neighboring elements
-      Trans.SetAllIntPoints(&ip);
-      // Trace finite element shape function
-      trial_face_fe.CalcPhysShape(Trans,face_shape);
-
-      // Finite element shape function
-      ElementTransformation * eltrans = (iel == elem) ? Trans.Elem1 : Trans.Elem2;
-      test_fe.CalcPhysShape(*eltrans, shape);
-
-      face_shape *= Trans.Weight()*ip.weight*scale;
-      for (i = 0; i < ndof; i++)
-      {
-         for (j = 0; j < face_ndof; j++)
-         {
-            elmat(i, j) += shape(i) * face_shape(j);
-         }
-      }
-   }
-}
-
-void NormalTraceIntegrator::AssembleTraceFaceMatrix(int elem,
-                                                    const FiniteElement &trial_face_fe,
-                                                    const FiniteElement &test_fe,
-                                                    FaceElementTransformations &Trans,
-                                                    DenseMatrix &elmat)
-{
-   int i, j, face_ndof, ndof, dim;
-   int order;
-
-   MFEM_VERIFY(test_fe.GetMapType() == FiniteElement::H_DIV,
-               "NormalTraceIntegrator::AssembleTraceFaceMatrix: Test space should be RT");
-   MFEM_VERIFY(trial_face_fe.GetMapType() == FiniteElement::VALUE,
-               "NormalTraceIntegrator::AssembleTraceFaceMatrix: Trial space should be H1 (trace)");
-
-   face_ndof = trial_face_fe.GetDof();
-   ndof = test_fe.GetDof();
-   dim = test_fe.GetDim();
-
-   face_shape.SetSize(face_ndof);
-   normal.SetSize(dim);
-   shape.SetSize(ndof,dim);
-   shape_n.SetSize(ndof);
-
-   elmat.SetSize(ndof, face_ndof);
-   elmat = 0.0;
-
-   const IntegrationRule *ir = IntRule;
-   if (ir == NULL)
-   {
-      order = test_fe.GetOrder();
-      order += trial_face_fe.GetOrder();
-      ir = &IntRules.Get(Trans.GetGeometryType(), order);
-   }
-
-   int iel = Trans.Elem1->ElementNo;
-   if (iel != elem)
-   {
-      MFEM_VERIFY(elem == Trans.Elem2->ElementNo, "Elem != Trans.Elem2->ElementNo");
-   }
-
-   double scale = 1.0;
-   if (iel != elem) { scale = -1.; }
-
-   for (int p = 0; p < ir->GetNPoints(); p++)
-   {
-      const IntegrationPoint &ip = ir->IntPoint(p);
-      Trans.SetAllIntPoints(&ip);
-      trial_face_fe.CalcPhysShape(Trans, face_shape);
-      CalcOrtho(Trans.Jacobian(),normal);
-      ElementTransformation * etrans = (iel == elem) ? Trans.Elem1 : Trans.Elem2;
-      test_fe.CalcVShape(*etrans, shape);
-      shape.Mult(normal, shape_n);
-      face_shape *= ip.weight*scale;
-
-      for (i = 0; i < ndof; i++)
-      {
-         for (j = 0; j < face_ndof; j++)
-         {
-            elmat(i, j) += shape_n(i) * face_shape(j);
-         }
-      }
-   }
-}
-
-void TangentTraceIntegrator::AssembleTraceFaceMatrix(int elem,
-                                                     const FiniteElement &trial_face_fe,
-                                                     const FiniteElement &test_fe,
-                                                     FaceElementTransformations & Trans,
-                                                     DenseMatrix &elmat)
-{
-
-   MFEM_VERIFY(test_fe.GetMapType() == FiniteElement::H_CURL,
-               "TangentTraceIntegrator::AssembleTraceFaceMatrix: Test space should be ND");
-
-   int face_ndof, ndof, dim;
-   int order;
-   dim = test_fe.GetDim();
-   if (dim == 3)
-   {
-      std::string msg =
-         "Trial space should be ND face trace and test space should be a ND vector field in 3D ";
-      MFEM_VERIFY(trial_face_fe.GetMapType() == FiniteElement::H_CURL &&
-                  trial_face_fe.GetDim() == 2 && test_fe.GetDim() == 3, msg);
-   }
-   else
-   {
-      std::string msg =
-         "Trial space should be H1 edge trace and test space should be a ND vector field in 2D";
-      MFEM_VERIFY(trial_face_fe.GetMapType() == FiniteElement::VALUE &&
-                  trial_face_fe.GetDim() == 1 && test_fe.GetDim() == 2, msg);
-   }
-   face_ndof = trial_face_fe.GetDof();
-   ndof = test_fe.GetDof();
-
-   int dimc = (dim == 3) ? 3 : 1;
-
-   face_shape.SetSize(face_ndof,dimc);
-   shape_n.SetSize(ndof,dimc);
-   shape.SetSize(ndof,dim);
-   normal.SetSize(dim);
-   DenseMatrix face_shape_n(face_ndof,dimc);
-
-   elmat.SetSize(ndof, face_ndof);
-   elmat = 0.0;
-
-   const IntegrationRule *ir = IntRule;
-   if (ir == NULL)
-   {
-      order = test_fe.GetOrder();
-      order += trial_face_fe.GetOrder();
-      ir = &IntRules.Get(Trans.GetGeometryType(), order);
-   }
-
-   int iel = Trans.Elem1->ElementNo;
-   if (iel != elem)
-   {
-      MFEM_VERIFY(elem == Trans.Elem2->ElementNo, "Elem != Trans.Elem2->ElementNo");
-   }
-
-   double scale = 1.0;
-   if (iel != elem) { scale = -1.; }
-   for (int p = 0; p < ir->GetNPoints(); p++)
-   {
-      const IntegrationPoint &ip = ir->IntPoint(p);
-      // Set the integration point in the face and the neighboring elements
-      Trans.SetAllIntPoints(&ip);
-      // Trace finite element shape function
-      if (dim == 3)
-      {
-         trial_face_fe.CalcVShape(Trans,face_shape);
-      }
-      else
-      {
-         face_shape.GetColumnReference(0,temp);
-         trial_face_fe.CalcPhysShape(Trans,temp);
-      }
-      CalcOrtho(Trans.Jacobian(),normal);
-      ElementTransformation * eltrans = (iel == elem) ? Trans.Elem1 : Trans.Elem2;
-      test_fe.CalcVShape(*eltrans, shape);
-
-      // rotate
-      cross_product(normal, shape, shape_n);
-
-      const double w = scale*ip.weight;
-      AddMult_a_ABt(w,shape_n, face_shape, elmat);
-   }
-}

 void NormalInterpolator::AssembleElementMatrix2(
   const FiniteElement &dom_fe, const FiniteElement &ran_fe,
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Tucker Hartland	0e5f41221f	including an obstacle problem variant wherein the essential dofs are not given to the optimizer as free variables that it must choose the values for.	2024-03-01 14:58:16 -08:00
Tucker Hartland	d2794dd0da	removing bug in the evaluation of the optimality measures wherein all processors now see the same value of the optimality error	2023-11-10 10:43:20 -08:00
Tucker Hartland	88372c99ee	clean up... using a GeneralOptimizationProblem class and an OptimizationProblem class and no longer using the specific ContactProblem class as the generality of the Parent class does not have anything per say to do with contact	2023-10-10 11:33:29 -07:00
Tucker Hartland	52bfc8618d	Merge branch 'contactIPM-dev' of https://github.com/mfem/mfem into contactIPM-dev	2023-08-07 10:07:55 -07:00
Tucker Hartland	977cb839b4	adding the ability to use the Hessian of the constraints in the optimizer, for the two sided Dirichlet obstacle problem it is observed that the number of outer Newton iterations is relatively constant only when the Hessian of the constraints are included in the Newton linear system	2023-08-07 10:06:35 -07:00
Socratis Petrides	a52c7f036a	bug fix	2023-07-25 19:20:04 -07:00
Socratis Petrides	39c32b9454	removing leftover print	2023-07-25 17:01:44 -07:00
Socratis Petrides	49041b062b	simplifying dirichlet dofs handling	2023-07-25 16:26:21 -07:00
Socratis Petrides	ed2d0846f6	small edits fixing compiler warnings	2023-07-24 14:34:22 -07:00
Tucker Hartland	a1f6b3cf42	renaming descriptions of Optimization Problems... the general optimization problem which is more useful for PDE- and bound-constrained optimization is now described just as that GeneralOptProblem... the problem format min E(u) s.t. g(u) >= 0 and where (as the interior-point solver sees) the bound-constrained variable is a slack variable is now the less general but fairly descriptive OptProblem... the QPOptProblem (formerly QPContactProblem) now is more aptly described as there is nothing specific to contact that the class invokes	2023-07-21 12:40:04 -07:00
Tucker Hartland	b8f4fbc84f	updating a problem description	2023-07-21 10:11:17 -07:00
Tucker Hartland	a2fef97289	removing the distinction between DirichletObstacle and ObstacleProblems	2023-07-21 10:06:58 -07:00
Tucker Hartland	07f22f98d3	removing DirichletObstacleProblem in favor of the ObstacleProblem class which has the ability to utilize Dirichlet boundary conditions	2023-07-20 18:44:34 -07:00
Tucker Hartland	9d5f4fd4ea	removed all usages of typedef Number and all unnecessary ipopt stuff... moving meshes to the example/contact subdirectory and out of data directory... makefile now copies the meshes to the out of source directory when appropriate... problems now have more functionality for the inclusion of the Hessian of the constraint functions	2023-07-20 16:40:48 -07:00
tuckerh	17139c5fe8	fixing issue of a null Jacobian that shows up with certain builds/compilers/systems... the order of the gap function evaluation and gap function Jacobian in exQPContact was out of order and the Jacobian that is owned by the problem and passed to main was being deleted after a subsequent call of the gap function. Strange that this issue did not show up with all compilers	2023-07-20 09:51:05 -07:00
Tucker Hartland	3d201cd4ef	removing comment about how function is leaking memory, since it is no longer leaking memory	2023-07-19 16:42:07 -07:00
Tucker Hartland	272f816245	fixing a memory leak via a FreeData call on the FindPointsGSLIB finder object in FindPointsInMesh	2023-07-19 16:29:54 -07:00
Tucker Hartland	d18539aae2	fixing bug when not compiled with SUITESPARSE	2023-07-19 14:13:25 -07:00
Tucker Hartland	03aa2308d1	using more utility functions... ownership of blocks of IP-Newton system now owned by the problem and not the optimizer/solver... less copying in QPcontact and removing more of the functionality for ipopt	2023-07-17 17:36:44 -07:00
Tucker Hartland	27bcb49294	removing unnecessary copy when using dyanmic_cast, as well as removing unnecessary delete of the dynamic_cast variable to avoid dangling pointers	2023-07-17 15:27:23 -07:00
Tucker Hartland	90ca8bd551	including the QP contact block problem example... the infastructure in problems, additional meshes in data	2023-07-17 11:09:18 -07:00
Tucker Hartland	762fbedb42	more descriptions in Parallel Spherical example problem, as well as outputting the error of the numerical solution and that of the analytic solution	2023-07-06 12:09:32 -07:00
Tucker Hartland	028bbfbc20	Spherical obstacle problem in parallel... more consistent/descriptive variables for the parallel interior-point solver...including a parallel Dirichlet obstacle problem description	2023-07-05 17:35:17 -07:00
Tucker Hartland	c2f85de2f9	added a new example wherein the Dirichlet condition and the obstacle coincide	2023-07-03 18:26:04 -07:00
Tucker Hartland	c4c9962a0b	small modification of the DirichletObstacleProblem that fixes a memory leak	2023-07-03 13:08:27 -07:00
tuckerh	5d15f6c2eb	removing an unnecessary item	2023-06-29 13:38:48 -07:00
tuckerh	64e3cf5a52	making the ParIPSolver consistent with the IPsolver.... in particular the reduced print messages and also what the linSolver i.d.s correspond to	2023-06-29 13:18:43 -07:00
Tucker Hartland	81c0094f96	Spherical obstacle problem -- solving an obstacle problem with Dirichlet boundary conditions, able to check numerical solution against analytic to see convergence of the optimizer under mesh refinement to the analytic solution. problems.*pp contains new functionality for being able to describe an obstacle problem with Dirichlet boundary conditions. There has been a minor change to the IPsolver, so that there are fewer statements about the intermediate status of the optimizer	2023-06-29 12:35:21 -07:00
Tucker Hartland	91d8d5da29	removing unnecessary iterative solve option with the Schur-complement approximation Huu + D of the true Schur-complement Huu + Ju^T D Ju	2023-06-06 13:37:57 -07:00
tuckerh	e4dd3399ac	adding direct and iterative method options to solve IP-Newton-contact system via the Schur complement/stiffness by contact system matrix... including option to set the linear solve tolerance used by the iterative solver from the application code	2023-05-25 08:41:24 -07:00
Tucker Hartland	aeff128c42	altering the logic of how the linear solver is chosen in the interior-point method, specifically to expose the newly available iterative method to builds that do not use SUITESPARSE	2023-05-25 07:14:05 -07:00
Tucker Hartland	617c9e2f21	adding new linear solver options and reducing the optimization tolerance so as to avoid conflicts with tolerances for Krylov-subspace solvers	2023-05-24 18:28:39 -07:00
Tucker Hartland	9fd2ae2229	using more descriptive language to describe various member functions	2023-05-24 10:57:18 -07:00
tuckerh	bec6766ddb	having sorted out the issue with the HypreMatrix from blocks function, we now remove unnecessary print statements and also include the fix	2023-05-15 17:07:02 -07:00
tuckerh	81fa2ca541	updating so that things are current even though there are failures with cpardiso on quartz	2023-05-15 14:12:35 -07:00
tuckerh	dadfad3321	updated makefile so we can use either MUMPS or CPARDISO for the parallel sparse direct solver... using NULL instead of nullptr in order to try to be more consistent with the use of HypreParMatrixFromBlocks	2023-05-15 10:00:39 -07:00
tuckerh	18d3d7182c	adding ability to use CPardiso sparse direct solvers	2023-05-12 13:23:47 -07:00
Socratis Petrides	994310c49e	bug fix	2023-05-10 18:27:10 -07:00
Socratis Petrides	6396772300	first iteration of obstacle problem in parallel	2023-05-09 16:30:50 -07:00
Socratis Petrides	a2b5fca7d4	minor changes to fix compiler warnings	2023-05-09 16:30:00 -07:00
Tucker Hartland	891a1aa72d	new Mult function included in IPSolver so that the user does not need to be aware of the variable for which the bound-constraints are applied m >= ml, this is useful for contact mechanics problems wherein the bound-constraint variable is actually a slack variable and will not be especially important to the user. obstacleProblem.cpp is now cleaned up a bit, there is no longer a need to create a BlockVector and use this as input to the Mult IPM solution call and so it has been removed in favor of just working with a Vector representation of the primal variable	2023-05-05 15:21:50 -07:00
Tucker Hartland	332cc0e9da	including another argument in the constructor of the abstract ContactProblem class, so that we pass the number of inequality constraints, this removes the need to copy the so-called boiler plate code into each of the child ContractProblem classes	2023-05-05 14:13:09 -07:00
Tucker Hartland	82c14b7544	altering various optimization problem member functions so that they no longer are of void type and take a reference to a SparseMatrix pointer but rather they just return a SparseMatrix pointer. the obstacleProblem now expects a function pointer to be passed when initialized, so that the right hand side forcing term is specified from the application side	2023-05-04 13:25:34 -07:00
Tucker Hartland	c4cc5b600b	adding initial serial features for contact... optimizer and the obstacleProblem.	2023-04-18 11:24:44 -07:00