Files
mfem/linalg/dtensor.hpp
T
John CamiercamierjsVladimir Z TomovMittal, Ketan <mittal3@llnl.gov>
50368046bc [TMOP] Simplify kernels (#3658)
* Simplify TMOP kernels, fix unit tests to run --all tests with adjusted tolerance

* make style

* Split TMOP h3s file with metrics

* TMOP kernel MFEM_HOST_DEVICE fix

* Cleanup TMOP CUDA kernels from base class

* Added TMOP PA metrics directory

* meld toward master

* [tmop] struct to class friends

* Simplify tmop file names

* make style

* Cleanup

* Style and vscode gitignore

* WIP resolve conflicts

* 2024 headers

* Tmop pass

* All tmop tests

* make style

* Add astyle to clang format

* make style

* Fix class visibility

* Include cleanup

* real_t pass

* style

* MFEM_REGISTER_KERNELS for TMOPAssembleGradPA_001

* make style

* add config files

* Update config

* metric_t

* wip with T

* wip

* wip T Specialization

* c++20, fmt make_format_args

* print types and values

* wip Kernel<decltype(M)>

* wip

* Working with metric_t, int, int

* C++20 ok

* C++17 cleaned

* Rename tmop files

* Sync TMOP kernels with dispatch

* make style

* Cleanup metrics

* Use TMOPKernel

* 3D metrics standalone

* Chdir assemble

* tmop 2d/3d directories

* TMOP assemble using specializations

* All TMOP kernel specializations

* MFEM_REPORT_KERNELS

* make style

* Sync with master

* Sync with master

* make style

* make style

* Removed 2d/3d TMOP sub-directories

* CMake TMOP file list update

* makefile directories order

* With style

* Re-enable vscode gitignore

* Fix merge conflicts

* make style

* Sync

* Meld toward master

* Changes toward master

* make style

* Meld back fem tmop files

* Fix TMOP_Integrator friends

* PA tests fix & history bump

* Cleanup test tmop and fix energy2 metric data

* Update copyright 2010-2025

* 2D energy metrics

* 3D energy metrics

* make style

* Simplify metric registration

* TMOP fem kernels with double buffering

* grad3, grad3_coef

* grad3_coef, grad3, mult3_coefs, mult3

* TMOP sm kernels tools

* Rename kernels smem and use regs

* Grad3 w/ vector reg grad

* Kernel register cleanup

* Add MAX_TMOP_1D and HIP tmop ctests

* Add kernels_foreach

* Add kernels foreach

* Prefix foreach_thread

* Kernels regs w/ foreach threads

* Swap Y and X in forward only

* Backward kernels_regs

* Use simplified grad3d

* Wip D1D Q1D

* Runtime D1D Q1D

* Remove T1D

* Cleanup

* AddKernelSpecializations

* Sync with SetMaxOf

* Rename to LoadDofs and use deduced templated parameters

* Grad2d & factorization

* Eval3d for grad3 coef

* Eval2d for grad2 coef

* Cleanup TMOP_SetupGradPA_C0_2D

* Use Bld and B

* Use other accessors

* TMOPAddMultPA3D

* TMOP_AddMultPA_C0_2D

* TMOP_AddMultGradPA_3D

* TMOP_AddMultGradPA_2D

* TMOP_AddMultGradPA_C0_3D

* AddMultGradPA_C0_2D

* TMOP_AssembleDiagonalPA_2D

* Wip TMOP_AssembleDiagonalPA_C0_3D

* TMOP_AssembleDiagonalPA_3D

* TMOP_MinDetJpr_3D

* TMOP_EnergyPA_C0_2D

* TMOPEnergyPA3D

* TMOP_TcIdealShapeGivenSize_3D

* TMOP_DatcSize_3D

* Remove MAX_TMOP_1D

* Remove smem kernels

* TMOP cleanup

* TMOP - solve for displacements #4694 changes

* Cleanup and move verifications

* Rename TMOP Assemble kernels

* Move kernel regs to TMOP pa

* make style

* Meld back toward master

* Meld back to master

* Use static constexpr

* Temporary branch-history

* Help msvc with namespaces

* MSVC inner static constexpr

* Move regs to mfem namespace

* MSVC all static constexpr

* TMOP_AssembleDiagPA_C0_3D w/o regs

* Avoid set but unused variable

* MSVC TMOP_AssembleDiagPA_C0_3D ternary test try

* MSVC MFEM_TMOP_REGISTER_MDQ_KERNEL

* Switch to MFEM_TMOP_MDQ_REGISTER

* MSVC help with static constexpr

* MSVC conversions try

* MSVC as_regs2d_ref

* MSVC Explicitly bind as reference

* MSCV with reinterpret_cast

* MSVC avoiding required l-values

* MSVC avoid explicit ref bindings

* MSVC avoid explicit ref bindings 2D

* Cleanup

* Enable MFEM_TMOP_PA_DEVICE with makefile

* TMOP tests w/o Kernel Specializations

* TMOP re-enable kernels specializations

* TMOP PA tests tolerances

* TMOP tests adjustments

* Fix transposed eval regs access

* MSVC remove not allowed dllimport definitions

* MSVC linalg vector warning fix

* MSVC avoiding definition of dllimport function not allowed

* Re-enable DetKernels specializations

* Sync latest TMOP changes

* TMOP PA tests normalization wip

* Sync TMOP tests

* Remove debug file

* Meld back toward master

* Add missing tmop make source dir

* tmop shadowing, CMake & make mpi tests

* TMOP periodic tests, shadowing fix

* TMOP pa mpi tests, fix shadowing

* TMOP tighten Square01 + Combo tests

* TMOP MSVC include ordering

* Revert TMOP MPI debug device tests

* Add TMOP_DatcSize_2D

* Use mfem::future for tensor

* Move TMOP PA specific kernels to sync'ed fem kernels

* makefile source dirs fix

* use explicit namespace to avoid clash (swap)

* Revert to MFEM_FOREACH_THREAD
Use scalar/vector regs types

* Sync kernels

* Sync kernels

* Avoid applying non-zero offset to null pointer runtime error

* Remove debug include

* TMOP rename coef to limit

* Comments.

* minor

* changelog

* Replace TMOP's MFEM_FOREACH_THREAD with MFEM_FOREACH_THREAD_DIRECT

* add some missing metric IDs

* Revert branch-history

* Add missing MFEM_SYNC_THREAD in kernels
Verify TMOP isfinite energy

* UseDevice for local vectors

* make style

* No grids in TMOP_DatcSize kernels

* Remove isfinite assertions
Cleanup unused header files
Add 3D energy finite verifications

* Filter out TMOP PA tests

---------

Co-authored-by: camierjs <camierjs@Io>
Co-authored-by: Vladimir Z Tomov <tomov2@llnl.gov>
Co-authored-by: Mittal, Ketan <mittal3@llnl.gov>
2025-11-13 08:47:32 -08:00

159 lines
4.4 KiB
C++

// Copyright (c) 2010-2025, Lawrence Livermore National Security, LLC. Produced
// at the Lawrence Livermore National Laboratory. All Rights reserved. See files
// LICENSE and NOTICE for details. LLNL-CODE-806117.
//
// This file is part of the MFEM library. For more information and source code
// availability visit https://mfem.org.
//
// MFEM is free software; you can redistribute it and/or modify it under the
// terms of the BSD-3 license. We welcome feedback and contributions, see file
// CONTRIBUTING.md for details.
#ifndef MFEM_DTENSOR
#define MFEM_DTENSOR
#include "../config/config.hpp"
#include "../general/error.hpp"
namespace mfem
{
/// A Class to compute the real index from the multi-indices of a tensor
template <int N, int Dim, typename T, typename... Args>
class TensorInd
{
public:
MFEM_HOST_DEVICE
static inline int result(const int* sizes, T first, Args... args)
{
#if !(defined(MFEM_USE_CUDA) || defined(MFEM_USE_HIP))
MFEM_ASSERT(first<sizes[N-1],"Trying to access out of boundary.");
#endif
return static_cast<int>(first + sizes[N - 1] * TensorInd < N + 1, Dim, Args... >
::result(sizes, args...));
}
};
// Terminal case
template <int Dim, typename T, typename... Args>
class TensorInd<Dim, Dim, T, Args...>
{
public:
MFEM_HOST_DEVICE
static inline int result(const int* sizes, T first, Args... args)
{
#if !(defined(MFEM_USE_CUDA) || defined(MFEM_USE_HIP))
MFEM_ASSERT(first<static_cast<T>(sizes[Dim-1]),
"Trying to access out of boundary.");
#endif
return static_cast<int>(first);
}
};
/// A class to initialize the size of a Tensor
template <int N, int Dim, typename T, typename... Args>
class Init
{
public:
MFEM_HOST_DEVICE
static inline int result(int* sizes, T first, Args... args)
{
sizes[N - 1] = first;
return first * Init < N + 1, Dim, Args... >::result(sizes, args...);
}
};
// Terminal case
template <int Dim, typename T, typename... Args>
class Init<Dim, Dim, T, Args...>
{
public:
MFEM_HOST_DEVICE
static inline int result(int* sizes, T first, Args... args)
{
sizes[Dim - 1] = first;
return first;
}
};
/// A basic generic Tensor class, appropriate for use on the GPU
template<int Dim, typename Scalar = real_t>
class DeviceTensor
{
protected:
int capacity;
Scalar *data;
int sizes[Dim];
public:
/// Default constructor
// DeviceTensor() = delete;
MFEM_HOST_DEVICE
DeviceTensor() {}
/// Constructor to initialize a tensor from the Scalar array data_
template <typename... Args> MFEM_HOST_DEVICE
DeviceTensor(Scalar* data_, Args... args)
{
static_assert(sizeof...(args) == Dim, "Wrong number of arguments");
// Initialize sizes, and compute the number of values
const long int nb = Init<1, Dim, Args...>::result(sizes, args...);
capacity = nb;
data = (capacity > 0) ? data_ : nullptr;
}
/// Copy constructor (default)
DeviceTensor(const DeviceTensor&) = default;
/// Copy assignment (default)
DeviceTensor& operator=(const DeviceTensor&) = default;
/// Conversion to `Scalar *`.
MFEM_HOST_DEVICE inline operator Scalar *() const { return data; }
/// Const accessor for the data
template <typename... Args> MFEM_HOST_DEVICE inline
Scalar& operator()(Args... args) const
{
static_assert(sizeof...(args) == Dim, "Wrong number of arguments");
return data[ TensorInd<1, Dim, Args...>::result(sizes, args...) ];
}
/// Subscript operator where the tensor is viewed as a 1D array.
MFEM_HOST_DEVICE inline Scalar& operator[](int i) const
{
return data[i];
}
/// Returns the shape of the tensor.
MFEM_HOST_DEVICE inline auto &GetShape() const { return sizes; }
};
/** @brief Wrap a pointer as a DeviceTensor with automatically deduced template
parameters */
template <typename T, typename... Dims> MFEM_HOST_DEVICE
inline DeviceTensor<sizeof...(Dims),T> Reshape(T *ptr, Dims... dims)
{
return DeviceTensor<sizeof...(Dims),T>(ptr, dims...);
}
using DeviceArray = DeviceTensor<1,int>;
using ConstDeviceArray = DeviceTensor<1,const int>;
using DeviceVector = DeviceTensor<1,real_t>;
using ConstDeviceVector = DeviceTensor<1,const real_t>;
using DeviceMatrix = DeviceTensor<2,real_t>;
using ConstDeviceMatrix = DeviceTensor<2,const real_t>;
using DeviceCube = DeviceTensor<3,real_t>;
using ConstDeviceCube = DeviceTensor<3,const real_t>;
} // mfem namespace
#endif // MFEM_DTENSOR