Merge pull request #3711 from rcurtin/lcc-doc

Document `LocalCoordinateCoding`

HISTORY.md

@@ -44,6 +44,11 @@
 
   * Fix divide-by-zero edge case for LARS (#3701).
 
+  * Templatize `SparseCoding` and `LocalCoordinateCoding` to allow different
+    matrix types (#3709, #3711).
+
+  * Fix handling of unused atoms in `LocalCoordinateCoding` (#3711).
+
   * Move minimum required C++ version from C++14 to C++17 (#3704).
 
 ### mlpack 4.3.0

doc/index.md

@@ -120,6 +120,8 @@ Prepare data for machine learning algorithms.
 
 Transform data from one space to another.
 
+ * [`LocalCoordinateCoding`](user/methods/local_coordinate_coding.md): local
+   coordinate coding with dictionary learning
  * [`NMF`](user/methods/nmf.md): non-negative matrix factorization
  * [`PCA`](user/methods/pca.md): principal components analysis
  * [`SparseCoding`](user/methods/sparse_coding.md): sparse coding with

doc/sidebar.html

@@ -179,6 +179,11 @@ when the sidebar is built for each page.
           </a>
         </summary>
         <ul>
+          <li>
+            <a href="LINKROOTuser/methods/local_coordinate_coding.html">
+              <code>LocalCoordinateCoding</code>
+            </a>
+          </li>
           <li>
             <a href="LINKROOTuser/methods/nmf.html">
               <code>NMF</code>

doc/user/core.md

@@ -81,7 +81,7 @@ avoid copies.
    - If `offset` is `0`, then the alias is identical: the first element of
      `a` is the first element of `mat`. Otherwise, the first element of `a`
      is the `offset`'th element of `mat`; elements in `mat` are ordered in
-     a [column-major way](../matrices.md#representing-data-in-mlpack).
+     a [column-major way](matrices.md#representing-data-in-mlpack).
    - If `strict` is `true`, the size of `a` cannot be changed.
    - `mat` and `a` should have the same matrix type (e.g. `arma::mat`,
      `arma::fmat`, `arma::sp_mat`).
@@ -93,7 +93,7 @@ avoid copies.
    - If `offset` is `0`, then the alias is identical: the first element of
      `a` is the first element of `cube`. Otherwise, the first element of `a`
      is the `offset`'th element of `cube`; elements in `cube` are ordered in
-     a [column-major way](../matrices.md#representing-data-in-mlpack).
+     a [column-major way](matrices.md#representing-data-in-mlpack).
    - If `strict` is `true`, the size of `a` cannot be changed.
    - `cube` and `a` should have the same cube type (e.g. `arma::cube`,
      `arma::fcube`).

doc/user/methods/local_coordinate_coding.md

@@ -0,0 +1,375 @@
+## `LocalCoordinateCoding`
+
+The `LocalCoordinateCoding` class implements local coordinate coding, a
+variation of [sparse coding](sparse_coding.md) with dictionary learning.  Local
+coordinate coding is a form of representation learning, and can be used to
+represent each point in a dataset as a linear combination of a few nearby
+*atoms* in the learned dictionary.
+
+#### Simple usage example:
+
+```c++
+// Create a random dataset with 100 points in 40 dimensions, and then a random
+// test dataset with 50 points.
+arma::mat data(40, 100, arma::fill::randn);
+arma::mat testData(40, 50, arma::fill::randn);
+
+// Perform local coordinate coding with 20 atoms and an L1 penalty of 0.1.
+mlpack::LocalCoordinateCoding lcc(20, 0.1); // Step 1: create object.
+double objective = lcc.Train(data);         // Step 2: learn dictionary.
+arma::mat codes;
+lcc.Encode(testData, codes);                // Step 3: encode new data.
+
+// Print some information about the test encoding.
+std::cout << "Average density of encoded test data: "
+    << 100.0 * arma::mean(arma::sum(codes != 0)) / codes.n_rows << "\%."
+    << std::endl;
+```
+<p style="text-align: center; font-size: 85%"><a href="#simple-examples">More examples...</a></p>
+
+#### Quick links:
+
+ * [Constructors](#constructors): create `LocalCoordinateCoding` objects.
+ * [`Train()`](#training): train model (learn dictionary).
+ * [`Encode()`](#encoding): encode points with a trained model.
+ * [Other functionality](#other-functionality) for loading, saving, and
+   inspecting.
+ * [Examples](#simple-examples) of simple usage and links to detailed example
+   projects.
+ * [Template parameters](#advanced-functionality-template-parameters) for
+   advanced functionality: different element types and dictionary initialization
+   strategies.
+
+#### See also:
+
+ * [`SparseCoding`](sparse_coding.md)
+ * [`LARS`](lars.md) (used internally by `LocalCoordinateCoding`)
+ * [mlpack transformations](../../index.md#transformations)
+ * [Sparse dictionary learning on Wikipedia](https://en.wikipedia.org/wiki/Sparse_dictionary_learning)
+ * [Nonlinear learning using local coordinate coding (pdf)](https://proceedings.neurips.cc/paper_files/paper/2009/file/2afe4567e1bf64d32a5527244d104cea-Paper.pdf)
+
+### Constructors
+
+ * `lcc = LocalCoordinateCoding()`
+ * `lcc = LocalCoordinateCoding(atoms=0, lambda=0.0, maxIter=0, tol=0.01)`
+   - Create a `LocalCoordinateCoding` object without learning a dictionary on
+     data.
+   - If `atoms` is set to `0` (the default), it will need to be set to a value
+     greater than `0` before `Train()` is called (`lcc.Atoms() = atoms` can be
+     used for this).
+
+ * `lcc = LocalCoordinateCoding(data, atoms, lambda=0.0, maxIter=0, tol=0.01)`
+   - Create a `LocalCoordinateCoding` object and train the dictionary on the
+     given `data`.
+   - The dictionary will contain `atoms` elements.
+
+ * `lcc = LocalCoordinateCoding(data, atoms, lambda, maxIter, tol, initializer)`
+   - *Advanced constructor*: create a `LocalCoordinateCoding` object that will
+     use a custom dictionary initializer and train on the given `data`.
+   - The dictionary will contain `atoms` elements.
+   - `initializer` will be used to initialize the dictionary; see [Advanced
+     Functionality: Different Dictionary Initialization
+     Strategies](#dictionaryinitializer-different-dictionary-initialization-strategies)
+     for details.
+
+#### Constructor Parameters:
+
+| **name** | **type** | **description** | **default** |
+|----------|----------|-----------------|-------------|
+| `data` | [`arma::mat`](../matrices.md) | [Column-major](../matrices.md#representing-data-in-mlpack) training matrix. | _(N/A)_ |
+| `atoms` | `size_t` | Number of atoms in dictionary. | _(N/A)_ |
+| `lambda` | `double` | L1 regularization penalty.  Used in both `Train()` and `Encode()` steps. | `0.0` |
+| `maxIter` | `size_t` | Maximum number of iterations for dictionary learning.  `0` means no limit. | `0` |
+| `tol` | `double` | Objective function tolerance for terminating dictionary learning. | `0.01` |
+
+As an alternative to passing `atoms`, `lambda`, `maxIter`, or `tol`, these can
+be set with a standalone method.  The following functions can be used before
+calling `Train()`:
+
+ * `lcc.Atoms() = a;` will set the number of atoms to use in the dictionary to
+   `a`.  Changing this after calling `Train()` will not make a difference to the
+   dictionary size.
+
+ * `lcc.Lambda() = l;` will set the L1 regularization penalty to `l1`.  This can
+   be set after `Train()` to force sparser encodings when `Encode()` is called.
+
+ * `lcc.MaxIterations() = m;` will set the maximum number of iterations for
+   dictionary learning to `m`.  `0` means that the algorithm will run until
+   convergence.
+
+ * `lcc.Tolerance() = t;` will set the objective tolerance for convergence of
+   the dictionary learning algorithm to `t`.
+
+***Caveats***:
+
+ * Larger settings of `atoms` (i.e. larger dictionary sizes) will be able to
+   more accurately represent the data, but may take longer to learn.
+
+ * Larger values of `lambda` will cause the model to use sparser encodings for
+   data (e.g. fewer nearby anchor points) when `Train()` and `Encode()` are
+   called, but when `lambda` is too large, the codings may be inaccurate
+   representations of the original points.
+
+<!-- TODO: indicate that you can get this info with MLPACK_PRINT_INFO and
+MLPACK_PRINT_WARN, once those are documented -->
+
+ * If `lambda` is set too large, encodings may be empty (e.g. all zeros).
+
+ * Training is not incremental; a second call to `Train()` will reinitialize the
+   dictionary and restart the learning process.
+
+### Training
+
+If training the dictionary is not done as part of the constructor call, it can
+be done with one of the following versions of the `Train()` member function:
+
+ * `lcc.Train(data)`
+ * `lcc.Train(data, initializer)`
+   - Train the local coordinate coding dictionary on the given `data`.
+   - Optionally, use the given `initializer` to initialize the dictionary (see
+     [`DictionaryInitializer`](#dictionaryinitializer-different-dictionary-initialization-strategies)
+     for more details).
+
+### Encoding
+
+Once a `LocalCoordinateCoding` model has a trained dictionary, the `Encode()`
+member function can be used to encode new data points.
+
+ * `lcc.Encode(data, codes)`
+   - Encode `data` (a [column-major data
+     matrix](../matrices.md#representing-data-in-mlpack)) as a sparse set of
+     local atoms of the dictionary, storing the result in `codes`.
+   - Both `data` and `codes` should be the same matrix type (e.g. `arma::mat`);
+     see [Different Element Types](#mattype-different-element-types) for more
+     details.
+   - `codes` will be set to have `atoms` rows and `data.n_cols` columns.
+   - Column `i` of `codes` corresponds to the coding of the `i`'th column of
+     `data`.  Each row represents the weight associated with each atom in the
+     dictionary.
+
+After encoding, the original data can be recovered (approximately) as
+`lcc.Dictionary() * data`.
+
+### Other Functionality
+
+ * A `LocalCoordinateCoding` model can be serialized with
+   [`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
+
+ * `lcc.Dictionary()` will return an `arma::mat&` containing the dictionary
+   matrix.  The matrix has `data.n_rows` rows and `atoms` columns; each column
+   corresponds to an atom in the dictionary.  Dictionary atoms are regularized
+   to be close to the manifold that data lie on.
+
+ * `double obj = lcc.Objective(data, codes)` computes the local coordinate
+   coding objective function on the given `data` and encodings `codes`.  This
+   can be used after `Encode()` to test the quality of the encodings (a smaller
+   objective is better).
+
+### Simple Examples
+
+See also the [simple usage example](#simple-usage-example) for a trivial usage
+of the `LocalCoordinateCoding` class.
+
+---
+
+Train a local coordinate coding model on the cloud dataset and print the
+reconstruction error.
+
+```c++
+// See https://datasets.mlpack.org/cloud.csv.
+arma::mat dataset;
+mlpack::data::Load("cloud.csv", dataset, true);
+
+mlpack::LocalCoordinateCoding lcc;
+lcc.Atoms() = 50;
+lcc.Lambda() = 1e-5;
+lcc.MaxIterations() = 25;
+lcc.Train(dataset);
+
+// Encode the training dataset.
+arma::mat codes;
+lcc.Encode(dataset, codes);
+
+std::cout << "Input matrix size: " << dataset.n_rows << " x " << dataset.n_cols
+    << "." << std::endl;
+std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
+    << "." << std::endl;
+
+// Reconstruct the original matrix.
+arma::mat recon = lcc.Dictionary() * codes;
+double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
+std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;
+```
+
+---
+
+Train a local coordinate coding model on the iris dataset and save the model to
+disk.
+
+```c++
+// See https://datasets.mlpack.org/iris.train.csv.
+arma::mat dataset;
+mlpack::data::Load("iris.train.csv", dataset, true);
+
+// Train the model in the constructor.
+mlpack::LocalCoordinateCoding lcc(dataset,
+                                  10 /* atoms */,
+                                  0.1 /* L1 penalty */);
+
+// Save the model to disk.
+mlpack::data::Save("lcc.bin", "lcc", lcc);
+```
+
+---
+
+Train a local coordinate coding model on the satellite dataset, trying several
+different regularization parameters and checking the objective value on a
+held-out test dataset.
+
+```c++
+// See https://datasets.mlpack.org/satellite.train.csv.
+arma::mat trainData;
+mlpack::data::Load("satellite.train.csv", trainData, true);
+// See https://datasets.mlpack.org/satellite.test.csv.
+arma::mat testData;
+mlpack::data::Load("satellite.test.csv", testData, true);
+
+for (double lambdaPow = -6; lambdaPow <= -2; lambdaPow += 1)
+{
+  const double lambda = std::pow(10.0, lambdaPow);
+  mlpack::LocalCoordinateCoding lcc(50 /* atoms */);
+  lcc.Lambda() = lambda;
+  lcc.MaxIterations() = 25; // Keep iterations low so this runs relatively fast.
+
+  const double trainObj = lcc.Train(trainData);
+
+  // Compute the objective on the test set.
+  arma::mat codes;
+  lcc.Encode(testData, codes);
+  const double testObj = lcc.Objective(testData, codes);
+
+  std::cout << "Lambda: " << std::setfill(' ') << std::setw(3) << lambda
+      << "; ";
+  std::cout << "training set objective: " << std::setw(6) << trainObj << "; ";
+  std::cout << "test set objective: " << std::setw(6) << testObj << "."
+      << std::endl;
+}
+```
+
+### Advanced Functionality: Template Parameters
+
+The `LocalCoordinateCoding` class has one class template parameter that can be
+used for custom behavior.  The full signature of the class is:
+
+```
+LocalCoordinateCoding<MatType>
+```
+
+In addition, the [constructors](#constructors) and [`Train()`
+functions](#training) have a template parameter `DictionaryInitializer` that can
+be used for custom behavior.
+
+ * `MatType`: the type of the matrix to use (e.g. `arma::mat`, `arma::fmat`,
+   etc.).  The given `MatType` must support the Armadillo API and hold a
+   floating-point element type (e.g. `float`, `double`, etc.).
+
+ * `DictionaryInitializer`: the strategy used to initialize the dictionary.  By
+   default, `DataDependentRandomInitializer` is used.
+
+#### `MatType`: Different Element Types
+
+`MatType` specifies the type of matrix used for training data and internal
+representation of the dictionary.  Any matrix type that implements the Armadillo
+API can be used.  The example below trains a local coordinate coding model on
+32-bit floating point data.
+
+```c++
+// See https://datasets.mlpack.org/cloud.csv.
+arma::fmat dataset;
+mlpack::data::Load("cloud.csv", dataset, true);
+
+mlpack::LocalCoordinateCoding<arma::fmat> lcc;
+lcc.Atoms() = 30;
+lcc.Lambda() = 1e-5;
+lcc.MaxIterations() = 100;
+lcc.Train(dataset);
+
+// Encode the training dataset.
+arma::fmat codes;
+lcc.Encode(dataset, codes);
+
+std::cout << "Input matrix size: " << dataset.n_rows << " x " << dataset.n_cols
+    << "." << std::endl;
+std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
+    << "." << std::endl;
+
+// Reconstruct the original matrix.
+arma::fmat recon = lcc.Dictionary() * codes;
+double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
+std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;
+```
+
+#### `DictionaryInitializer`: Different Dictionary Initialization Strategies
+
+The `DictionaryInitializer` template class specifies the strategy to be used to
+initialize the dictionary when `Train()` is called.
+
+ * The `DataDependentRandomInitalizer` class (the default) uses the average of
+   three random points in the dataset to initialize each atom in the dictionary.
+
+ * The `NothingInitializer` class does not modify the dictionary matrix in any
+   way, and could be used either to set a specific dictionary before training
+   with `sc.Dictionary()`, or to allow incremental training that does not modify
+   the existing dictionary when `Train()` is called a second time.
+
+ * The `RandomInitializer` class initializes the dictionary by sampling norm-1
+   atoms from a normal distribution.
+
+***Note:*** none of the classes above have any members, and as such it is not
+necessary to use the constructor or `Train()` variants that take an initialized
+`initializer` object.  That would only be necessary for a custom
+`DictionaryInitializer` class that stored internal members.
+
+---
+
+The example below uses `NothingInitializer` to set a specific initial
+dictionary.
+
+```c++
+// See https://datasets.mlpack.org/satellite.train.csv.
+arma::mat trainData;
+mlpack::data::Load("satellite.train.csv", trainData, true);
+
+const size_t atoms = 25;
+const double lambda = 1e-5;
+const size_t maxIterations = 50;
+
+// Use a uniform random matrix as the initial dictionary.
+arma::mat initialDictionary(trainData.n_rows, atoms, arma::fill::randu);
+
+mlpack::LocalCoordinateCoding lcc(atoms, lambda, maxIterations);
+lcc.Dictionary() = initialDictionary;
+
+const double obj = lcc.Train<mlpack::NothingInitializer>(trainData);
+std::cout << "Training set objective: " << obj << "." << std::endl;
+```
+
+---
+
+ * An entirely custom class can also be implemented.  The class must implement
+   one method, `Initialize()`:
+
+```c++
+// You can use this as a starting point for implementation.
+class CustomDictionaryInitializer
+{
+ public:
+  // Initialize the dictionary to have the given number of atoms, given the
+  // dataset.  MatType will be the matrix type used by the local coordinate
+  // coding model (e.g. `arma::mat`, `arma::fmat`, etc.).
+  template<typename MatType>
+  void Initialize(const MatType& data,
+                  const size_t atoms,
+                  MatType& dictionary);
+};
+```

doc/user/methods/sparse_coding.md

@@ -41,8 +41,7 @@ std::cout << "Average density of encoded test data: "
 
 #### See also:
 
-<!-- TODO: add LCC link -->
-
+ * [`LocalCoordinateCoding`](local_coordinate_coding.md)
  * [`LARS`](lars.md) (used internally by `SparseCoding`)
  * [mlpack transformations](../../index.md#transformations)
  * [Sparse dictionary learning on Wikipedia](https://en.wikipedia.org/wiki/Sparse_dictionary_learning)
@@ -150,7 +149,8 @@ function can be used to encode new data points.
      of `data`.  Each row represents the weight associated with each atom in
      the dictionary.
 
-After encoding, the original data can be recovered as `sc.Dictionary() * data`.
+After encoding, the original data can be recovered (approximately) as
+`sc.Dictionary() * data`.
 
 ### Other Functionality
 
@@ -288,8 +288,6 @@ In addition, the [constructors](#constructors) and
 [`Train()` functions](#training) have a template parameter
 `DictionaryInitializer` that can be used for custom behavior.
 
-<!-- TODO: check whether it works on sparse data -->
-
  * `MatType`: the type of the matrix to use (e.g. `arma::mat`, `arma::fmat`,
    etc.).  The given `MatType` must support the Armadillo API and hold a
    floating-point element type (e.g. `float`, `double`, etc.).
@@ -297,7 +295,7 @@ In addition, the [constructors](#constructors) and
  * `DictionaryInitializer`: the strategy used to initialize the dictionary.  By
    default, `DataDependentRandomInitializer` is used.
 
-#### ```MatType```: Different Element Types
+#### `MatType`: Different Element Types
 
 `MatType` specifies the type of matrix used for training data and internal
 representation of the dictionary.  Any matrix type that implements the Armadillo
@@ -332,10 +330,9 @@ std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
 arma::fmat recon = sc.Dictionary() * codes;
 double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
 std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;
-
 ```
 
-#### ```DictionaryInitializer```: Different Dictionary Initialization Strategies
+#### `DictionaryInitializer`: Different Dictionary Initialization Strategies
 
 The `DictionaryInitializer` template class specifies the strategy to be used to
 initialize the dictionary when `Train()` is called.

scripts/build-docs.sh

@@ -176,26 +176,26 @@ EOF
 # appends a sidebar list to the output HTML.
 create_page_sidebar_section()
 {
-  input_file="$1";
-  output_file="$2";
-  dir_name="$3"; # The directory containing the documentation.
-  input_file_base=`basename "$input_file" .html.tmp`;
+  sb_input_file="$1";
+  sb_output_file="$2";
+  sb_dir_name="$3"; # The directory containing the documentation.
+  sb_input_file_base=`basename "$sb_input_file" .html.tmp`;
 
   # Extract h2/h3 anchors into a list.  For individual method documentation, we
   # only extract h3 anchors because those use h2s as their headings.  And, for
   # core.md, we want to extract both h2 and h3 anchors.
-  if [[ "$dir_name" == "user/methods" ]];
+  if [[ "$sb_dir_name" == "user/methods" ]];
   then
     # The page title on individual methods is encoded as an h2.
-    page_title=`grep '<h2 id=' "$input_file" |\
+    page_title=`grep '<h2 id=' "$sb_input_file" |\
         head -1 |\
         sed 's/^<h2 id="[^"]*">\(.*\)<\/h2>/\1/'`;
 
-    grep '<h3 id=' "$input_file" | sed 's/<h3 id="\([^"]*\)">\(.*\)<\/h3>/<li><a href="#\1">\2<\/a><\/li>/' > "$output_file.side.tmp";
-  elif [[ "$input_file_base" == "core" ]];
+    grep '<h3 id=' "$sb_input_file" | sed 's/<h3 id="\([^"]*\)">\(.*\)<\/h3>/<li><a href="#\1">\2<\/a><\/li>/' > "$sb_output_file.side.tmp";
+  elif [[ "$sb_input_file_base" == "core" ]];
   then
     # The page title on the core class documentation page is encoded as an h1.
-    page_title=`grep '<h1 id=' "$input_file" |\
+    page_title=`grep '<h1 id=' "$sb_input_file" |\
         head -1 |\
         sed 's/^<h1 id="[^"]*">\(.*\)<\/h1>/\1/'`;
 
@@ -208,8 +208,9 @@ create_page_sidebar_section()
     # ...
     #
     # and then we'll construct the actual sidebar using that list.
-    grep '<h[23] id=' "$input_file" |\
-        sed 's/^<\(h[23]\) id="\([^"]*\)">\(.*\)<\/h[23]>/\1\t\2\t\3/' > "$output_file.side.list.tmp";
+    grep '<h[23] id=' "$sb_input_file" |\
+        sed 's/^<\(h[23]\) id="\([^"]*\)">\(.*\)<\/h[23]>/\1\t\2\t\3/' \
+        > "$sb_output_file.side.list.tmp";
     in_block=0;
     while read line; do
       # First, extract the pieces of each line.
@@ -225,62 +226,64 @@ create_page_sidebar_section()
         if [ "$in_block" = "1" ];
         then
           # We have to close the previous block.
-          echo "</ul></details></li>" >> "$output_file.side.tmp";
+          echo "</ul></details></li>" >> "$sb_output_file.side.tmp";
         fi
 
         # Create the new details block.
-        echo "<li><details><summary>" >> "$output_file.side.tmp";
-        echo "<a href=\"#$anchor_name\">" >> "$output_file.side.tmp";
-        echo "$anchor_title" >> "$output_file.side.tmp";
-        echo "</a>" >> "$output_file.side.tmp";
-        echo "</summary>" >> "$output_file.side.tmp";
-        echo "<ul>" >> "$output_file.side.tmp";
+        echo "<li><details><summary>" >> "$sb_output_file.side.tmp";
+        echo "<a href=\"#$anchor_name\">" >> "$sb_output_file.side.tmp";
+        echo "$anchor_title" >> "$sb_output_file.side.tmp";
+        echo "</a>" >> "$sb_output_file.side.tmp";
+        echo "</summary>" >> "$sb_output_file.side.tmp";
+        echo "<ul>" >> "$sb_output_file.side.tmp";
         in_block=1;
       else
-        echo "  <li><a href=\"#$anchor_name\">" >> "$output_file.side.tmp";
-        echo "  $anchor_title" >> "$output_file.side.tmp";
-        echo "  </a></li>" >> "$output_file.side.tmp";
+        echo "  <li><a href=\"#$anchor_name\">" >> "$sb_output_file.side.tmp";
+        echo "  $anchor_title" >> "$sb_output_file.side.tmp";
+        echo "  </a></li>" >> "$sb_output_file.side.tmp";
       fi
-    done < "$output_file.side.list.tmp";
+    done < "$sb_output_file.side.list.tmp";
 
     # Close the last h2 block, if we need to.
     if [ "$in_block" = "1" ];
     then
-      echo "</ul></details></li>" >> "$output_file.side.tmp";
+      echo "</ul></details></li>" >> "$sb_output_file.side.tmp";
     fi
 
-    rm -f "$output_file.side.list.tmp";
+    rm -f "$sb_output_file.side.list.tmp";
   else
     # On other pages, the page title is encoded as an h1.
-    page_title=`grep '<h1 id=' "$input_file" |\
+    page_title=`grep '<h1 id=' "$sb_input_file" |\
         head -1 |\
         sed 's/^<h1 id="[^"]*">\(.*\)<\/h1>/\1/'`;
 
-    grep '<h2 id=' "$input_file" | sed 's/<h2 id="\([^"]*\)">\(.*\)<\/h2>/<li><a href="#\1">\2<\/a><\/li>/' > "$output_file.side.tmp";
+    grep '<h2 id=' "$sb_input_file" |\
+        sed 's/<h2 id="\([^"]*\)">\(.*\)<\/h2>/<li><a href="#\1">\2<\/a><\/li>/' \
+        > "$sb_output_file.side.tmp";
   fi
-  lines=`cat "$output_file.side.tmp" | wc -l`;
+  lines=`cat "$sb_output_file.side.tmp" | wc -l`;
 
-  echo "<ul>" >> "$output_file";
+  echo "<ul>" >> "$sb_output_file";
 
   # Make the top of the sidebar.
   if [ -n "$page_title" ];
   then
-    echo "<li class=\"page_title\"><b>$page_title</b> <a href=\"#\">[top]</a>" >> "$output_file";
+    echo "<li class=\"page_title\"><b>$page_title</b> <a href=\"#\">[top]</a>" >> "$sb_output_file";
   else
-    echo "<li><a href=\"#\">[top of page]</a>" >> "$output_file";
+    echo "<li><a href=\"#\">[top of page]</a>" >> "$sb_output_file";
   fi
 
   if [[ "$lines" -gt 0 ]];
   then
-    echo "<ul>" >> "$output_file";
-    cat "$output_file.side.tmp" >> "$output_file";
-    echo "</ul>" >> "$output_file";
+    echo "<ul>" >> "$sb_output_file";
+    cat "$sb_output_file.side.tmp" >> "$sb_output_file";
+    echo "</ul>" >> "$sb_output_file";
   fi
-  echo "</li>" >> "$output_file";
-  echo "</ul>" >> "$output_file";
-  echo "</div>" >> "$output_file";
+  echo "</li>" >> "$sb_output_file";
+  echo "</ul>" >> "$sb_output_file";
+  echo "</div>" >> "$sb_output_file";
 
-  rm -f "$output_file.side.tmp";
+  rm -f "$sb_output_file.side.tmp";
 }
 
 rm -rf "$output_dir";

src/mlpack/methods/local_coordinate_coding/lcc.hpp

@@ -75,9 +75,13 @@ namespace mlpack {
  * }
  * @endcode
  */
+template<typename MatType = arma::mat>
 class LocalCoordinateCoding
 {
  public:
+  typedef typename GetColType<MatType>::type ColType;
+  typedef typename GetRowType<MatType>::type RowType;
+
   /**
    * Set the parameters to LocalCoordinateCoding, and train the dictionary.
    * This constructor will also initialize the dictionary using the given
@@ -85,7 +89,7 @@ class LocalCoordinateCoding
    *
    * If you want to initialize the dictionary to a custom matrix, consider
    * either writing your own DictionaryInitializer class (with void
-   * Initialize(const arma::mat& data, arma::mat& dictionary) function), or call
+   * Initialize(const MatType& data, MatType& dictionary) function), or call
    * the constructor that does not take a data matrix, then call Dictionary() to
    * set the dictionary matrix to a matrix of your choosing, and then call
    * Train() with NothingInitializer (i.e.  Train<NothingInitializer>(data)).
@@ -99,7 +103,7 @@ class LocalCoordinateCoding
    * @param initializer Intializer to use.
    */
   template<typename DictionaryInitializer = DataDependentRandomInitializer>
-  LocalCoordinateCoding(const arma::mat& data,
+  LocalCoordinateCoding(const MatType& data,
                         const size_t atoms,
                         const double lambda,
                         const size_t maxIterations = 0,
@@ -132,7 +136,7 @@ class LocalCoordinateCoding
    * @return The final objective value.
    */
   template<typename DictionaryInitializer = DataDependentRandomInitializer>
-  double Train(const arma::mat& data,
+  double Train(const MatType& data,
                const DictionaryInitializer& initializer =
                    DictionaryInitializer());
 
@@ -142,7 +146,7 @@ class LocalCoordinateCoding
    * @param data Matrix containing points to encode.
    * @param codes Output matrix to store codes in.
    */
-  void Encode(const arma::mat& data, arma::mat& codes);
+  void Encode(const MatType& data, MatType& codes);
 
   /**
    * Learn dictionary by solving linear system.
@@ -153,10 +157,19 @@ class LocalCoordinateCoding
    *    the coding matrix Z that are non-zero (the adjacency matrix for the
    *    bipartite graph of points and atoms)
    */
-  void OptimizeDictionary(const arma::mat& data,
-                          const arma::mat& codes,
+  void OptimizeDictionary(const MatType& data,
+                          const MatType& codes,
                           const arma::uvec& adjacencies);
 
+  /**
+   * Compute objective function given the list of adjacencies.
+   *
+   * @param data Matrix containing points to encode.
+   * @param codes Output matrix to store codes in.
+   */
+  double Objective(const MatType& data,
+                   const MatType& codes) const;
+
   /**
    * Compute objective function given the list of adjacencies.
    *
@@ -166,8 +179,8 @@ class LocalCoordinateCoding
    *    the coding matrix Z that are non-zero (the adjacency matrix for the
    *    bipartite graph of points and atoms)
    */
-  double Objective(const arma::mat& data,
-                   const arma::mat& codes,
+  double Objective(const MatType& data,
+                   const MatType& codes,
                    const arma::uvec& adjacencies) const;
 
   //! Get the number of atoms.
@@ -176,9 +189,9 @@ class LocalCoordinateCoding
   size_t& Atoms() { return atoms; }
 
   //! Accessor for dictionary.
-  const arma::mat& Dictionary() const { return dictionary; }
+  const MatType& Dictionary() const { return dictionary; }
   //! Mutator for dictionary.
-  arma::mat& Dictionary() { return dictionary; }
+  MatType& Dictionary() { return dictionary; }
 
   //! Get the L1 regularization parameter.
   double Lambda() const { return lambda; }
@@ -204,7 +217,7 @@ class LocalCoordinateCoding
   size_t atoms;
 
   //! Dictionary (columns are atoms).
-  arma::mat dictionary;
+  MatType dictionary;
 
   //! l1 regularization term.
   double lambda;
@@ -217,6 +230,9 @@ class LocalCoordinateCoding
 
 } // namespace mlpack
 
+CEREAL_TEMPLATE_CLASS_VERSION((typename MatType),
+    (mlpack::LocalCoordinateCoding<MatType>), (1));
+
 // Include implementation.
 #include "lcc_impl.hpp"
 

src/mlpack/methods/local_coordinate_coding/lcc_impl.hpp

@@ -17,9 +17,10 @@
 
 namespace mlpack {
 
+template<typename MatType>
 template<typename DictionaryInitializer>
-LocalCoordinateCoding::LocalCoordinateCoding(
-    const arma::mat& data,
+LocalCoordinateCoding<MatType>::LocalCoordinateCoding(
+    const MatType& data,
     const size_t atoms,
     const double lambda,
     const size_t maxIterations,
@@ -34,7 +35,8 @@ LocalCoordinateCoding::LocalCoordinateCoding(
   Train(data, initializer);
 }
 
-inline LocalCoordinateCoding::LocalCoordinateCoding(
+template<typename MatType>
+inline LocalCoordinateCoding<MatType>::LocalCoordinateCoding(
     const size_t atoms,
     const double lambda,
     const size_t maxIterations,
@@ -47,9 +49,10 @@ inline LocalCoordinateCoding::LocalCoordinateCoding(
   // Nothing to do.
 }
 
+template<typename MatType>
 template<typename DictionaryInitializer>
-double LocalCoordinateCoding::Train(
-    const arma::mat& data,
+double LocalCoordinateCoding<MatType>::Train(
+    const MatType& data,
     const DictionaryInitializer& initializer)
 {
   // Initialize the dictionary.
@@ -61,7 +64,7 @@ double LocalCoordinateCoding::Train(
   // loop.
   Log::Info << "Initial Coding Step." << std::endl;
 
-  arma::mat codes;
+  MatType codes;
   Encode(data, codes);
   arma::uvec adjacencies = find(codes);
 
@@ -115,15 +118,16 @@ double LocalCoordinateCoding::Train(
   return lastObjVal;
 }
 
-inline void LocalCoordinateCoding::Encode(const arma::mat& data,
-                                          arma::mat& codes)
+template<typename MatType>
+inline void LocalCoordinateCoding<MatType>::Encode(const MatType& data,
+                                                   MatType& codes)
 {
-  arma::mat invSqDists = 1.0 / (repmat(trans(sum(square(dictionary))), 1,
+  MatType invSqDists = 1.0 / (repmat(trans(sum(square(dictionary))), 1,
       data.n_cols) + repmat(sum(square(data)), atoms, 1) - 2 * trans(dictionary)
       * data);
 
-  arma::mat dictGram = trans(dictionary) * dictionary;
-  arma::mat dictGramTD(dictGram.n_rows, dictGram.n_cols);
+  MatType dictGram = trans(dictionary) * dictionary;
+  MatType dictGramTD(dictGram.n_rows, dictGram.n_cols);
 
   codes.set_size(atoms, data.n_cols);
   for (size_t i = 0; i < data.n_cols; ++i)
@@ -134,29 +138,31 @@ inline void LocalCoordinateCoding::Encode(const arma::mat& data,
       Log::Debug << "Optimization at point " << i << "." << std::endl;
     }
 
-    arma::vec invW = invSqDists.unsafe_col(i);
-    arma::mat dictPrime = dictionary * diagmat(invW);
+    ColType invW = invSqDists.unsafe_col(i);
+    MatType dictPrime = dictionary * diagmat(invW);
 
-    arma::mat dictGramTD = diagmat(invW) * dictGram * diagmat(invW);
+    MatType dictGramTD = diagmat(invW) * dictGram * diagmat(invW);
 
     bool useCholesky = false;
     // Normalization and fitting and intercept are disabled.
-    LARS<> lars(useCholesky, 0.5 * lambda, 0, 1e-16 /* default tolerance */,
-        false, false);
+    const double tol = std::is_same<typename MatType::elem_type, float>::value ?
+        1e-8 : 1e-16;
+    LARS<MatType> lars(useCholesky, 0.5 * lambda, 0, tol, false, false);
 
     // Run LARS for this point, by making an alias of the point and passing
     // that.
-    arma::vec beta = codes.unsafe_col(i);
-    arma::rowvec responses = data.unsafe_col(i).t();
+    ColType beta = codes.unsafe_col(i);
+    RowType responses = data.unsafe_col(i).t();
     lars.Train(dictPrime, responses, false, useCholesky, dictGramTD);
     beta = lars.Beta();
     beta %= invW; // Remember, beta is an alias of codes.col(i).
   }
 }
 
-inline void LocalCoordinateCoding::OptimizeDictionary(
-    const arma::mat& data,
-    const arma::mat& codes,
+template<typename MatType>
+inline void LocalCoordinateCoding<MatType>::OptimizeDictionary(
+    const MatType& data,
+    const MatType& codes,
     const arma::uvec& adjacencies)
 {
   // Count number of atomic neighbors for each point x^i.
@@ -184,7 +190,8 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
 
   // Build dataPrime := [X x^1 ... x^1 ... x^n ... x^n]
   // where each x^i is repeated for the number of neighbors x^i has.
-  arma::mat dataPrime = zeros(data.n_rows, data.n_cols + adjacencies.n_elem);
+  MatType dataPrime = zeros<MatType>(data.n_rows,
+      data.n_cols + adjacencies.n_elem);
 
   dataPrime(arma::span::all, arma::span(0, data.n_cols - 1)) = data;
 
@@ -209,9 +216,9 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
   const size_t nInactiveAtoms = atoms - nActiveAtoms;
 
   // Efficient construction of codes restricted to active atoms.
-  arma::mat codesPrime = zeros(nActiveAtoms, data.n_cols +
+  MatType codesPrime = zeros<MatType>(nActiveAtoms, data.n_cols +
       adjacencies.n_elem);
-  arma::vec wSquared = ones(data.n_cols + adjacencies.n_elem, 1);
+  ColType wSquared = ones<ColType>(data.n_cols + adjacencies.n_elem, 1);
 
   if (nInactiveAtoms > 0)
   {
@@ -219,13 +226,13 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
         << " inactive atoms.  They will be re-initialized randomly.\n";
 
     // Create matrix holding only active codes.
-    arma::mat activeCodes = codes.rows(arma::uvec(activeAtoms));
+    MatType activeCodes = codes.rows(arma::uvec(activeAtoms));
 
     // Create reverse atom lookup for active atoms.
     arma::uvec atomReverseLookup(atoms);
     for (size_t i = 0; i < activeAtoms.size(); ++i)
     {
-      atomReverseLookup[i] = activeAtoms[i];
+      atomReverseLookup[activeAtoms[i]] = i;
     }
 
     codesPrime(arma::span::all, arma::span(0, data.n_cols - 1)) = activeCodes;
@@ -259,15 +266,18 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
     }
   }
 
-  wSquared.subvec(data.n_cols, wSquared.n_elem - 1) = lambda *
-      abs(wSquared.subvec(data.n_cols, wSquared.n_elem - 1));
+  if (adjacencies.n_elem > 0)
+  {
+    wSquared.subvec(data.n_cols, wSquared.n_elem - 1) = lambda *
+        abs(wSquared.subvec(data.n_cols, wSquared.n_elem - 1));
+  }
 
   // Solve system.
   if (nInactiveAtoms == 0)
   {
     // No inactive atoms.  We can solve directly.
-    arma::mat A = codesPrime * diagmat(wSquared) * trans(codesPrime);
-    arma::mat B = codesPrime * diagmat(wSquared) * trans(dataPrime);
+    MatType A = codesPrime * diagmat(wSquared) * trans(codesPrime);
+    MatType B = codesPrime * diagmat(wSquared) * trans(dataPrime);
 
     dictionary = trans(solve(A, B));
     /*
@@ -279,7 +289,7 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
   {
     // Inactive atoms must be reinitialized randomly, so we cannot solve
     // directly for the entire dictionary estimate.
-    arma::mat dictionaryActive =
+    MatType dictionaryActive =
         trans(solve(codesPrime * diagmat(wSquared) * trans(codesPrime),
                     codesPrime * diagmat(wSquared) * trans(dataPrime)));
 
@@ -310,9 +320,19 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
   }
 }
 
-inline double LocalCoordinateCoding::Objective(
-    const arma::mat& data,
-    const arma::mat& codes,
+template<typename MatType>
+inline double LocalCoordinateCoding<MatType>::Objective(
+    const MatType& data,
+    const MatType& codes) const
+{
+  // Compute adjacencies and pass off to other overload.
+  return Objective(data, codes, find(codes));
+}
+
+template<typename MatType>
+inline double LocalCoordinateCoding<MatType>::Objective(
+    const MatType& data,
+    const MatType& codes,
     const arma::uvec& adjacencies) const
 {
   double weightedL1NormZ = 0;
@@ -331,11 +351,26 @@ inline double LocalCoordinateCoding::Objective(
   return std::pow(froNormResidual, 2.0) + lambda * weightedL1NormZ;
 }
 
+template<typename MatType>
 template<typename Archive>
-void LocalCoordinateCoding::serialize(Archive& ar,
-                                      const uint32_t /* version */)
+void LocalCoordinateCoding<MatType>::serialize(Archive& ar,
+                                               const uint32_t version)
 {
   ar(CEREAL_NVP(atoms));
+
+  if (cereal::is_loading<Archive>() && version == 0)
+  {
+    // Older versions of LocalCoordinateCoding always stored dictionary as an
+    // arma::mat.
+    arma::mat dictionaryTmp;
+    ar(cereal::make_nvp("dictionary", dictionaryTmp));
+    dictionary = ConvTo<MatType>::From(dictionaryTmp);
+  }
+  else
+  {
+    ar(CEREAL_NVP(dictionary));
+  }
+
   ar(CEREAL_NVP(dictionary));
   ar(CEREAL_NVP(lambda));
   ar(CEREAL_NVP(maxIterations));

src/mlpack/methods/local_coordinate_coding/local_coordinate_coding_main.cpp

@@ -90,8 +90,7 @@ BINDING_SEE_ALSO("Nonlinear learning using local coordinate coding (pdf)",
         "https://papers.nips.cc/paper/3875-nonlinear-learning-using-local-"
         "coordinate-coding.pdf");
 BINDING_SEE_ALSO("LocalCoordinateCoding C++ class documentation",
-        "@src/mlpack/methods/local_coordinate_coding/local_coordinate_coding."
-        "hpp");
+        "@doc/user/methods/local_coordinate_coding.md");
 
 // Training parameters.
 PARAM_MATRIX_IN("training", "Matrix of training data (X).", "t");
@@ -106,9 +105,9 @@ PARAM_FLAG("normalize", "If set, the input data matrix will be normalized "
 PARAM_DOUBLE_IN("tolerance", "Tolerance for objective function.", "o", 0.01);
 
 // Load/save a model.
-PARAM_MODEL_IN(LocalCoordinateCoding, "input_model", "Input LCC model.", "m");
-PARAM_MODEL_OUT(LocalCoordinateCoding, "output_model", "Output for trained LCC "
-    "model.", "M");
+PARAM_MODEL_IN(LocalCoordinateCoding<>, "input_model", "Input LCC model.", "m");
+PARAM_MODEL_OUT(LocalCoordinateCoding<>, "output_model",
+    "Output for trained LCC model.", "M");
 
 // Test on another dataset.
 PARAM_MATRIX_IN("test", "Test points to encode.", "T");
@@ -143,9 +142,9 @@ void BINDING_FUNCTION(util::Params& params, util::Timers& timers)
   ReportIgnoredParam(params, {{ "training", false }}, "tolerance");
 
   // Do we have an existing model?
-  LocalCoordinateCoding* lcc = NULL;
+  LocalCoordinateCoding<>* lcc = NULL;
   if (params.Has("input_model"))
-    lcc = params.Get<LocalCoordinateCoding*>("input_model");
+    lcc = params.Get<LocalCoordinateCoding<>*>("input_model");
 
   if (params.Has("training"))
   {
@@ -171,7 +170,7 @@ void BINDING_FUNCTION(util::Params& params, util::Timers& timers)
         [](double x) { return x > 0; }, 1,
         "Tolerance should be a positive real number");
 
-    lcc = new LocalCoordinateCoding(0, 0.0);
+    lcc = new LocalCoordinateCoding<>(0, 0.0);
 
     lcc->Lambda() = params.Get<double>("lambda");
     lcc->Atoms() = (size_t) params.Get<int>("atoms");
@@ -257,5 +256,5 @@ void BINDING_FUNCTION(util::Params& params, util::Timers& timers)
 
   // Save the dictionary and the model.
   params.Get<mat>("dictionary") = lcc->Dictionary();
-  params.Get<LocalCoordinateCoding*>("output_model") = lcc;
+  params.Get<LocalCoordinateCoding<>*>("output_model") = lcc;
 }

src/mlpack/methods/sparse_coding/sparse_coding.hpp

@@ -108,7 +108,7 @@ namespace mlpack {
  * the Encode() function.
  *
  * @tparam DictionaryInitializationPolicy The class to use to initialize the
- *     dictionary; must have 'void Initialize(const arma::mat& data, arma::mat&
+ *     dictionary; must have 'void Initialize(const MatType& data, MatType&
  *     dictionary)' function.
  */
 template<typename MatType = arma::mat>
@@ -127,7 +127,7 @@ class SparseCoding
    *
    * If you want to initialize the dictionary to a custom matrix, consider
    * either writing your own DictionaryInitializer class (with void
-   * Initialize(const arma::mat& data, arma::mat& dictionary) function), or call
+   * Initialize(const MatType& data, MatType& dictionary) function), or call
    * the constructor that does not take a data matrix, then call Dictionary() to
    * set the dictionary matrix to a matrix of your choosing, and then call
    * Train() with NothingInitializer (i.e. Train<NothingInitializer>(data)).

src/mlpack/tests/ann/convolutional_network_test.cpp

@@ -71,7 +71,7 @@ void CheckMoveFunction(ModelType* network1,
 TEST_CASE("PaddingTest", "[ConvolutionalNetworktest]")
 {
   arma::mat X;
-  X.load("mnist_first250_training_4s_and_9s.arm");
+  X.load("mnist_first250_training_4s_and_9s.csv");
 
   // Create the network.
   FFN<NegativeLogLikelihood, RandomInitialization> model;
@@ -148,7 +148,7 @@ TEST_CASE("MaxPoolingTest", "[ConvolutionalNetworkTest]")
 TEST_CASE("VanillaNetworkTest", "[ConvolutionalNetworkTest]")
 {
   arma::mat X;
-  X.load("mnist_first250_training_4s_and_9s.arm");
+  X.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   arma::uword nPoints = X.n_cols;
@@ -264,7 +264,7 @@ TEST_CASE("VanillaNetworkBatchSizeTest", "[ConvolutionalNetworkTest]")
   model.InputDimensions() = std::vector<size_t>({ 28, 28 });
 
   arma::mat X;
-  X.load("mnist_first250_training_4s_and_9s.arm");
+  X.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   arma::uword nPoints = X.n_cols;
@@ -347,7 +347,7 @@ TEST_CASE("VanillaNetworkBatchSizeTest", "[ConvolutionalNetworkTest]")
 TEST_CASE("CheckCopyVanillaNetworkTest", "[ConvolutionalNetworkTest]")
 {
   arma::mat X;
-  X.load("mnist_first250_training_4s_and_9s.arm");
+  X.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   arma::uword nPoints = X.n_cols;

src/mlpack/tests/ann/feedforward_network_test.cpp

@@ -400,7 +400,7 @@ TEST_CASE("FFVanillaNetworkTest", "[FeedForwardNetworkTest]")
   TestNetwork<>(model, trainData, trainLabels, testData, testLabels, 10, 0.1);
 
   arma::mat dataset;
-  dataset.load("mnist_first250_training_4s_and_9s.arm");
+  dataset.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -421,7 +421,7 @@ TEST_CASE("FFVanillaNetworkTest", "[FeedForwardNetworkTest]")
 TEST_CASE("ForwardBackwardTest", "[FeedForwardNetworkTest]")
 {
   arma::mat dataset;
-  dataset.load("mnist_first250_training_4s_and_9s.arm");
+  dataset.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -548,7 +548,7 @@ TEST_CASE("DropoutNetworkTest", "[FeedForwardNetworkTest]")
   // network must be significant better than 92%.
   TestNetwork<>(model, trainData, trainLabels, testData, testLabels, 10, 0.1);
   arma::mat dataset;
-  dataset.load("mnist_first250_training_4s_and_9s.arm");
+  dataset.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -627,7 +627,7 @@ TEST_CASE("DropConnectNetworkTest", "[FeedForwardNetworkTest]")
   TestNetwork(model, trainData, trainLabels, testData, testLabels, 10, 0.1);
 
   arma::mat dataset;
-  dataset.load("mnist_first250_training_4s_and_9s.arm");
+  dataset.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -952,7 +952,7 @@ TEST_CASE("RBFNetworkTest", "[FeedForwardNetworkTest]")
   TestNetwork<>(model, trainData, trainLabels1, testData, testLabels, 10, 0.1);
 
   arma::mat dataset;
-  dataset.load("mnist_first250_training_4s_and_9s.arm");
+  dataset.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   for (size_t i = 0; i < dataset.n_cols; ++i)

src/mlpack/tests/ann/not_adapted/dcgan_test.cpp

@@ -53,7 +53,7 @@ TEST_CASE("DCGANMNISTTest", "[DCGANNetworkTest]")
       << " shuffle = " << shuffle << std::endl;
 
   arma::mat trainData;
-  trainData.load("mnist_first250_training_4s_and_9s.arm");
+  trainData.load("mnist_first250_training_4s_and_9s.csv");
   Log::Info << arma::size(trainData) << std::endl;
 
   trainData = trainData.cols(0, datasetMaxCols - 1);
@@ -212,7 +212,7 @@ TEST_CASE("DCGANMNISTTest", "[DCGANNetworkTest]")
       << " shuffle = " << shuffle << std::endl;
 
   arma::mat trainData;
-  trainData.load("mnist_first250_training_4s_and_9s.arm");
+  trainData.load("mnist_first250_training_4s_and_9s.csv");
   Log::Info << arma::size(trainData) << std::endl;
 
   // trainData = trainData.cols(0, datasetMaxCols - 1);

src/mlpack/tests/ann/not_adapted/feedforward_network_test.cpp

@@ -134,7 +134,7 @@ TEST_CASE("CheckCopyMovingReparametrizationNetworkTest",
 TEST_CASE("HighwayNetworkTest", "[FeedForwardNetworkTest]")
 {
   arma::mat dataset;
-  dataset.load("mnist_first250_training_4s_and_9s.arm");
+  dataset.load("mnist_first250_training_4s_and_9s.csv");
 
   // Normalize each point since these are images.
   for (size_t i = 0; i < dataset.n_cols; ++i)

src/mlpack/tests/ann/not_adapted/gan_test.cpp

@@ -156,7 +156,7 @@ TEST_CASE("GANMNISTTest", "[GANNetworkTest]")
       << " shuffle = " << shuffle << std::endl;
 
   arma::mat trainData;
-  trainData.load("mnist_first250_training_4s_and_9s.arm");
+  trainData.load("mnist_first250_training_4s_and_9s.csv");
   Log::Info << arma::size(trainData) << std::endl;
 
   trainData = trainData.cols(0, datasetMaxCols - 1);

src/mlpack/tests/ann/not_adapted/wgan_test.cpp

@@ -60,7 +60,7 @@ TEST_CASE("WGANMNISTTest", "[WGANNetworkTest]")
       << " shuffle = " << shuffle << std::endl;
 
   arma::mat trainData;
-  trainData.load("mnist_first250_training_4s_and_9s.arm");
+  trainData.load("mnist_first250_training_4s_and_9s.csv");
   Log::Info << arma::size(trainData) << std::endl;
 
   trainData = trainData.cols(0, datasetMaxCols - 1);
@@ -222,7 +222,7 @@ TEST_CASE("WGANGPMNISTTest", "[WGANNetworkTest]")
       << " shuffle = " << shuffle << std::endl;
 
   arma::mat trainData;
-  trainData.load("mnist_first250_training_4s_and_9s.arm");
+  trainData.load("mnist_first250_training_4s_and_9s.csv");
   Log::Info << arma::size(trainData) << std::endl;
 
   trainData = trainData.cols(0, datasetMaxCols - 1);

src/mlpack/tests/local_coordinate_coding_test.cpp

@@ -18,7 +18,10 @@
 using namespace arma;
 using namespace mlpack;
 
-void VerifyCorrectness(const vec& beta, const vec& errCorr, double lambda)
+template<typename MatType, typename VecType>
+void VerifyCorrectness(const MatType& beta,
+                       const VecType& errCorr,
+                       const double lambda)
 {
   const double tol = 0.1;
   size_t nDims = beta.n_elem;
@@ -43,15 +46,18 @@ void VerifyCorrectness(const vec& beta, const vec& errCorr, double lambda)
   }
 }
 
-
-TEST_CASE("LocalCoordinateCodingTestCodingStep",
-          "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingTestCodingStep",
+    "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
 {
+  typedef TestType MatType;
+  typedef arma::Col<typename MatType::elem_type> VecType;
+
   double lambda1 = 0.1;
   uword nAtoms = 10;
 
-  mat X;
-  X.load("mnist_first250_training_4s_and_9s.arm");
+  mat inX; // The .arm file is saved as an arma::mat.
+  inX.load("mnist_first250_training_4s_and_9s.csv");
+  MatType X = arma::conv_to<MatType>::from(inX);
   uword nPoints = X.n_cols;
 
   // normalize each point since these are images
@@ -60,37 +66,40 @@ TEST_CASE("LocalCoordinateCodingTestCodingStep",
     X.col(i) /= norm(X.col(i), 2);
   }
 
-  mat Z;
-  LocalCoordinateCoding lcc(X, nAtoms, lambda1, 10);
+  MatType Z;
+  LocalCoordinateCoding<MatType> lcc(X, nAtoms, lambda1, 10);
   lcc.Encode(X, Z);
 
-  mat D = lcc.Dictionary();
+  MatType D = lcc.Dictionary();
 
   for (uword i = 0; i < nPoints; ++i)
   {
-    vec sqDists = vec(nAtoms);
+    VecType sqDists(nAtoms);
     for (uword j = 0; j < nAtoms; ++j)
     {
       sqDists[j] = arma::norm(D.col(j) - X.col(i));
     }
-    mat Dprime = D * diagmat(1.0 / sqDists);
-    mat zPrime = Z.unsafe_col(i) % sqDists;
+    MatType Dprime = D * diagmat(1.0 / sqDists);
+    MatType zPrime = Z.unsafe_col(i) % sqDists;
 
-    vec errCorr = trans(Dprime) * (Dprime * zPrime - X.unsafe_col(i));
+    VecType errCorr = trans(Dprime) * (Dprime * zPrime - X.unsafe_col(i));
     VerifyCorrectness(zPrime, errCorr, 0.5 * lambda1);
   }
 }
 
-TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
-          "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
+    "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
 {
+  typedef TestType MatType;
+
   const double tol = 0.1;
 
   double lambda = 0.1;
   uword nAtoms = 10;
 
-  mat X;
-  X.load("mnist_first250_training_4s_and_9s.arm");
+  mat inX; // File is saved as an arma::mat.
+  inX.load("mnist_first250_training_4s_and_9s.csv");
+  MatType X = arma::conv_to<MatType>::from(inX);
   uword nPoints = X.n_cols;
 
   // normalize each point since these are images
@@ -99,15 +108,15 @@ TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
     X.col(i) /= norm(X.col(i), 2);
   }
 
-  mat Z;
-  LocalCoordinateCoding lcc(X, nAtoms, lambda, 10);
+  MatType Z;
+  LocalCoordinateCoding<MatType> lcc(X, nAtoms, lambda, 10);
   lcc.Encode(X, Z);
   uvec adjacencies = find(Z);
   lcc.OptimizeDictionary(X, Z, adjacencies);
 
-  mat D = lcc.Dictionary();
+  MatType D = lcc.Dictionary();
 
-  mat grad = zeros(D.n_rows, D.n_cols);
+  MatType grad = zeros<MatType>(D.n_rows, D.n_cols);
   for (uword i = 0; i < nPoints; ++i)
   {
     grad += (D - repmat(X.unsafe_col(i), 1, nAtoms)) *
@@ -118,26 +127,30 @@ TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
   REQUIRE(norm(grad, "fro") == Approx(0.0).margin(tol));
 }
 
-TEST_CASE("LocalCoordinateCodingSerializationTest",
-          "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingSerializationTest",
+    "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
 {
-  mat X = randu<mat>(100, 100);
+  typedef TestType MatType;
+
+  MatType X = randu<MatType>(100, 100);
   size_t nAtoms = 10;
 
-  LocalCoordinateCoding lcc(nAtoms, 0.05, 2 /* don't care about quality */);
+  LocalCoordinateCoding<MatType> lcc(nAtoms, 0.05,
+      2 /* don't care about quality */);
   lcc.Train(X);
 
-  mat Y = randu<mat>(100, 200);
-  mat codes;
+  MatType Y = randu<MatType>(100, 200);
+  MatType codes;
   lcc.Encode(Y, codes);
 
-  LocalCoordinateCoding lccXml(50, 0.1), lccJson(12, 0.0), lccBinary(0, 0.0);
+  LocalCoordinateCoding<MatType> lccXml(50, 0.1), lccJson(12, 0.0),
+      lccBinary(0, 0.0);
   SerializeObjectAll(lcc, lccXml, lccJson, lccBinary);
 
   CheckMatrices(lcc.Dictionary(), lccXml.Dictionary(), lccJson.Dictionary(),
       lccBinary.Dictionary());
 
-  mat xmlCodes, jsonCodes, binaryCodes;
+  MatType xmlCodes, jsonCodes, binaryCodes;
   lccXml.Encode(Y, xmlCodes);
   lccJson.Encode(Y, jsonCodes);
   lccBinary.Encode(Y, binaryCodes);
@@ -167,14 +180,17 @@ TEST_CASE("LocalCoordinateCodingSerializationTest",
  * Test that LocalCoordinateCoding::Train() returns finite final objective
  * value.
  */
-TEST_CASE("LocalCoordinateCodingTrainReturnObjective",
-          "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingTrainReturnObjective",
+    "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
 {
+  typedef TestType MatType;
+
   double lambda1 = 0.1;
   uword nAtoms = 10;
 
-  mat X;
-  X.load("mnist_first250_training_4s_and_9s.arm");
+  mat inX; // File is saved as arma::mat.
+  inX.load("mnist_first250_training_4s_and_9s.csv");
+  MatType X = arma::conv_to<MatType>::from(inX);
   uword nPoints = X.n_cols;
 
   // Normalize each point since these are images.
@@ -183,7 +199,7 @@ TEST_CASE("LocalCoordinateCodingTrainReturnObjective",
     X.col(i) /= norm(X.col(i), 2);
   }
 
-  LocalCoordinateCoding lcc(nAtoms, lambda1, 10);
+  LocalCoordinateCoding<MatType> lcc(nAtoms, lambda1, 10);
   double objVal = lcc.Train(X);
 
   REQUIRE(std::isfinite(objVal) == true);

src/mlpack/tests/main_tests/local_coordinate_coding_test.cpp

@@ -32,7 +32,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCDimensionsTest",
                  "[LCCMainTest][BindingTests]")
 {
   arma::mat x;
-  x.load("mnist_first250_training_4s_and_9s.arm");
+  x.load("mnist_first250_training_4s_and_9s.csv");
   int rows = x.n_rows, cols = x.n_cols;
   arma::mat t = x;
   int atoms = 10;
@@ -58,7 +58,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCOutputModelTest",
                  "[LCCMainTest][BindingTests]")
 {
   arma::mat x;
-  x.load("mnist_first250_training_4s_and_9s.arm");
+  x.load("mnist_first250_training_4s_and_9s.csv");
   arma::mat t = x;
 
   SetInputParam("training", std::move(x));
@@ -71,8 +71,8 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCOutputModelTest",
   // Get the encoded output and dictionary after training.
   arma::mat initCodes = std::move(params.Get<arma::mat>("codes"));
   arma::mat initDict = std::move(params.Get<arma::mat>("dictionary"));
-  LocalCoordinateCoding* outputModel =
-      std::move(params.Get<LocalCoordinateCoding*>("output_model"));
+  LocalCoordinateCoding<>* outputModel =
+      std::move(params.Get<LocalCoordinateCoding<>*>("output_model"));
 
   ResetSettings();
 
@@ -129,7 +129,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainAndTestDataDimTest",
                  "[LCCMainTest][BindingTests]")
 {
   arma::mat x;
-  x.load("mnist_first250_training_4s_and_9s.arm");
+  x.load("mnist_first250_training_4s_and_9s.csv");
   arma::mat t = x;
 
   t.shed_rows(1, 2);
@@ -150,7 +150,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainAndInputModelTest",
                  "[LCCMainTest][BindingTests]")
 {
   arma::mat x;
-  x.load("mnist_first250_training_4s_and_9s.arm");
+  x.load("mnist_first250_training_4s_and_9s.csv");
 
   SetInputParam("training", x);
   SetInputParam("atoms", (int) 10);
@@ -158,8 +158,8 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainAndInputModelTest",
 
   RUN_BINDING();
 
-  LocalCoordinateCoding* outputModel =
-      std::move(params.Get<LocalCoordinateCoding*>("output_model"));
+  LocalCoordinateCoding<>* outputModel =
+      std::move(params.Get<LocalCoordinateCoding<>*>("output_model"));
 
   // No need to input training data again.
   SetInputParam("input_model", std::move(outputModel));
@@ -175,7 +175,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainedModelDimTest",
                  "[LCCMainTest][BindingTests]")
 {
   arma::mat x;
-  x.load("mnist_first250_training_4s_and_9s.arm");
+  x.load("mnist_first250_training_4s_and_9s.csv");
   arma:: mat t = x;
   t.shed_rows(1, 2);
 
@@ -185,8 +185,8 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainedModelDimTest",
 
   RUN_BINDING();
 
-  LocalCoordinateCoding* outputModel =
-      std::move(params.Get<LocalCoordinateCoding*>("output_model"));
+  LocalCoordinateCoding<>* outputModel =
+      std::move(params.Get<LocalCoordinateCoding<>*>("output_model"));
 
   SetInputParam("input_model", std::move(outputModel));
   SetInputParam("test", std::move(t));

src/mlpack/tests/sparse_coding_test.cpp

@@ -57,7 +57,7 @@ TEMPLATE_TEST_CASE("SparseCodingTestCodingStepLasso", "[SparseCodingTest]",
   uword nAtoms = 25;
 
   arma::mat inX; // The .arm file contains an arma::mat.
-  inX.load("mnist_first250_training_4s_and_9s.arm");
+  inX.load("mnist_first250_training_4s_and_9s.csv");
   MatType X = arma::conv_to<MatType>::from(inX);
   uword nPoints = X.n_cols;
 
@@ -92,7 +92,7 @@ TEMPLATE_TEST_CASE("SparseCodingTestCodingStepElasticNet", "[SparseCodingTest]",
   uword nAtoms = 25;
 
   arma::mat inX; // The .arm file contains an arma::mat.
-  inX.load("mnist_first250_training_4s_and_9s.arm");
+  inX.load("mnist_first250_training_4s_and_9s.csv");
   MatType X = arma::conv_to<MatType>::from(inX);
   uword nPoints = X.n_cols;
 
@@ -129,7 +129,7 @@ TEMPLATE_TEST_CASE("SparseCodingTestDictionaryStep", "[SparseCodingTest]",
   uword nAtoms = 25;
 
   arma::mat inX; // The .arm file contains an arma::mat.
-  inX.load("mnist_first250_training_4s_and_9s.arm");
+  inX.load("mnist_first250_training_4s_and_9s.csv");
   MatType X = arma::conv_to<MatType>::from(inX);
   uword nPoints = X.n_cols;
 
@@ -222,7 +222,7 @@ TEMPLATE_TEST_CASE("SparseCodingTrainReturnObjective", "[SparseCodingTest]",
   uword nAtoms = 25;
 
   arma::mat inX; // The .arm file contains an arma::mat.
-  inX.load("mnist_first250_training_4s_and_9s.arm");
+  inX.load("mnist_first250_training_4s_and_9s.csv");
   MatType X = arma::conv_to<MatType>::from(inX);
   uword nPoints = X.n_cols;