diff --git a/HISTORY.md b/HISTORY.md
index e3ba45d0a7..a85d71443a 100644
--- a/HISTORY.md
+++ b/HISTORY.md
@@ -44,6 +44,11 @@
* Fix divide-by-zero edge case for LARS (#3701).
+ * Templatize `SparseCoding` and `LocalCoordinateCoding` to allow different
+ matrix types (#3709, #3711).
+
+ * Fix handling of unused atoms in `LocalCoordinateCoding` (#3711).
+
* Move minimum required C++ version from C++14 to C++17 (#3704).
### mlpack 4.3.0
diff --git a/doc/index.md b/doc/index.md
index ad139d423b..d733726ec9 100644
--- a/doc/index.md
+++ b/doc/index.md
@@ -120,6 +120,8 @@ Prepare data for machine learning algorithms.
Transform data from one space to another.
+ * [`LocalCoordinateCoding`](user/methods/local_coordinate_coding.md): local
+ coordinate coding with dictionary learning
* [`NMF`](user/methods/nmf.md): non-negative matrix factorization
* [`PCA`](user/methods/pca.md): principal components analysis
* [`SparseCoding`](user/methods/sparse_coding.md): sparse coding with
diff --git a/doc/sidebar.html b/doc/sidebar.html
index 1f5d15fe67..3f4aaeabd8 100644
--- a/doc/sidebar.html
+++ b/doc/sidebar.html
@@ -179,6 +179,11 @@ when the sidebar is built for each page.
+ -
+
+
LocalCoordinateCoding
+
+
-
NMF
diff --git a/doc/user/core.md b/doc/user/core.md
index ee3c0dd1f6..3ed5cd073e 100644
--- a/doc/user/core.md
+++ b/doc/user/core.md
@@ -81,7 +81,7 @@ avoid copies.
- If `offset` is `0`, then the alias is identical: the first element of
`a` is the first element of `mat`. Otherwise, the first element of `a`
is the `offset`'th element of `mat`; elements in `mat` are ordered in
- a [column-major way](../matrices.md#representing-data-in-mlpack).
+ a [column-major way](matrices.md#representing-data-in-mlpack).
- If `strict` is `true`, the size of `a` cannot be changed.
- `mat` and `a` should have the same matrix type (e.g. `arma::mat`,
`arma::fmat`, `arma::sp_mat`).
@@ -93,7 +93,7 @@ avoid copies.
- If `offset` is `0`, then the alias is identical: the first element of
`a` is the first element of `cube`. Otherwise, the first element of `a`
is the `offset`'th element of `cube`; elements in `cube` are ordered in
- a [column-major way](../matrices.md#representing-data-in-mlpack).
+ a [column-major way](matrices.md#representing-data-in-mlpack).
- If `strict` is `true`, the size of `a` cannot be changed.
- `cube` and `a` should have the same cube type (e.g. `arma::cube`,
`arma::fcube`).
diff --git a/doc/user/methods/local_coordinate_coding.md b/doc/user/methods/local_coordinate_coding.md
new file mode 100644
index 0000000000..5d68945416
--- /dev/null
+++ b/doc/user/methods/local_coordinate_coding.md
@@ -0,0 +1,375 @@
+## `LocalCoordinateCoding`
+
+The `LocalCoordinateCoding` class implements local coordinate coding, a
+variation of [sparse coding](sparse_coding.md) with dictionary learning. Local
+coordinate coding is a form of representation learning, and can be used to
+represent each point in a dataset as a linear combination of a few nearby
+*atoms* in the learned dictionary.
+
+#### Simple usage example:
+
+```c++
+// Create a random dataset with 100 points in 40 dimensions, and then a random
+// test dataset with 50 points.
+arma::mat data(40, 100, arma::fill::randn);
+arma::mat testData(40, 50, arma::fill::randn);
+
+// Perform local coordinate coding with 20 atoms and an L1 penalty of 0.1.
+mlpack::LocalCoordinateCoding lcc(20, 0.1); // Step 1: create object.
+double objective = lcc.Train(data); // Step 2: learn dictionary.
+arma::mat codes;
+lcc.Encode(testData, codes); // Step 3: encode new data.
+
+// Print some information about the test encoding.
+std::cout << "Average density of encoded test data: "
+ << 100.0 * arma::mean(arma::sum(codes != 0)) / codes.n_rows << "\%."
+ << std::endl;
+```
+More examples...
+
+#### Quick links:
+
+ * [Constructors](#constructors): create `LocalCoordinateCoding` objects.
+ * [`Train()`](#training): train model (learn dictionary).
+ * [`Encode()`](#encoding): encode points with a trained model.
+ * [Other functionality](#other-functionality) for loading, saving, and
+ inspecting.
+ * [Examples](#simple-examples) of simple usage and links to detailed example
+ projects.
+ * [Template parameters](#advanced-functionality-template-parameters) for
+ advanced functionality: different element types and dictionary initialization
+ strategies.
+
+#### See also:
+
+ * [`SparseCoding`](sparse_coding.md)
+ * [`LARS`](lars.md) (used internally by `LocalCoordinateCoding`)
+ * [mlpack transformations](../../index.md#transformations)
+ * [Sparse dictionary learning on Wikipedia](https://en.wikipedia.org/wiki/Sparse_dictionary_learning)
+ * [Nonlinear learning using local coordinate coding (pdf)](https://proceedings.neurips.cc/paper_files/paper/2009/file/2afe4567e1bf64d32a5527244d104cea-Paper.pdf)
+
+### Constructors
+
+ * `lcc = LocalCoordinateCoding()`
+ * `lcc = LocalCoordinateCoding(atoms=0, lambda=0.0, maxIter=0, tol=0.01)`
+ - Create a `LocalCoordinateCoding` object without learning a dictionary on
+ data.
+ - If `atoms` is set to `0` (the default), it will need to be set to a value
+ greater than `0` before `Train()` is called (`lcc.Atoms() = atoms` can be
+ used for this).
+
+ * `lcc = LocalCoordinateCoding(data, atoms, lambda=0.0, maxIter=0, tol=0.01)`
+ - Create a `LocalCoordinateCoding` object and train the dictionary on the
+ given `data`.
+ - The dictionary will contain `atoms` elements.
+
+ * `lcc = LocalCoordinateCoding(data, atoms, lambda, maxIter, tol, initializer)`
+ - *Advanced constructor*: create a `LocalCoordinateCoding` object that will
+ use a custom dictionary initializer and train on the given `data`.
+ - The dictionary will contain `atoms` elements.
+ - `initializer` will be used to initialize the dictionary; see [Advanced
+ Functionality: Different Dictionary Initialization
+ Strategies](#dictionaryinitializer-different-dictionary-initialization-strategies)
+ for details.
+
+#### Constructor Parameters:
+
+| **name** | **type** | **description** | **default** |
+|----------|----------|-----------------|-------------|
+| `data` | [`arma::mat`](../matrices.md) | [Column-major](../matrices.md#representing-data-in-mlpack) training matrix. | _(N/A)_ |
+| `atoms` | `size_t` | Number of atoms in dictionary. | _(N/A)_ |
+| `lambda` | `double` | L1 regularization penalty. Used in both `Train()` and `Encode()` steps. | `0.0` |
+| `maxIter` | `size_t` | Maximum number of iterations for dictionary learning. `0` means no limit. | `0` |
+| `tol` | `double` | Objective function tolerance for terminating dictionary learning. | `0.01` |
+
+As an alternative to passing `atoms`, `lambda`, `maxIter`, or `tol`, these can
+be set with a standalone method. The following functions can be used before
+calling `Train()`:
+
+ * `lcc.Atoms() = a;` will set the number of atoms to use in the dictionary to
+ `a`. Changing this after calling `Train()` will not make a difference to the
+ dictionary size.
+
+ * `lcc.Lambda() = l;` will set the L1 regularization penalty to `l1`. This can
+ be set after `Train()` to force sparser encodings when `Encode()` is called.
+
+ * `lcc.MaxIterations() = m;` will set the maximum number of iterations for
+ dictionary learning to `m`. `0` means that the algorithm will run until
+ convergence.
+
+ * `lcc.Tolerance() = t;` will set the objective tolerance for convergence of
+ the dictionary learning algorithm to `t`.
+
+***Caveats***:
+
+ * Larger settings of `atoms` (i.e. larger dictionary sizes) will be able to
+ more accurately represent the data, but may take longer to learn.
+
+ * Larger values of `lambda` will cause the model to use sparser encodings for
+ data (e.g. fewer nearby anchor points) when `Train()` and `Encode()` are
+ called, but when `lambda` is too large, the codings may be inaccurate
+ representations of the original points.
+
+
+
+ * If `lambda` is set too large, encodings may be empty (e.g. all zeros).
+
+ * Training is not incremental; a second call to `Train()` will reinitialize the
+ dictionary and restart the learning process.
+
+### Training
+
+If training the dictionary is not done as part of the constructor call, it can
+be done with one of the following versions of the `Train()` member function:
+
+ * `lcc.Train(data)`
+ * `lcc.Train(data, initializer)`
+ - Train the local coordinate coding dictionary on the given `data`.
+ - Optionally, use the given `initializer` to initialize the dictionary (see
+ [`DictionaryInitializer`](#dictionaryinitializer-different-dictionary-initialization-strategies)
+ for more details).
+
+### Encoding
+
+Once a `LocalCoordinateCoding` model has a trained dictionary, the `Encode()`
+member function can be used to encode new data points.
+
+ * `lcc.Encode(data, codes)`
+ - Encode `data` (a [column-major data
+ matrix](../matrices.md#representing-data-in-mlpack)) as a sparse set of
+ local atoms of the dictionary, storing the result in `codes`.
+ - Both `data` and `codes` should be the same matrix type (e.g. `arma::mat`);
+ see [Different Element Types](#mattype-different-element-types) for more
+ details.
+ - `codes` will be set to have `atoms` rows and `data.n_cols` columns.
+ - Column `i` of `codes` corresponds to the coding of the `i`'th column of
+ `data`. Each row represents the weight associated with each atom in the
+ dictionary.
+
+After encoding, the original data can be recovered (approximately) as
+`lcc.Dictionary() * data`.
+
+### Other Functionality
+
+ * A `LocalCoordinateCoding` model can be serialized with
+ [`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
+
+ * `lcc.Dictionary()` will return an `arma::mat&` containing the dictionary
+ matrix. The matrix has `data.n_rows` rows and `atoms` columns; each column
+ corresponds to an atom in the dictionary. Dictionary atoms are regularized
+ to be close to the manifold that data lie on.
+
+ * `double obj = lcc.Objective(data, codes)` computes the local coordinate
+ coding objective function on the given `data` and encodings `codes`. This
+ can be used after `Encode()` to test the quality of the encodings (a smaller
+ objective is better).
+
+### Simple Examples
+
+See also the [simple usage example](#simple-usage-example) for a trivial usage
+of the `LocalCoordinateCoding` class.
+
+---
+
+Train a local coordinate coding model on the cloud dataset and print the
+reconstruction error.
+
+```c++
+// See https://datasets.mlpack.org/cloud.csv.
+arma::mat dataset;
+mlpack::data::Load("cloud.csv", dataset, true);
+
+mlpack::LocalCoordinateCoding lcc;
+lcc.Atoms() = 50;
+lcc.Lambda() = 1e-5;
+lcc.MaxIterations() = 25;
+lcc.Train(dataset);
+
+// Encode the training dataset.
+arma::mat codes;
+lcc.Encode(dataset, codes);
+
+std::cout << "Input matrix size: " << dataset.n_rows << " x " << dataset.n_cols
+ << "." << std::endl;
+std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
+ << "." << std::endl;
+
+// Reconstruct the original matrix.
+arma::mat recon = lcc.Dictionary() * codes;
+double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
+std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;
+```
+
+---
+
+Train a local coordinate coding model on the iris dataset and save the model to
+disk.
+
+```c++
+// See https://datasets.mlpack.org/iris.train.csv.
+arma::mat dataset;
+mlpack::data::Load("iris.train.csv", dataset, true);
+
+// Train the model in the constructor.
+mlpack::LocalCoordinateCoding lcc(dataset,
+ 10 /* atoms */,
+ 0.1 /* L1 penalty */);
+
+// Save the model to disk.
+mlpack::data::Save("lcc.bin", "lcc", lcc);
+```
+
+---
+
+Train a local coordinate coding model on the satellite dataset, trying several
+different regularization parameters and checking the objective value on a
+held-out test dataset.
+
+```c++
+// See https://datasets.mlpack.org/satellite.train.csv.
+arma::mat trainData;
+mlpack::data::Load("satellite.train.csv", trainData, true);
+// See https://datasets.mlpack.org/satellite.test.csv.
+arma::mat testData;
+mlpack::data::Load("satellite.test.csv", testData, true);
+
+for (double lambdaPow = -6; lambdaPow <= -2; lambdaPow += 1)
+{
+ const double lambda = std::pow(10.0, lambdaPow);
+ mlpack::LocalCoordinateCoding lcc(50 /* atoms */);
+ lcc.Lambda() = lambda;
+ lcc.MaxIterations() = 25; // Keep iterations low so this runs relatively fast.
+
+ const double trainObj = lcc.Train(trainData);
+
+ // Compute the objective on the test set.
+ arma::mat codes;
+ lcc.Encode(testData, codes);
+ const double testObj = lcc.Objective(testData, codes);
+
+ std::cout << "Lambda: " << std::setfill(' ') << std::setw(3) << lambda
+ << "; ";
+ std::cout << "training set objective: " << std::setw(6) << trainObj << "; ";
+ std::cout << "test set objective: " << std::setw(6) << testObj << "."
+ << std::endl;
+}
+```
+
+### Advanced Functionality: Template Parameters
+
+The `LocalCoordinateCoding` class has one class template parameter that can be
+used for custom behavior. The full signature of the class is:
+
+```
+LocalCoordinateCoding
+```
+
+In addition, the [constructors](#constructors) and [`Train()`
+functions](#training) have a template parameter `DictionaryInitializer` that can
+be used for custom behavior.
+
+ * `MatType`: the type of the matrix to use (e.g. `arma::mat`, `arma::fmat`,
+ etc.). The given `MatType` must support the Armadillo API and hold a
+ floating-point element type (e.g. `float`, `double`, etc.).
+
+ * `DictionaryInitializer`: the strategy used to initialize the dictionary. By
+ default, `DataDependentRandomInitializer` is used.
+
+#### `MatType`: Different Element Types
+
+`MatType` specifies the type of matrix used for training data and internal
+representation of the dictionary. Any matrix type that implements the Armadillo
+API can be used. The example below trains a local coordinate coding model on
+32-bit floating point data.
+
+```c++
+// See https://datasets.mlpack.org/cloud.csv.
+arma::fmat dataset;
+mlpack::data::Load("cloud.csv", dataset, true);
+
+mlpack::LocalCoordinateCoding lcc;
+lcc.Atoms() = 30;
+lcc.Lambda() = 1e-5;
+lcc.MaxIterations() = 100;
+lcc.Train(dataset);
+
+// Encode the training dataset.
+arma::fmat codes;
+lcc.Encode(dataset, codes);
+
+std::cout << "Input matrix size: " << dataset.n_rows << " x " << dataset.n_cols
+ << "." << std::endl;
+std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
+ << "." << std::endl;
+
+// Reconstruct the original matrix.
+arma::fmat recon = lcc.Dictionary() * codes;
+double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
+std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;
+```
+
+#### `DictionaryInitializer`: Different Dictionary Initialization Strategies
+
+The `DictionaryInitializer` template class specifies the strategy to be used to
+initialize the dictionary when `Train()` is called.
+
+ * The `DataDependentRandomInitalizer` class (the default) uses the average of
+ three random points in the dataset to initialize each atom in the dictionary.
+
+ * The `NothingInitializer` class does not modify the dictionary matrix in any
+ way, and could be used either to set a specific dictionary before training
+ with `sc.Dictionary()`, or to allow incremental training that does not modify
+ the existing dictionary when `Train()` is called a second time.
+
+ * The `RandomInitializer` class initializes the dictionary by sampling norm-1
+ atoms from a normal distribution.
+
+***Note:*** none of the classes above have any members, and as such it is not
+necessary to use the constructor or `Train()` variants that take an initialized
+`initializer` object. That would only be necessary for a custom
+`DictionaryInitializer` class that stored internal members.
+
+---
+
+The example below uses `NothingInitializer` to set a specific initial
+dictionary.
+
+```c++
+// See https://datasets.mlpack.org/satellite.train.csv.
+arma::mat trainData;
+mlpack::data::Load("satellite.train.csv", trainData, true);
+
+const size_t atoms = 25;
+const double lambda = 1e-5;
+const size_t maxIterations = 50;
+
+// Use a uniform random matrix as the initial dictionary.
+arma::mat initialDictionary(trainData.n_rows, atoms, arma::fill::randu);
+
+mlpack::LocalCoordinateCoding lcc(atoms, lambda, maxIterations);
+lcc.Dictionary() = initialDictionary;
+
+const double obj = lcc.Train(trainData);
+std::cout << "Training set objective: " << obj << "." << std::endl;
+```
+
+---
+
+ * An entirely custom class can also be implemented. The class must implement
+ one method, `Initialize()`:
+
+```c++
+// You can use this as a starting point for implementation.
+class CustomDictionaryInitializer
+{
+ public:
+ // Initialize the dictionary to have the given number of atoms, given the
+ // dataset. MatType will be the matrix type used by the local coordinate
+ // coding model (e.g. `arma::mat`, `arma::fmat`, etc.).
+ template
+ void Initialize(const MatType& data,
+ const size_t atoms,
+ MatType& dictionary);
+};
+```
diff --git a/doc/user/methods/sparse_coding.md b/doc/user/methods/sparse_coding.md
index ffdaa5c37b..e21ecfa9ec 100644
--- a/doc/user/methods/sparse_coding.md
+++ b/doc/user/methods/sparse_coding.md
@@ -41,8 +41,7 @@ std::cout << "Average density of encoded test data: "
#### See also:
-
-
+ * [`LocalCoordinateCoding`](local_coordinate_coding.md)
* [`LARS`](lars.md) (used internally by `SparseCoding`)
* [mlpack transformations](../../index.md#transformations)
* [Sparse dictionary learning on Wikipedia](https://en.wikipedia.org/wiki/Sparse_dictionary_learning)
@@ -150,7 +149,8 @@ function can be used to encode new data points.
of `data`. Each row represents the weight associated with each atom in
the dictionary.
-After encoding, the original data can be recovered as `sc.Dictionary() * data`.
+After encoding, the original data can be recovered (approximately) as
+`sc.Dictionary() * data`.
### Other Functionality
@@ -288,8 +288,6 @@ In addition, the [constructors](#constructors) and
[`Train()` functions](#training) have a template parameter
`DictionaryInitializer` that can be used for custom behavior.
-
-
* `MatType`: the type of the matrix to use (e.g. `arma::mat`, `arma::fmat`,
etc.). The given `MatType` must support the Armadillo API and hold a
floating-point element type (e.g. `float`, `double`, etc.).
@@ -297,7 +295,7 @@ In addition, the [constructors](#constructors) and
* `DictionaryInitializer`: the strategy used to initialize the dictionary. By
default, `DataDependentRandomInitializer` is used.
-#### ```MatType```: Different Element Types
+#### `MatType`: Different Element Types
`MatType` specifies the type of matrix used for training data and internal
representation of the dictionary. Any matrix type that implements the Armadillo
@@ -332,10 +330,9 @@ std::cout << "Codes matrix size: " << codes.n_rows << " x " << codes.n_cols
arma::fmat recon = sc.Dictionary() * codes;
double error = std::sqrt(arma::norm(dataset - recon, "fro") / dataset.n_elem);
std::cout << "RMSE of reconstructed matrix: " << error << "." << std::endl;
-
```
-#### ```DictionaryInitializer```: Different Dictionary Initialization Strategies
+#### `DictionaryInitializer`: Different Dictionary Initialization Strategies
The `DictionaryInitializer` template class specifies the strategy to be used to
initialize the dictionary when `Train()` is called.
diff --git a/scripts/build-docs.sh b/scripts/build-docs.sh
index f456167332..a8b20533a1 100755
--- a/scripts/build-docs.sh
+++ b/scripts/build-docs.sh
@@ -176,26 +176,26 @@ EOF
# appends a sidebar list to the output HTML.
create_page_sidebar_section()
{
- input_file="$1";
- output_file="$2";
- dir_name="$3"; # The directory containing the documentation.
- input_file_base=`basename "$input_file" .html.tmp`;
+ sb_input_file="$1";
+ sb_output_file="$2";
+ sb_dir_name="$3"; # The directory containing the documentation.
+ sb_input_file_base=`basename "$sb_input_file" .html.tmp`;
# Extract h2/h3 anchors into a list. For individual method documentation, we
# only extract h3 anchors because those use h2s as their headings. And, for
# core.md, we want to extract both h2 and h3 anchors.
- if [[ "$dir_name" == "user/methods" ]];
+ if [[ "$sb_dir_name" == "user/methods" ]];
then
# The page title on individual methods is encoded as an h2.
- page_title=`grep '\(.*\)<\/h2>/\1/'`;
- grep '
- \2<\/a><\/li>/' > "$output_file.side.tmp";
- elif [[ "$input_file_base" == "core" ]];
+ grep '
- \2<\/a><\/li>/' > "$sb_output_file.side.tmp";
+ elif [[ "$sb_input_file_base" == "core" ]];
then
# The page title on the core class documentation page is encoded as an h1.
- page_title=`grep '
\(.*\)<\/h1>/\1/'`;
@@ -208,8 +208,9 @@ create_page_sidebar_section()
# ...
#
# and then we'll construct the actual sidebar using that list.
- grep '\(.*\)<\/h[23]>/\1\t\2\t\3/' > "$output_file.side.list.tmp";
+ grep '\(.*\)<\/h[23]>/\1\t\2\t\3/' \
+ > "$sb_output_file.side.list.tmp";
in_block=0;
while read line; do
# First, extract the pieces of each line.
@@ -225,62 +226,64 @@ create_page_sidebar_section()
if [ "$in_block" = "1" ];
then
# We have to close the previous block.
- echo "
" >> "$output_file.side.tmp";
+ echo "" >> "$sb_output_file.side.tmp";
fi
# Create the new details block.
- echo "" >> "$output_file.side.tmp";
- echo "" >> "$output_file.side.tmp";
- echo "$anchor_title" >> "$output_file.side.tmp";
- echo "" >> "$output_file.side.tmp";
- echo "
" >> "$output_file.side.tmp";
- echo "\(.*\)<\/h1>/\1/'`;
- grep '
\2<\/a><\/li>/' > "$output_file.side.tmp";
+ grep '\2<\/a><\/li>/' \
+ > "$sb_output_file.side.tmp";
fi
- lines=`cat "$output_file.side.tmp" | wc -l`;
+ lines=`cat "$sb_output_file.side.tmp" | wc -l`;
- echo "" >> "$output_file";
+ echo "" >> "$sb_output_file";
# Make the top of the sidebar.
if [ -n "$page_title" ];
then
- echo "- $page_title [top]" >> "$output_file";
+ echo "
- $page_title [top]" >> "$sb_output_file";
else
- echo "
- [top of page]" >> "$output_file";
+ echo "
- [top of page]" >> "$sb_output_file";
fi
if [[ "$lines" -gt 0 ]];
then
- echo "
" >> "$output_file";
- cat "$output_file.side.tmp" >> "$output_file";
- echo "
" >> "$output_file";
+ echo "" >> "$sb_output_file";
+ cat "$sb_output_file.side.tmp" >> "$sb_output_file";
+ echo "
" >> "$sb_output_file";
fi
- echo " " >> "$output_file";
- echo "
" >> "$output_file";
- echo "" >> "$output_file";
+ echo "
" >> "$sb_output_file";
+ echo "" >> "$sb_output_file";
+ echo "" >> "$sb_output_file";
- rm -f "$output_file.side.tmp";
+ rm -f "$sb_output_file.side.tmp";
}
rm -rf "$output_dir";
diff --git a/src/mlpack/methods/local_coordinate_coding/lcc.hpp b/src/mlpack/methods/local_coordinate_coding/lcc.hpp
index 4ff9904595..11b96d53b3 100644
--- a/src/mlpack/methods/local_coordinate_coding/lcc.hpp
+++ b/src/mlpack/methods/local_coordinate_coding/lcc.hpp
@@ -75,9 +75,13 @@ namespace mlpack {
* }
* @endcode
*/
+template
class LocalCoordinateCoding
{
public:
+ typedef typename GetColType::type ColType;
+ typedef typename GetRowType::type RowType;
+
/**
* Set the parameters to LocalCoordinateCoding, and train the dictionary.
* This constructor will also initialize the dictionary using the given
@@ -85,7 +89,7 @@ class LocalCoordinateCoding
*
* If you want to initialize the dictionary to a custom matrix, consider
* either writing your own DictionaryInitializer class (with void
- * Initialize(const arma::mat& data, arma::mat& dictionary) function), or call
+ * Initialize(const MatType& data, MatType& dictionary) function), or call
* the constructor that does not take a data matrix, then call Dictionary() to
* set the dictionary matrix to a matrix of your choosing, and then call
* Train() with NothingInitializer (i.e. Train(data)).
@@ -99,7 +103,7 @@ class LocalCoordinateCoding
* @param initializer Intializer to use.
*/
template
- LocalCoordinateCoding(const arma::mat& data,
+ LocalCoordinateCoding(const MatType& data,
const size_t atoms,
const double lambda,
const size_t maxIterations = 0,
@@ -132,7 +136,7 @@ class LocalCoordinateCoding
* @return The final objective value.
*/
template
- double Train(const arma::mat& data,
+ double Train(const MatType& data,
const DictionaryInitializer& initializer =
DictionaryInitializer());
@@ -142,7 +146,7 @@ class LocalCoordinateCoding
* @param data Matrix containing points to encode.
* @param codes Output matrix to store codes in.
*/
- void Encode(const arma::mat& data, arma::mat& codes);
+ void Encode(const MatType& data, MatType& codes);
/**
* Learn dictionary by solving linear system.
@@ -153,10 +157,19 @@ class LocalCoordinateCoding
* the coding matrix Z that are non-zero (the adjacency matrix for the
* bipartite graph of points and atoms)
*/
- void OptimizeDictionary(const arma::mat& data,
- const arma::mat& codes,
+ void OptimizeDictionary(const MatType& data,
+ const MatType& codes,
const arma::uvec& adjacencies);
+ /**
+ * Compute objective function given the list of adjacencies.
+ *
+ * @param data Matrix containing points to encode.
+ * @param codes Output matrix to store codes in.
+ */
+ double Objective(const MatType& data,
+ const MatType& codes) const;
+
/**
* Compute objective function given the list of adjacencies.
*
@@ -166,8 +179,8 @@ class LocalCoordinateCoding
* the coding matrix Z that are non-zero (the adjacency matrix for the
* bipartite graph of points and atoms)
*/
- double Objective(const arma::mat& data,
- const arma::mat& codes,
+ double Objective(const MatType& data,
+ const MatType& codes,
const arma::uvec& adjacencies) const;
//! Get the number of atoms.
@@ -176,9 +189,9 @@ class LocalCoordinateCoding
size_t& Atoms() { return atoms; }
//! Accessor for dictionary.
- const arma::mat& Dictionary() const { return dictionary; }
+ const MatType& Dictionary() const { return dictionary; }
//! Mutator for dictionary.
- arma::mat& Dictionary() { return dictionary; }
+ MatType& Dictionary() { return dictionary; }
//! Get the L1 regularization parameter.
double Lambda() const { return lambda; }
@@ -204,7 +217,7 @@ class LocalCoordinateCoding
size_t atoms;
//! Dictionary (columns are atoms).
- arma::mat dictionary;
+ MatType dictionary;
//! l1 regularization term.
double lambda;
@@ -217,6 +230,9 @@ class LocalCoordinateCoding
} // namespace mlpack
+CEREAL_TEMPLATE_CLASS_VERSION((typename MatType),
+ (mlpack::LocalCoordinateCoding), (1));
+
// Include implementation.
#include "lcc_impl.hpp"
diff --git a/src/mlpack/methods/local_coordinate_coding/lcc_impl.hpp b/src/mlpack/methods/local_coordinate_coding/lcc_impl.hpp
index bbaf7817ba..2894d015f6 100644
--- a/src/mlpack/methods/local_coordinate_coding/lcc_impl.hpp
+++ b/src/mlpack/methods/local_coordinate_coding/lcc_impl.hpp
@@ -17,9 +17,10 @@
namespace mlpack {
+template
template
-LocalCoordinateCoding::LocalCoordinateCoding(
- const arma::mat& data,
+LocalCoordinateCoding::LocalCoordinateCoding(
+ const MatType& data,
const size_t atoms,
const double lambda,
const size_t maxIterations,
@@ -34,7 +35,8 @@ LocalCoordinateCoding::LocalCoordinateCoding(
Train(data, initializer);
}
-inline LocalCoordinateCoding::LocalCoordinateCoding(
+template
+inline LocalCoordinateCoding::LocalCoordinateCoding(
const size_t atoms,
const double lambda,
const size_t maxIterations,
@@ -47,9 +49,10 @@ inline LocalCoordinateCoding::LocalCoordinateCoding(
// Nothing to do.
}
+template
template
-double LocalCoordinateCoding::Train(
- const arma::mat& data,
+double LocalCoordinateCoding::Train(
+ const MatType& data,
const DictionaryInitializer& initializer)
{
// Initialize the dictionary.
@@ -61,7 +64,7 @@ double LocalCoordinateCoding::Train(
// loop.
Log::Info << "Initial Coding Step." << std::endl;
- arma::mat codes;
+ MatType codes;
Encode(data, codes);
arma::uvec adjacencies = find(codes);
@@ -115,15 +118,16 @@ double LocalCoordinateCoding::Train(
return lastObjVal;
}
-inline void LocalCoordinateCoding::Encode(const arma::mat& data,
- arma::mat& codes)
+template
+inline void LocalCoordinateCoding::Encode(const MatType& data,
+ MatType& codes)
{
- arma::mat invSqDists = 1.0 / (repmat(trans(sum(square(dictionary))), 1,
+ MatType invSqDists = 1.0 / (repmat(trans(sum(square(dictionary))), 1,
data.n_cols) + repmat(sum(square(data)), atoms, 1) - 2 * trans(dictionary)
* data);
- arma::mat dictGram = trans(dictionary) * dictionary;
- arma::mat dictGramTD(dictGram.n_rows, dictGram.n_cols);
+ MatType dictGram = trans(dictionary) * dictionary;
+ MatType dictGramTD(dictGram.n_rows, dictGram.n_cols);
codes.set_size(atoms, data.n_cols);
for (size_t i = 0; i < data.n_cols; ++i)
@@ -134,29 +138,31 @@ inline void LocalCoordinateCoding::Encode(const arma::mat& data,
Log::Debug << "Optimization at point " << i << "." << std::endl;
}
- arma::vec invW = invSqDists.unsafe_col(i);
- arma::mat dictPrime = dictionary * diagmat(invW);
+ ColType invW = invSqDists.unsafe_col(i);
+ MatType dictPrime = dictionary * diagmat(invW);
- arma::mat dictGramTD = diagmat(invW) * dictGram * diagmat(invW);
+ MatType dictGramTD = diagmat(invW) * dictGram * diagmat(invW);
bool useCholesky = false;
// Normalization and fitting and intercept are disabled.
- LARS<> lars(useCholesky, 0.5 * lambda, 0, 1e-16 /* default tolerance */,
- false, false);
+ const double tol = std::is_same::value ?
+ 1e-8 : 1e-16;
+ LARS lars(useCholesky, 0.5 * lambda, 0, tol, false, false);
// Run LARS for this point, by making an alias of the point and passing
// that.
- arma::vec beta = codes.unsafe_col(i);
- arma::rowvec responses = data.unsafe_col(i).t();
+ ColType beta = codes.unsafe_col(i);
+ RowType responses = data.unsafe_col(i).t();
lars.Train(dictPrime, responses, false, useCholesky, dictGramTD);
beta = lars.Beta();
beta %= invW; // Remember, beta is an alias of codes.col(i).
}
}
-inline void LocalCoordinateCoding::OptimizeDictionary(
- const arma::mat& data,
- const arma::mat& codes,
+template
+inline void LocalCoordinateCoding::OptimizeDictionary(
+ const MatType& data,
+ const MatType& codes,
const arma::uvec& adjacencies)
{
// Count number of atomic neighbors for each point x^i.
@@ -184,7 +190,8 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
// Build dataPrime := [X x^1 ... x^1 ... x^n ... x^n]
// where each x^i is repeated for the number of neighbors x^i has.
- arma::mat dataPrime = zeros(data.n_rows, data.n_cols + adjacencies.n_elem);
+ MatType dataPrime = zeros(data.n_rows,
+ data.n_cols + adjacencies.n_elem);
dataPrime(arma::span::all, arma::span(0, data.n_cols - 1)) = data;
@@ -209,9 +216,9 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
const size_t nInactiveAtoms = atoms - nActiveAtoms;
// Efficient construction of codes restricted to active atoms.
- arma::mat codesPrime = zeros(nActiveAtoms, data.n_cols +
+ MatType codesPrime = zeros(nActiveAtoms, data.n_cols +
adjacencies.n_elem);
- arma::vec wSquared = ones(data.n_cols + adjacencies.n_elem, 1);
+ ColType wSquared = ones(data.n_cols + adjacencies.n_elem, 1);
if (nInactiveAtoms > 0)
{
@@ -219,13 +226,13 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
<< " inactive atoms. They will be re-initialized randomly.\n";
// Create matrix holding only active codes.
- arma::mat activeCodes = codes.rows(arma::uvec(activeAtoms));
+ MatType activeCodes = codes.rows(arma::uvec(activeAtoms));
// Create reverse atom lookup for active atoms.
arma::uvec atomReverseLookup(atoms);
for (size_t i = 0; i < activeAtoms.size(); ++i)
{
- atomReverseLookup[i] = activeAtoms[i];
+ atomReverseLookup[activeAtoms[i]] = i;
}
codesPrime(arma::span::all, arma::span(0, data.n_cols - 1)) = activeCodes;
@@ -259,15 +266,18 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
}
}
- wSquared.subvec(data.n_cols, wSquared.n_elem - 1) = lambda *
- abs(wSquared.subvec(data.n_cols, wSquared.n_elem - 1));
+ if (adjacencies.n_elem > 0)
+ {
+ wSquared.subvec(data.n_cols, wSquared.n_elem - 1) = lambda *
+ abs(wSquared.subvec(data.n_cols, wSquared.n_elem - 1));
+ }
// Solve system.
if (nInactiveAtoms == 0)
{
// No inactive atoms. We can solve directly.
- arma::mat A = codesPrime * diagmat(wSquared) * trans(codesPrime);
- arma::mat B = codesPrime * diagmat(wSquared) * trans(dataPrime);
+ MatType A = codesPrime * diagmat(wSquared) * trans(codesPrime);
+ MatType B = codesPrime * diagmat(wSquared) * trans(dataPrime);
dictionary = trans(solve(A, B));
/*
@@ -279,7 +289,7 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
{
// Inactive atoms must be reinitialized randomly, so we cannot solve
// directly for the entire dictionary estimate.
- arma::mat dictionaryActive =
+ MatType dictionaryActive =
trans(solve(codesPrime * diagmat(wSquared) * trans(codesPrime),
codesPrime * diagmat(wSquared) * trans(dataPrime)));
@@ -310,9 +320,19 @@ inline void LocalCoordinateCoding::OptimizeDictionary(
}
}
-inline double LocalCoordinateCoding::Objective(
- const arma::mat& data,
- const arma::mat& codes,
+template
+inline double LocalCoordinateCoding::Objective(
+ const MatType& data,
+ const MatType& codes) const
+{
+ // Compute adjacencies and pass off to other overload.
+ return Objective(data, codes, find(codes));
+}
+
+template
+inline double LocalCoordinateCoding::Objective(
+ const MatType& data,
+ const MatType& codes,
const arma::uvec& adjacencies) const
{
double weightedL1NormZ = 0;
@@ -331,11 +351,26 @@ inline double LocalCoordinateCoding::Objective(
return std::pow(froNormResidual, 2.0) + lambda * weightedL1NormZ;
}
+template
template
-void LocalCoordinateCoding::serialize(Archive& ar,
- const uint32_t /* version */)
+void LocalCoordinateCoding::serialize(Archive& ar,
+ const uint32_t version)
{
ar(CEREAL_NVP(atoms));
+
+ if (cereal::is_loading() && version == 0)
+ {
+ // Older versions of LocalCoordinateCoding always stored dictionary as an
+ // arma::mat.
+ arma::mat dictionaryTmp;
+ ar(cereal::make_nvp("dictionary", dictionaryTmp));
+ dictionary = ConvTo::From(dictionaryTmp);
+ }
+ else
+ {
+ ar(CEREAL_NVP(dictionary));
+ }
+
ar(CEREAL_NVP(dictionary));
ar(CEREAL_NVP(lambda));
ar(CEREAL_NVP(maxIterations));
diff --git a/src/mlpack/methods/local_coordinate_coding/local_coordinate_coding_main.cpp b/src/mlpack/methods/local_coordinate_coding/local_coordinate_coding_main.cpp
index 7a028867cf..50418d9d1f 100644
--- a/src/mlpack/methods/local_coordinate_coding/local_coordinate_coding_main.cpp
+++ b/src/mlpack/methods/local_coordinate_coding/local_coordinate_coding_main.cpp
@@ -90,8 +90,7 @@ BINDING_SEE_ALSO("Nonlinear learning using local coordinate coding (pdf)",
"https://papers.nips.cc/paper/3875-nonlinear-learning-using-local-"
"coordinate-coding.pdf");
BINDING_SEE_ALSO("LocalCoordinateCoding C++ class documentation",
- "@src/mlpack/methods/local_coordinate_coding/local_coordinate_coding."
- "hpp");
+ "@doc/user/methods/local_coordinate_coding.md");
// Training parameters.
PARAM_MATRIX_IN("training", "Matrix of training data (X).", "t");
@@ -106,9 +105,9 @@ PARAM_FLAG("normalize", "If set, the input data matrix will be normalized "
PARAM_DOUBLE_IN("tolerance", "Tolerance for objective function.", "o", 0.01);
// Load/save a model.
-PARAM_MODEL_IN(LocalCoordinateCoding, "input_model", "Input LCC model.", "m");
-PARAM_MODEL_OUT(LocalCoordinateCoding, "output_model", "Output for trained LCC "
- "model.", "M");
+PARAM_MODEL_IN(LocalCoordinateCoding<>, "input_model", "Input LCC model.", "m");
+PARAM_MODEL_OUT(LocalCoordinateCoding<>, "output_model",
+ "Output for trained LCC model.", "M");
// Test on another dataset.
PARAM_MATRIX_IN("test", "Test points to encode.", "T");
@@ -143,9 +142,9 @@ void BINDING_FUNCTION(util::Params& params, util::Timers& timers)
ReportIgnoredParam(params, {{ "training", false }}, "tolerance");
// Do we have an existing model?
- LocalCoordinateCoding* lcc = NULL;
+ LocalCoordinateCoding<>* lcc = NULL;
if (params.Has("input_model"))
- lcc = params.Get("input_model");
+ lcc = params.Get*>("input_model");
if (params.Has("training"))
{
@@ -171,7 +170,7 @@ void BINDING_FUNCTION(util::Params& params, util::Timers& timers)
[](double x) { return x > 0; }, 1,
"Tolerance should be a positive real number");
- lcc = new LocalCoordinateCoding(0, 0.0);
+ lcc = new LocalCoordinateCoding<>(0, 0.0);
lcc->Lambda() = params.Get("lambda");
lcc->Atoms() = (size_t) params.Get("atoms");
@@ -257,5 +256,5 @@ void BINDING_FUNCTION(util::Params& params, util::Timers& timers)
// Save the dictionary and the model.
params.Get("dictionary") = lcc->Dictionary();
- params.Get("output_model") = lcc;
+ params.Get*>("output_model") = lcc;
}
diff --git a/src/mlpack/methods/sparse_coding/sparse_coding.hpp b/src/mlpack/methods/sparse_coding/sparse_coding.hpp
index 7beb09e258..ba22c17798 100644
--- a/src/mlpack/methods/sparse_coding/sparse_coding.hpp
+++ b/src/mlpack/methods/sparse_coding/sparse_coding.hpp
@@ -108,7 +108,7 @@ namespace mlpack {
* the Encode() function.
*
* @tparam DictionaryInitializationPolicy The class to use to initialize the
- * dictionary; must have 'void Initialize(const arma::mat& data, arma::mat&
+ * dictionary; must have 'void Initialize(const MatType& data, MatType&
* dictionary)' function.
*/
template
@@ -127,7 +127,7 @@ class SparseCoding
*
* If you want to initialize the dictionary to a custom matrix, consider
* either writing your own DictionaryInitializer class (with void
- * Initialize(const arma::mat& data, arma::mat& dictionary) function), or call
+ * Initialize(const MatType& data, MatType& dictionary) function), or call
* the constructor that does not take a data matrix, then call Dictionary() to
* set the dictionary matrix to a matrix of your choosing, and then call
* Train() with NothingInitializer (i.e. Train(data)).
diff --git a/src/mlpack/tests/ann/convolutional_network_test.cpp b/src/mlpack/tests/ann/convolutional_network_test.cpp
index 313cee7c17..209ac5b131 100644
--- a/src/mlpack/tests/ann/convolutional_network_test.cpp
+++ b/src/mlpack/tests/ann/convolutional_network_test.cpp
@@ -71,7 +71,7 @@ void CheckMoveFunction(ModelType* network1,
TEST_CASE("PaddingTest", "[ConvolutionalNetworktest]")
{
arma::mat X;
- X.load("mnist_first250_training_4s_and_9s.arm");
+ X.load("mnist_first250_training_4s_and_9s.csv");
// Create the network.
FFN model;
@@ -148,7 +148,7 @@ TEST_CASE("MaxPoolingTest", "[ConvolutionalNetworkTest]")
TEST_CASE("VanillaNetworkTest", "[ConvolutionalNetworkTest]")
{
arma::mat X;
- X.load("mnist_first250_training_4s_and_9s.arm");
+ X.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
arma::uword nPoints = X.n_cols;
@@ -264,7 +264,7 @@ TEST_CASE("VanillaNetworkBatchSizeTest", "[ConvolutionalNetworkTest]")
model.InputDimensions() = std::vector({ 28, 28 });
arma::mat X;
- X.load("mnist_first250_training_4s_and_9s.arm");
+ X.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
arma::uword nPoints = X.n_cols;
@@ -347,7 +347,7 @@ TEST_CASE("VanillaNetworkBatchSizeTest", "[ConvolutionalNetworkTest]")
TEST_CASE("CheckCopyVanillaNetworkTest", "[ConvolutionalNetworkTest]")
{
arma::mat X;
- X.load("mnist_first250_training_4s_and_9s.arm");
+ X.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
arma::uword nPoints = X.n_cols;
diff --git a/src/mlpack/tests/ann/feedforward_network_test.cpp b/src/mlpack/tests/ann/feedforward_network_test.cpp
index 13007e71db..4f349b2bfa 100644
--- a/src/mlpack/tests/ann/feedforward_network_test.cpp
+++ b/src/mlpack/tests/ann/feedforward_network_test.cpp
@@ -400,7 +400,7 @@ TEST_CASE("FFVanillaNetworkTest", "[FeedForwardNetworkTest]")
TestNetwork<>(model, trainData, trainLabels, testData, testLabels, 10, 0.1);
arma::mat dataset;
- dataset.load("mnist_first250_training_4s_and_9s.arm");
+ dataset.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -421,7 +421,7 @@ TEST_CASE("FFVanillaNetworkTest", "[FeedForwardNetworkTest]")
TEST_CASE("ForwardBackwardTest", "[FeedForwardNetworkTest]")
{
arma::mat dataset;
- dataset.load("mnist_first250_training_4s_and_9s.arm");
+ dataset.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -548,7 +548,7 @@ TEST_CASE("DropoutNetworkTest", "[FeedForwardNetworkTest]")
// network must be significant better than 92%.
TestNetwork<>(model, trainData, trainLabels, testData, testLabels, 10, 0.1);
arma::mat dataset;
- dataset.load("mnist_first250_training_4s_and_9s.arm");
+ dataset.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -627,7 +627,7 @@ TEST_CASE("DropConnectNetworkTest", "[FeedForwardNetworkTest]")
TestNetwork(model, trainData, trainLabels, testData, testLabels, 10, 0.1);
arma::mat dataset;
- dataset.load("mnist_first250_training_4s_and_9s.arm");
+ dataset.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
for (size_t i = 0; i < dataset.n_cols; ++i)
@@ -952,7 +952,7 @@ TEST_CASE("RBFNetworkTest", "[FeedForwardNetworkTest]")
TestNetwork<>(model, trainData, trainLabels1, testData, testLabels, 10, 0.1);
arma::mat dataset;
- dataset.load("mnist_first250_training_4s_and_9s.arm");
+ dataset.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
for (size_t i = 0; i < dataset.n_cols; ++i)
diff --git a/src/mlpack/tests/ann/not_adapted/dcgan_test.cpp b/src/mlpack/tests/ann/not_adapted/dcgan_test.cpp
index a38ebbecb3..4ed46ec2d9 100644
--- a/src/mlpack/tests/ann/not_adapted/dcgan_test.cpp
+++ b/src/mlpack/tests/ann/not_adapted/dcgan_test.cpp
@@ -53,7 +53,7 @@ TEST_CASE("DCGANMNISTTest", "[DCGANNetworkTest]")
<< " shuffle = " << shuffle << std::endl;
arma::mat trainData;
- trainData.load("mnist_first250_training_4s_and_9s.arm");
+ trainData.load("mnist_first250_training_4s_and_9s.csv");
Log::Info << arma::size(trainData) << std::endl;
trainData = trainData.cols(0, datasetMaxCols - 1);
@@ -212,7 +212,7 @@ TEST_CASE("DCGANMNISTTest", "[DCGANNetworkTest]")
<< " shuffle = " << shuffle << std::endl;
arma::mat trainData;
- trainData.load("mnist_first250_training_4s_and_9s.arm");
+ trainData.load("mnist_first250_training_4s_and_9s.csv");
Log::Info << arma::size(trainData) << std::endl;
// trainData = trainData.cols(0, datasetMaxCols - 1);
diff --git a/src/mlpack/tests/ann/not_adapted/feedforward_network_test.cpp b/src/mlpack/tests/ann/not_adapted/feedforward_network_test.cpp
index 014f9b4579..f8ea2f199b 100644
--- a/src/mlpack/tests/ann/not_adapted/feedforward_network_test.cpp
+++ b/src/mlpack/tests/ann/not_adapted/feedforward_network_test.cpp
@@ -134,7 +134,7 @@ TEST_CASE("CheckCopyMovingReparametrizationNetworkTest",
TEST_CASE("HighwayNetworkTest", "[FeedForwardNetworkTest]")
{
arma::mat dataset;
- dataset.load("mnist_first250_training_4s_and_9s.arm");
+ dataset.load("mnist_first250_training_4s_and_9s.csv");
// Normalize each point since these are images.
for (size_t i = 0; i < dataset.n_cols; ++i)
diff --git a/src/mlpack/tests/ann/not_adapted/gan_test.cpp b/src/mlpack/tests/ann/not_adapted/gan_test.cpp
index 194ac4759a..6ba7447d23 100644
--- a/src/mlpack/tests/ann/not_adapted/gan_test.cpp
+++ b/src/mlpack/tests/ann/not_adapted/gan_test.cpp
@@ -156,7 +156,7 @@ TEST_CASE("GANMNISTTest", "[GANNetworkTest]")
<< " shuffle = " << shuffle << std::endl;
arma::mat trainData;
- trainData.load("mnist_first250_training_4s_and_9s.arm");
+ trainData.load("mnist_first250_training_4s_and_9s.csv");
Log::Info << arma::size(trainData) << std::endl;
trainData = trainData.cols(0, datasetMaxCols - 1);
diff --git a/src/mlpack/tests/ann/not_adapted/wgan_test.cpp b/src/mlpack/tests/ann/not_adapted/wgan_test.cpp
index 9772181216..8aacc73ac3 100644
--- a/src/mlpack/tests/ann/not_adapted/wgan_test.cpp
+++ b/src/mlpack/tests/ann/not_adapted/wgan_test.cpp
@@ -60,7 +60,7 @@ TEST_CASE("WGANMNISTTest", "[WGANNetworkTest]")
<< " shuffle = " << shuffle << std::endl;
arma::mat trainData;
- trainData.load("mnist_first250_training_4s_and_9s.arm");
+ trainData.load("mnist_first250_training_4s_and_9s.csv");
Log::Info << arma::size(trainData) << std::endl;
trainData = trainData.cols(0, datasetMaxCols - 1);
@@ -222,7 +222,7 @@ TEST_CASE("WGANGPMNISTTest", "[WGANNetworkTest]")
<< " shuffle = " << shuffle << std::endl;
arma::mat trainData;
- trainData.load("mnist_first250_training_4s_and_9s.arm");
+ trainData.load("mnist_first250_training_4s_and_9s.csv");
Log::Info << arma::size(trainData) << std::endl;
trainData = trainData.cols(0, datasetMaxCols - 1);
diff --git a/src/mlpack/tests/data/mnist_first250_training_4s_and_9s.tar.bz2 b/src/mlpack/tests/data/mnist_first250_training_4s_and_9s.tar.bz2
index 63a9e279a6..4eb4d3ff1f 100644
Binary files a/src/mlpack/tests/data/mnist_first250_training_4s_and_9s.tar.bz2 and b/src/mlpack/tests/data/mnist_first250_training_4s_and_9s.tar.bz2 differ
diff --git a/src/mlpack/tests/local_coordinate_coding_test.cpp b/src/mlpack/tests/local_coordinate_coding_test.cpp
index 89e960f65f..6809a11bf8 100644
--- a/src/mlpack/tests/local_coordinate_coding_test.cpp
+++ b/src/mlpack/tests/local_coordinate_coding_test.cpp
@@ -18,7 +18,10 @@
using namespace arma;
using namespace mlpack;
-void VerifyCorrectness(const vec& beta, const vec& errCorr, double lambda)
+template
+void VerifyCorrectness(const MatType& beta,
+ const VecType& errCorr,
+ const double lambda)
{
const double tol = 0.1;
size_t nDims = beta.n_elem;
@@ -43,15 +46,18 @@ void VerifyCorrectness(const vec& beta, const vec& errCorr, double lambda)
}
}
-
-TEST_CASE("LocalCoordinateCodingTestCodingStep",
- "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingTestCodingStep",
+ "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
{
+ typedef TestType MatType;
+ typedef arma::Col VecType;
+
double lambda1 = 0.1;
uword nAtoms = 10;
- mat X;
- X.load("mnist_first250_training_4s_and_9s.arm");
+ mat inX; // The .arm file is saved as an arma::mat.
+ inX.load("mnist_first250_training_4s_and_9s.csv");
+ MatType X = arma::conv_to::from(inX);
uword nPoints = X.n_cols;
// normalize each point since these are images
@@ -60,37 +66,40 @@ TEST_CASE("LocalCoordinateCodingTestCodingStep",
X.col(i) /= norm(X.col(i), 2);
}
- mat Z;
- LocalCoordinateCoding lcc(X, nAtoms, lambda1, 10);
+ MatType Z;
+ LocalCoordinateCoding lcc(X, nAtoms, lambda1, 10);
lcc.Encode(X, Z);
- mat D = lcc.Dictionary();
+ MatType D = lcc.Dictionary();
for (uword i = 0; i < nPoints; ++i)
{
- vec sqDists = vec(nAtoms);
+ VecType sqDists(nAtoms);
for (uword j = 0; j < nAtoms; ++j)
{
sqDists[j] = arma::norm(D.col(j) - X.col(i));
}
- mat Dprime = D * diagmat(1.0 / sqDists);
- mat zPrime = Z.unsafe_col(i) % sqDists;
+ MatType Dprime = D * diagmat(1.0 / sqDists);
+ MatType zPrime = Z.unsafe_col(i) % sqDists;
- vec errCorr = trans(Dprime) * (Dprime * zPrime - X.unsafe_col(i));
+ VecType errCorr = trans(Dprime) * (Dprime * zPrime - X.unsafe_col(i));
VerifyCorrectness(zPrime, errCorr, 0.5 * lambda1);
}
}
-TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
- "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
+ "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
{
+ typedef TestType MatType;
+
const double tol = 0.1;
double lambda = 0.1;
uword nAtoms = 10;
- mat X;
- X.load("mnist_first250_training_4s_and_9s.arm");
+ mat inX; // File is saved as an arma::mat.
+ inX.load("mnist_first250_training_4s_and_9s.csv");
+ MatType X = arma::conv_to::from(inX);
uword nPoints = X.n_cols;
// normalize each point since these are images
@@ -99,15 +108,15 @@ TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
X.col(i) /= norm(X.col(i), 2);
}
- mat Z;
- LocalCoordinateCoding lcc(X, nAtoms, lambda, 10);
+ MatType Z;
+ LocalCoordinateCoding lcc(X, nAtoms, lambda, 10);
lcc.Encode(X, Z);
uvec adjacencies = find(Z);
lcc.OptimizeDictionary(X, Z, adjacencies);
- mat D = lcc.Dictionary();
+ MatType D = lcc.Dictionary();
- mat grad = zeros(D.n_rows, D.n_cols);
+ MatType grad = zeros(D.n_rows, D.n_cols);
for (uword i = 0; i < nPoints; ++i)
{
grad += (D - repmat(X.unsafe_col(i), 1, nAtoms)) *
@@ -118,26 +127,30 @@ TEST_CASE("LocalCoordinateCodingTestDictionaryStep",
REQUIRE(norm(grad, "fro") == Approx(0.0).margin(tol));
}
-TEST_CASE("LocalCoordinateCodingSerializationTest",
- "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingSerializationTest",
+ "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
{
- mat X = randu(100, 100);
+ typedef TestType MatType;
+
+ MatType X = randu(100, 100);
size_t nAtoms = 10;
- LocalCoordinateCoding lcc(nAtoms, 0.05, 2 /* don't care about quality */);
+ LocalCoordinateCoding lcc(nAtoms, 0.05,
+ 2 /* don't care about quality */);
lcc.Train(X);
- mat Y = randu(100, 200);
- mat codes;
+ MatType Y = randu(100, 200);
+ MatType codes;
lcc.Encode(Y, codes);
- LocalCoordinateCoding lccXml(50, 0.1), lccJson(12, 0.0), lccBinary(0, 0.0);
+ LocalCoordinateCoding lccXml(50, 0.1), lccJson(12, 0.0),
+ lccBinary(0, 0.0);
SerializeObjectAll(lcc, lccXml, lccJson, lccBinary);
CheckMatrices(lcc.Dictionary(), lccXml.Dictionary(), lccJson.Dictionary(),
lccBinary.Dictionary());
- mat xmlCodes, jsonCodes, binaryCodes;
+ MatType xmlCodes, jsonCodes, binaryCodes;
lccXml.Encode(Y, xmlCodes);
lccJson.Encode(Y, jsonCodes);
lccBinary.Encode(Y, binaryCodes);
@@ -167,14 +180,17 @@ TEST_CASE("LocalCoordinateCodingSerializationTest",
* Test that LocalCoordinateCoding::Train() returns finite final objective
* value.
*/
-TEST_CASE("LocalCoordinateCodingTrainReturnObjective",
- "[LocalCoordinateCodingTest]")
+TEMPLATE_TEST_CASE("LocalCoordinateCodingTrainReturnObjective",
+ "[LocalCoordinateCodingTest]", arma::mat, arma::fmat)
{
+ typedef TestType MatType;
+
double lambda1 = 0.1;
uword nAtoms = 10;
- mat X;
- X.load("mnist_first250_training_4s_and_9s.arm");
+ mat inX; // File is saved as arma::mat.
+ inX.load("mnist_first250_training_4s_and_9s.csv");
+ MatType X = arma::conv_to::from(inX);
uword nPoints = X.n_cols;
// Normalize each point since these are images.
@@ -183,7 +199,7 @@ TEST_CASE("LocalCoordinateCodingTrainReturnObjective",
X.col(i) /= norm(X.col(i), 2);
}
- LocalCoordinateCoding lcc(nAtoms, lambda1, 10);
+ LocalCoordinateCoding lcc(nAtoms, lambda1, 10);
double objVal = lcc.Train(X);
REQUIRE(std::isfinite(objVal) == true);
diff --git a/src/mlpack/tests/main_tests/local_coordinate_coding_test.cpp b/src/mlpack/tests/main_tests/local_coordinate_coding_test.cpp
index 74c271282d..b8cfbc02f0 100644
--- a/src/mlpack/tests/main_tests/local_coordinate_coding_test.cpp
+++ b/src/mlpack/tests/main_tests/local_coordinate_coding_test.cpp
@@ -32,7 +32,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCDimensionsTest",
"[LCCMainTest][BindingTests]")
{
arma::mat x;
- x.load("mnist_first250_training_4s_and_9s.arm");
+ x.load("mnist_first250_training_4s_and_9s.csv");
int rows = x.n_rows, cols = x.n_cols;
arma::mat t = x;
int atoms = 10;
@@ -58,7 +58,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCOutputModelTest",
"[LCCMainTest][BindingTests]")
{
arma::mat x;
- x.load("mnist_first250_training_4s_and_9s.arm");
+ x.load("mnist_first250_training_4s_and_9s.csv");
arma::mat t = x;
SetInputParam("training", std::move(x));
@@ -71,8 +71,8 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCOutputModelTest",
// Get the encoded output and dictionary after training.
arma::mat initCodes = std::move(params.Get("codes"));
arma::mat initDict = std::move(params.Get("dictionary"));
- LocalCoordinateCoding* outputModel =
- std::move(params.Get("output_model"));
+ LocalCoordinateCoding<>* outputModel =
+ std::move(params.Get*>("output_model"));
ResetSettings();
@@ -129,7 +129,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainAndTestDataDimTest",
"[LCCMainTest][BindingTests]")
{
arma::mat x;
- x.load("mnist_first250_training_4s_and_9s.arm");
+ x.load("mnist_first250_training_4s_and_9s.csv");
arma::mat t = x;
t.shed_rows(1, 2);
@@ -150,7 +150,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainAndInputModelTest",
"[LCCMainTest][BindingTests]")
{
arma::mat x;
- x.load("mnist_first250_training_4s_and_9s.arm");
+ x.load("mnist_first250_training_4s_and_9s.csv");
SetInputParam("training", x);
SetInputParam("atoms", (int) 10);
@@ -158,8 +158,8 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainAndInputModelTest",
RUN_BINDING();
- LocalCoordinateCoding* outputModel =
- std::move(params.Get("output_model"));
+ LocalCoordinateCoding<>* outputModel =
+ std::move(params.Get*>("output_model"));
// No need to input training data again.
SetInputParam("input_model", std::move(outputModel));
@@ -175,7 +175,7 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainedModelDimTest",
"[LCCMainTest][BindingTests]")
{
arma::mat x;
- x.load("mnist_first250_training_4s_and_9s.arm");
+ x.load("mnist_first250_training_4s_and_9s.csv");
arma:: mat t = x;
t.shed_rows(1, 2);
@@ -185,8 +185,8 @@ TEST_CASE_METHOD(LCCTestFixture, "LCCTrainedModelDimTest",
RUN_BINDING();
- LocalCoordinateCoding* outputModel =
- std::move(params.Get("output_model"));
+ LocalCoordinateCoding<>* outputModel =
+ std::move(params.Get*>("output_model"));
SetInputParam("input_model", std::move(outputModel));
SetInputParam("test", std::move(t));
diff --git a/src/mlpack/tests/sparse_coding_test.cpp b/src/mlpack/tests/sparse_coding_test.cpp
index 582ddad38e..fa22dbc9cc 100644
--- a/src/mlpack/tests/sparse_coding_test.cpp
+++ b/src/mlpack/tests/sparse_coding_test.cpp
@@ -57,7 +57,7 @@ TEMPLATE_TEST_CASE("SparseCodingTestCodingStepLasso", "[SparseCodingTest]",
uword nAtoms = 25;
arma::mat inX; // The .arm file contains an arma::mat.
- inX.load("mnist_first250_training_4s_and_9s.arm");
+ inX.load("mnist_first250_training_4s_and_9s.csv");
MatType X = arma::conv_to::from(inX);
uword nPoints = X.n_cols;
@@ -92,7 +92,7 @@ TEMPLATE_TEST_CASE("SparseCodingTestCodingStepElasticNet", "[SparseCodingTest]",
uword nAtoms = 25;
arma::mat inX; // The .arm file contains an arma::mat.
- inX.load("mnist_first250_training_4s_and_9s.arm");
+ inX.load("mnist_first250_training_4s_and_9s.csv");
MatType X = arma::conv_to::from(inX);
uword nPoints = X.n_cols;
@@ -129,7 +129,7 @@ TEMPLATE_TEST_CASE("SparseCodingTestDictionaryStep", "[SparseCodingTest]",
uword nAtoms = 25;
arma::mat inX; // The .arm file contains an arma::mat.
- inX.load("mnist_first250_training_4s_and_9s.arm");
+ inX.load("mnist_first250_training_4s_and_9s.csv");
MatType X = arma::conv_to::from(inX);
uword nPoints = X.n_cols;
@@ -222,7 +222,7 @@ TEMPLATE_TEST_CASE("SparseCodingTrainReturnObjective", "[SparseCodingTest]",
uword nAtoms = 25;
arma::mat inX; // The .arm file contains an arma::mat.
- inX.load("mnist_first250_training_4s_and_9s.arm");
+ inX.load("mnist_first250_training_4s_and_9s.csv");
MatType X = arma::conv_to::from(inX);
uword nPoints = X.n_cols;