apaam/mlpack
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #18 from mlpack/master

Browse Source 
Merge Issue
This commit is contained in:
jeffin sam
2019-05-09 23:50:04 +05:30
committed by GitHub
parent 4d987305b6 1e2e3854e4
commit ec6caf3a77
56 changed files with 1114 additions and 479 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Doxyfile
+1 -1
View File
@@ -4,7 +4,7 @@
# Project related configuration options
#---------------------------------------------------------------------------
PROJECT_NAME = mlpack
PROJECT_NUMBER = git-master
PROJECT_NUMBER = 3.1.0
OUTPUT_DIRECTORY = ./doc
CREATE_SUBDIRS = NO
OUTPUT_LANGUAGE = English
HISTORY.md
+20 -2
View File
@@ -1,5 +1,8 @@
### mlpack 3.1.0
###### ????-??-??
### mlpack 3.1.0
###### 2019-04-25
* Add DiagonalGaussianDistribution and DiagonalGMM classes to speed up the
diagonal covariance computation and deprecate DiagonalConstraint (#1666).
@@ -13,10 +16,25 @@
* Add implementation for linear support vector machine (see
`src/mlpack/methods/linear_svm`).
### mlpack 3.0.5
###### ????-??-??
* Change DBSCAN to use PointSelectionPolicy and add OrderedPointSelection (#1625).
* Residual block support (#1594).
* Bidirectional RNN (#1626).
* Dice loss layer (#1674, #1714) and hard sigmoid layer (#1776).
* `output` option changed to `predictions` and `output_probabilities` to
`probabilities` for Naive Bayes binding (`mlpack_nbc`/`nbc()`). Old options
are now deprecated and will be preserved until mlpack 4.0.0 (#1616).
* Add support for Diagonal GMMs to HMM code (#1658, #1666). This can provide
large speedup when a diagonal GMM is acceptable as an emission probability
distribution.
* Python binding improvements: check parameter type (#1717), avoid copying
Pandas dataframes (#1711), handle Pandas Series objects (#1700).
### mlpack 3.0.4
###### 2018-11-13
* Bump minimum CMake version to 3.3.2.
README.md
+1 -1
View File
@@ -23,7 +23,7 @@ src="https://cdn.rawgit.com/mlpack/mlpack.org/e7d36ed8/mlpack-black.svg" style="
<p align="center">
<em>
Download:
<a href="https://www.mlpack.org/files/mlpack-3.0.4.tar.gz">current stable version (3.0.4)</a>
<a href="https://www.mlpack.org/files/mlpack-3.1.0.tar.gz">current stable version (3.1.0)</a>
</em>
</p>
doc/guide/build_windows.hpp
+64 -56
View File
@@ -8,16 +8,10 @@
@section build_windows_intro Introduction
This document discusses how to build mlpack for Windows from source, so you can
later create your own C++ applications. There are a couple of other tutorials
for Windows, but they may be out of date:
* <a href="https://github.com/mlpack/mlpack/wiki/WindowsBuild">Github wiki Windows Build page</a><br/>
* <a href="http://keon.io/mlpack-on-windows">Keon's tutorial for mlpack 2.0.3</a><br/>
* <a href="https://overdosedblog.wordpress.com/2016/08/15/once_again/">Kirizaki's tutorial for mlpack 2</a><br/>
Those guides could be used in addition to this tutorial. Furthermore, mlpack is
now available for Windows installation through vcpkg:
This tutorial will show you how to build mlpack for Windows from source, so you can
later create your own C++ applications. Before you try building mlpack, you may
want to install mlpack using vcpkg for Windows. If you don't want to install
using vcpkg, skip this section and continue with the build tutorial.
- Install Git (https://git-scm.com/downloads and execute setup)
@@ -25,7 +19,7 @@ now available for Windows installation through vcpkg:
- Install vcpkg (https://github.com/Microsoft/vcpkg and execute setup)
- To install only mlpack library:
- To install the mlpack library only:
@code
PS> .\vcpkg install mlpack:x64-windows
@@ -41,12 +35,12 @@ an existing one). The library is immediately ready to be included
(via preprocessor directives) and used in your project without additional
configuration.
@section build_windows_env Environment
@section build_windows_env Build Environment
This tutorial has been designed and tested using:
- Windows 10
- Visual Studio 2017 (toolset v141)
- mlpack-3.0.4
- mlpack
- OpenBLAS.0.2.14.1
- boost_1_66_0-msvc-14.1-64
- armadillo-8.500.1
@@ -64,10 +58,10 @@ and make sure you can use it from the Command Prompt (may need to add to the PAT
@section build_windows_instructions Windows build instructions
- Unzip mlpack to "C:\mlpack\mlpack-3.0.4"
- Unzip mlpack to "C:\mlpack\mlpack"
- Open Visual Studio and select: File > New > Project from Existing Code
- Type of project: Visual C++
- Project location: "C:\mlpack\mlpack-3.0.4"
- Project location: "C:\mlpack\mlpack"
- Project name: mlpack
- Finish
- We will use this Visual Studio project to get the OpenBLAS dependency in the next section
@@ -86,72 +80,86 @@ and make sure you can use it from the Command Prompt (may need to add to the PAT
You can either get Boost via NuGet or you can download the prebuilt Windows binaries separately.
This tutorial follows the second approach for simplicity.
- Download the "Prebuilt Windows binaries" of the Boost library ("boost_1_66_0-msvc-14.1-64") from
- Download the "Prebuilt Windows binaries" of the Boost library ("boost_1_66_0-msvc-14.1-64") from
<a href="https://sourceforge.net/projects/boost/files/boost-binaries/">Sourceforge</a>
@note Make sure you download the MSVC version that matches your Visual Studio
- Install or unzip to "C:\boost\boost_1_66_0"
- Install or unzip to "C:\boost\"
<b> Armadillo Dependency </b>
- Download "Armadillo" (armadillo-8.500.1.tar.xz) from <a href="http://arma.sourceforge.net/download.html">Sourceforge</a>
- Unzip to "C:\mlpack\armadillo-8.500.1"
- Create a "build" directory into "C:\mlpack\armadillo-8.500.1\"
- Open the Command Prompt and navigate to "C:\mlpack\armadillo-8.500.1\build"
- Run cmake:
- Download the newest version of Armadillo from <a href="http://arma.sourceforge.net/download.html">Sourceforge</a>
- Unzip to "C:\mlpack\armadillo"
- Create a "build" directory into "C:\mlpack\armadillo\"
- Open the Command Prompt and navigate to "C:\mlpack\armadillo\build"
- Run cmake:
@code
cmake -G "Visual Studio 15 2017 Win64" -DBLAS_LIBRARY:FILEPATH="C:/mlpack/mlpack-3.0.4/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DLAPACK_LIBRARY:FILEPATH="C:/mlpack/mlpack-3.0.4/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DCMAKE_PREFIX:FILEPATH="C:/mlpack/armadillo" ..
cmake -G "Visual Studio 15 2017 Win64" -DBLAS_LIBRARY:FILEPATH="C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DLAPACK_LIBRARY:FILEPATH="C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" ..
@endcode
@note If you are using different directory paths, a different configuration (e.g. Release)
or a different VS version, update the cmake command accordingly.
- Once it has successfully finished, open "C:\mlpack\armadillo-8.500.1\build\armadillo.sln"
- Once it has successfully finished, open "C:\mlpack\armadillo\build\armadillo.sln"
- Build > Build Solution
- Once it has successfully finished, close Visual Studio
@section build_windows_mlpack Building mlpack
- Create a "build" directory into "C:\mlpack\mlpack-3.0.4\"
- Use either the CMake GUI or the CMake command line to configure Armadillo.
- To use the CMake GUI, open "CMake".
- For "Where is the source code:" set `C:\mlpack\mlpack-3.0.4\`
- For "Where to build the binaries:" set `C:\mlpack\mlpack-3.0.4\build`
- Click `Configure`
- If there is an error and Armadillo is not found, try "Add Entry" with the
following variables and reconfigure:
- Name: `ARMADILLO_INCLUDE_DIR`; type `PATH`; value `C:/mlpack/armadillo-8.500.1/include/`
- Name: `ARMADILLO_LIBRARY`; type `FILEPATH`; value `C:/mlpack/armadillo-8.500.1/build/Debug/armadillo.lib`
- Name: `BLAS_LIBRARY`; type `FILEPATH`; value `C:/mlpack/mlpack-3.0.4/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a`
- Name: `LAPACK_LIBRARY`; type `FILEPATH`; value `C:/mlpack/mlpack-3.0.4/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a`
- If there is an error and Boost is not found, try "Add Entry" with the
following variables and reconfigure:
- Name: `BOOST_INCLUDEDIR`; type `PATH`; value `C:/boost/boost_1_66_0/`
- Name: `BOOST_LIBRARYDIR`; type `PATH`; value `C:/boost/boost_1_66_0/lib64-msvc-14.1`
- If Boost is still not found, try adding the following variables and
reconfigure:
- Name: `Boost_INCLUDE_DIR`; type `PATH`; value `C:/boost/boost_1_66_0/`
- Name: `Boost_PROGRAM_OPTIONS_LIBRARY_DEBUG`; type `FILEPATH`; value should be `C:/boost/boost_1_66_0/lib64-msvc-14.1/boost_program_options-vc141-mt-gd-x64-1_66.lib`
- Name: `Boost_PROGRAM_OPTIONS_LIBRARY_RELEASE`; type `FILEPATH`; value should be `C:/boost/boost_1_66_0/lib64-msvc-14.1/boost_program_options-vc141-mt-x64-1_66.lib`
- Name: `Boost_SERIALIZATION_LIBRARY_DEBUG`; type `FILEPATH`; value should be `C:/boost/boost_1_66_0/lib64-msvc-14.1/boost_serialization-vc141-mt-gd-x64-1_66.lib`
- Name: `Boost_SERIALIZATION_LIBRARY_RELEASE`; type `FILEPATH`; value should be `C:/boost/boost_1_66_0/lib64-msvc-14.1/boost_program_options-vc141-mt-x64-1_66.lib`
- Name: `Boost_UNIT_TEST_FRAMEWORK_LIBRARY_DEBUG`; type `FILEPATH`; value should be `C:/boost/boost_1_66_0/lib64-msvc-14.1/boost_unit_test_framework-vc141-mt-gd-x64-1_66.lib`
- Name: `Boost_UNIT_TEST_FRAMEWORK_LIBRARY_RELEASE`; type `FILEPATH`; value should be `C:/boost/boost_1_66_0/lib64-msvc-14.1/boost_unit_test_framework-vc141-mt-x64-1_66.lib`
- Once CMake has configured successfully, hit "Generate" to create the `.sln` file.
- To use the CMake command line prompt:
- Open the Command Prompt and navigate to "C:\mlpack\mlpack-3.0.4\build"
- Run cmake:
- Create a "build" directory into "C:\mlpack\mlpack\"
- You can generate the project using either cmake via command line or GUI. If you prefer to use GUI, refer to the \ref build_windows_appendix "appendix"
- To use the CMake command line prompt, open the Command Prompt and navigate to "C:\mlpack\mlpack\build"
- Run cmake:
@code
cmake -G "Visual Studio 15 2017 Win64" -DBLAS_LIBRARY:FILEPATH="C:/mlpack/mlpack-3.0.4/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DLAPACK_LIBRARY:FILEPATH="C:/mlpack/mlpack-3.0.4/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DARMADILLO_INCLUDE_DIR="C:/mlpack/armadillo-8.500.1/include" -DARMADILLO_LIBRARY:FILEPATH="C:/mlpack/armadillo-8.500.1/build/Debug/armadillo.lib" -DBOOST_INCLUDEDIR:PATH="C:/boost/boost_1_66_0/" -DBOOST_LIBRARYDIR:PATH="C:/boost/boost_1_66_0/lib64-msvc-14.1" -DDEBUG=OFF -DPROFILE=OFF ..
cmake -G "Visual Studio 15 2017 Win64" -DBLAS_LIBRARY:FILEPATH="C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DLAPACK_LIBRARY:FILEPATH="C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a" -DARMADILLO_INCLUDE_DIR="C:/mlpack/armadillo/include" -DARMADILLO_LIBRARY:FILEPATH="C:/mlpack/armadillo/build/Debug/armadillo.lib" -DBOOST_INCLUDEDIR:PATH="C:/boost/" -DBOOST_LIBRARYDIR:PATH="C:/boost/lib64-msvc-14.1" -DDEBUG=OFF -DPROFILE=OFF ..
@endcode
- Once CMake configuration has successfully finished, open "C:\mlpack\mlpack-3.0.4\build\mlpack.sln"
@note cmake will attempt to automatically download the ensmallen dependency. If for some reason cmake can't download the dependency, you will need to manually download ensmallen from http://ensmallen.org/ and extract it to "C:\mlpack\mlpack\deps\". Then, specify the path to ensmallen using the flag: -DENSMALLEN_INCLUDE_DIR=C:/mlpack/mlpack/deps/ensmallen/include
- Once CMake configuration has successfully finished, open "C:\mlpack\mlpack\build\mlpack.sln"
- Build > Build Solution (this may be by default in Debug mode)
- Once it has sucessfully finished, you will find the library files you need in: "C:\mlpack\mlpack-3.0.4\build\Debug" (or "C:\mlpack\mlpack-3.0.4\build\Release" if you changed to Release mode)
- Once it has sucessfully finished, you will find the library files you need in: "C:\mlpack\mlpack\build\Debug" (or "C:\mlpack\mlpack\build\Release" if you changed to Release mode)
You are ready to create your first application, take a look at the @ref sample_ml_app "Sample C++ ML App"
@section build_windows_appendix Appendix
If you prefer to use cmake GUI, follow these instructions:
- To use the CMake GUI, open "CMake".
- For "Where is the source code:" set `C:\mlpack\mlpack\`
- For "Where to build the binaries:" set `C:\mlpack\mlpack\build`
- Click `Configure`
- If there is an error and Armadillo is not found, try "Add Entry" with the
following variables and reconfigure:
- Name: `ARMADILLO_INCLUDE_DIR`; type `PATH`; value `C:/mlpack/armadillo/include/`
- Name: `ARMADILLO_LIBRARY`; type `FILEPATH`; value `C:/mlpack/armadillo/build/Debug/armadillo.lib`
- Name: `BLAS_LIBRARY`; type `FILEPATH`; value `C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a`
- Name: `LAPACK_LIBRARY`; type `FILEPATH`; value `C:/mlpack/mlpack/packages/OpenBLAS.0.2.14.1/lib/native/lib/x64/libopenblas.dll.a`
- If there is an error and Boost is not found, try "Add Entry" with the
following variables and reconfigure:
- Name: `BOOST_INCLUDEDIR`; type `PATH`; value `C:/boost/`
- Name: `BOOST_LIBRARYDIR`; type `PATH`; value `C:/boost/lib64-msvc-14.1`
- If Boost is still not found, try adding the following variables and
reconfigure:
- Name: `Boost_INCLUDE_DIR`; type `PATH`; value `C:/boost/`
- Name: `Boost_PROGRAM_OPTIONS_LIBRARY_DEBUG`; type `FILEPATH`; value should be `C:/boost/lib64-msvc-14.1/boost_program_options-vc141-mt-gd-x64-1_66.lib`
- Name: `Boost_PROGRAM_OPTIONS_LIBRARY_RELEASE`; type `FILEPATH`; value should be `C:/boost/lib64-msvc-14.1/boost_program_options-vc141-mt-x64-1_66.lib`
- Name: `Boost_SERIALIZATION_LIBRARY_DEBUG`; type `FILEPATH`; value should be `C:/boost/lib64-msvc-14.1/boost_serialization-vc141-mt-gd-x64-1_66.lib`
- Name: `Boost_SERIALIZATION_LIBRARY_RELEASE`; type `FILEPATH`; value should be `C:/boost/lib64-msvc-14.1/boost_program_options-vc141-mt-x64-1_66.lib`
- Name: `Boost_UNIT_TEST_FRAMEWORK_LIBRARY_DEBUG`; type `FILEPATH`; value should be `C:/boost/lib64-msvc-14.1/boost_unit_test_framework-vc141-mt-gd-x64-1_66.lib`
- Name: `Boost_UNIT_TEST_FRAMEWORK_LIBRARY_RELEASE`; type `FILEPATH`; value should be `C:/boost/lib64-msvc-14.1/boost_unit_test_framework-vc141-mt-x64-1_66.lib`
- Once CMake has configured successfully, hit "Generate" to create the `.sln` file.
@section build_windows_additional_information Additional Information
If you are facing issues during the build process of mlpack, you may take a look at other third-party tutorials for Windows, but they may be out of date:
* <a href="https://github.com/mlpack/mlpack/wiki/WindowsBuild">Github wiki Windows Build page</a><br/>
* <a href="http://keon.io/mlpack-on-windows">Keon's tutorial for mlpack 2.0.3</a><br/>
* <a href="https://overdosedblog.wordpress.com/2016/08/15/once_again/">Kirizaki's tutorial for mlpack 2</a><br/>
*/
src/mlpack/CMakeLists.txt
+1 -1
View File
@@ -44,7 +44,7 @@ target_link_libraries(mlpack ${MLPACK_LIBRARIES})
set_target_properties(mlpack
PROPERTIES
VERSION 3.0
VERSION 3.1
SOVERSION 3
)
src/mlpack/core.hpp
+1
View File
@@ -81,6 +81,7 @@
#include <mlpack/core/math/range.hpp>
#include <mlpack/core/math/round.hpp>
#include <mlpack/core/math/shuffle_data.hpp>
#include <mlpack/core/math/ccov.hpp>
#include <mlpack/core/math/make_alias.hpp>
#include <mlpack/core/dists/discrete_distribution.hpp>
#include <mlpack/core/dists/gaussian_distribution.hpp>
src/mlpack/core/arma_extend/CMakeLists.txt
-5
View File
@@ -2,13 +2,8 @@
# Anything not in this list will not be compiled into mlpack.
set(SOURCES
arma_extend.hpp
fn_ccov.hpp
fn_inplace_reshape.hpp
glue_ccov_meat.hpp
glue_ccov_proto.hpp
hdf5_misc.hpp
op_ccov_meat.hpp
op_ccov_proto.hpp
SpMat_extra_bones.hpp
SpMat_extra_meat.hpp
Mat_extra_bones.hpp
src/mlpack/core/arma_extend/arma_extend.hpp
-9
View File
@@ -5,9 +5,6 @@
* Include Armadillo extensions which currently are not part of the main
* Armadillo codebase.
*
* This will allow the use of the ccov() function (which performs the same
* function as cov(trans(X)) but without the cost of computing trans(X)). This
* also gives sparse matrix support, if it is necessary.
*/
#ifndef MLPACK_CORE_ARMA_EXTEND_ARMA_EXTEND_HPP
#define MLPACK_CORE_ARMA_EXTEND_ARMA_EXTEND_HPP
@@ -55,12 +52,6 @@ namespace arma {
// u64/s64
#include "hdf5_misc.hpp"
// ccov()
#include "op_ccov_proto.hpp"
#include "op_ccov_meat.hpp"
#include "glue_ccov_proto.hpp"
#include "glue_ccov_meat.hpp"
#include "fn_ccov.hpp"
// inplace_reshape()
#include "fn_inplace_reshape.hpp"
src/mlpack/core/arma_extend/fn_ccov.hpp
-34
View File
@@ -1,34 +0,0 @@
//! \addtogroup fn_ccov
//! @{
template<typename T1>
inline
const Op<T1, op_ccov>
ccov(const Base<typename T1::elem_type,T1>& X, const uword norm_type = 0)
{
arma_extra_debug_sigprint();
arma_debug_check( (norm_type > 1), "ccov(): norm_type must be 0 or 1");
return Op<T1, op_ccov>(X.get_ref(), norm_type, 0);
}
template<typename T1, typename T2>
inline
const Glue<T1,T2,glue_ccov>
cov(const Base<typename T1::elem_type, T1>& A, const Base<typename T1::elem_type,T2>& B, const uword norm_type = 0)
{
arma_extra_debug_sigprint();
arma_debug_check( (norm_type > 1), "ccov(): norm_type must be 0 or 1");
return Glue<T1, T2, glue_ccov>(A.get_ref(), B.get_ref(), norm_type);
}
//! @}
src/mlpack/core/arma_extend/glue_ccov_meat.hpp
-144
View File
@@ -1,144 +0,0 @@
//! \addtogroup glue_cov
//! @{
template<typename eT>
inline
void
glue_ccov::direct_ccov(Mat<eT>& out, const Mat<eT>& A, const Mat<eT>& B, const uword norm_type)
{
arma_extra_debug_sigprint();
if(A.is_vec() && B.is_vec())
{
arma_debug_check( (A.n_elem != B.n_elem), "ccov(): the number of elements in A and B must match" );
const eT* A_ptr = A.memptr();
const eT* B_ptr = B.memptr();
eT A_acc = eT(0);
eT B_acc = eT(0);
eT out_acc = eT(0);
const uword N = A.n_elem;
for(uword i=0; i<N; ++i)
{
const eT A_tmp = A_ptr[i];
const eT B_tmp = B_ptr[i];
A_acc += A_tmp;
B_acc += B_tmp;
out_acc += A_tmp * B_tmp;
}
out_acc -= (A_acc * B_acc)/eT(N);
const eT norm_val = (norm_type == 0) ? ( (N > 1) ? eT(N-1) : eT(1) ) : eT(N);
out.set_size(1,1);
out[0] = out_acc/norm_val;
}
else
{
arma_debug_assert_same_size(A, B, "ccov()");
const uword N = A.n_cols;
const eT norm_val = (norm_type == 0) ? ( (N > 1) ? eT(N-1) : eT(1) ) : eT(N);
out = A * trans(B);
out -= (sum(A) * trans(sum(B))) / eT(N);
out /= norm_val;
}
}
template<typename T>
inline
void
glue_ccov::direct_ccov(Mat< std::complex<T> >& out, const Mat< std::complex<T> >& A, const Mat< std::complex<T> >& B, const uword norm_type)
{
arma_extra_debug_sigprint();
typedef typename std::complex<T> eT;
if(A.is_vec() && B.is_vec())
{
arma_debug_check( (A.n_elem != B.n_elem), "cov(): the number of elements in A and B must match" );
const eT* A_ptr = A.memptr();
const eT* B_ptr = B.memptr();
eT A_acc = eT(0);
eT B_acc = eT(0);
eT out_acc = eT(0);
const uword N = A.n_elem;
for(uword i=0; i<N; ++i)
{
const eT A_tmp = A_ptr[i];
const eT B_tmp = B_ptr[i];
A_acc += A_tmp;
B_acc += B_tmp;
out_acc += std::conj(A_tmp) * B_tmp;
}
out_acc -= (std::conj(A_acc) * B_acc)/eT(N);
const eT norm_val = (norm_type == 0) ? ( (N > 1) ? eT(N-1) : eT(1) ) : eT(N);
out.set_size(1,1);
out[0] = out_acc/norm_val;
}
else
{
arma_debug_assert_same_size(A, B, "ccov()");
const uword N = A.n_cols;
const eT norm_val = (norm_type == 0) ? ( (N > 1) ? eT(N-1) : eT(1) ) : eT(N);
out = A * trans(conj(B));
out -= (sum(A) * trans(conj(sum(B)))) / eT(N);
out /= norm_val;
}
}
template<typename T1, typename T2>
inline
void
glue_ccov::apply(Mat<typename T1::elem_type>& out, const Glue<T1,T2,glue_ccov>& X)
{
arma_extra_debug_sigprint();
typedef typename T1::elem_type eT;
const unwrap_check<T1> A_tmp(X.A, out);
const unwrap_check<T2> B_tmp(X.B, out);
const Mat<eT>& A = A_tmp.M;
const Mat<eT>& B = B_tmp.M;
const uword norm_type = X.aux_uword;
if(&A != &B)
{
glue_ccov::direct_ccov(out, A, B, norm_type);
}
else
{
op_ccov::direct_ccov(out, A, norm_type);
}
}
//! @}
src/mlpack/core/arma_extend/glue_ccov_proto.hpp
-15
View File
@@ -1,15 +0,0 @@
//! \addtogroup glue_ccov
//! @{
class glue_ccov
{
public:
template<typename eT> inline static void direct_ccov(Mat<eT>& out, const Mat<eT>& A, const Mat<eT>& B, const uword norm_type);
template<typename T> inline static void direct_ccov(Mat< std::complex<T> >& out, const Mat< std::complex<T> >& A, const Mat< std::complex<T> >& B, const uword norm_type);
template<typename T1, typename T2> inline static void apply(Mat<typename T1::elem_type>& out, const Glue<T1, T2, glue_ccov>& X);
};
//! @}
src/mlpack/core/arma_extend/op_ccov_meat.hpp
-97
View File
@@ -1,97 +0,0 @@
//! \addtogroup op_cov
//! @{
template<typename eT>
inline
void
op_ccov::direct_ccov(Mat<eT>& out, const Mat<eT>& A, const uword norm_type)
{
arma_extra_debug_sigprint();
if(A.is_vec())
{
if(A.n_rows == 1)
{
out = var(trans(A), norm_type);
}
else
{
out = var(A, norm_type);
}
}
else
{
const uword N = A.n_cols;
const eT norm_val = (norm_type == 0) ? ( (N > 1) ? eT(N-1) : eT(1) ) : eT(N);
const Col<eT> acc = sum(A, 1);
out = A * trans(A);
out -= (acc * trans(acc)) / eT(N);
out /= norm_val;
}
}
template<typename T>
inline
void
op_ccov::direct_ccov(Mat< std::complex<T> >& out, const Mat< std::complex<T> >& A, const uword norm_type)
{
arma_extra_debug_sigprint();
typedef typename std::complex<T> eT;
if(A.is_vec())
{
if(A.n_rows == 1)
{
const Mat<T> tmp_mat = var(trans(A), norm_type);
out.set_size(1,1);
out[0] = tmp_mat[0];
}
else
{
const Mat<T> tmp_mat = var(A, norm_type);
out.set_size(1,1);
out[0] = tmp_mat[0];
}
}
else
{
const uword N = A.n_cols;
const eT norm_val = (norm_type == 0) ? ( (N > 1) ? eT(N-1) : eT(1) ) : eT(N);
const Col<eT> acc = sum(A, 1);
out = A * trans(conj(A));
out -= (acc * trans(conj(acc))) / eT(N);
out /= norm_val;
}
}
template<typename T1>
inline
void
op_ccov::apply(Mat<typename T1::elem_type>& out, const Op<T1,op_ccov>& in)
{
arma_extra_debug_sigprint();
typedef typename T1::elem_type eT;
const unwrap_check<T1> tmp(in.m, out);
const Mat<eT>& A = tmp.M;
const uword norm_type = in.aux_uword_a;
op_ccov::direct_ccov(out, A, norm_type);
}
//! @}
src/mlpack/core/arma_extend/op_ccov_proto.hpp
-18
View File
@@ -1,18 +0,0 @@
//! \addtogroup op_cov
//! @{
class op_ccov
{
public:
template<typename eT> inline static void direct_ccov(Mat<eT>& out, const Mat<eT>& X, const uword norm_type);
template<typename T> inline static void direct_ccov(Mat< std::complex<T> >& out, const Mat< std::complex<T> >& X, const uword norm_type);
template<typename T1> inline static void apply(Mat<typename T1::elem_type>& out, const Op<T1,op_ccov>& in);
};
//! @}
src/mlpack/core/data/normalize_labels_impl.hpp
+13 -14
View File
@@ -39,32 +39,31 @@ void NormalizeLabels(const RowType& labelsIn,
// we'll resize it back down to its actual size.
mapping.set_size(labelsIn.n_elem);
labels.set_size(labelsIn.n_elem);
// Map for mapping labelIn to their label.
std::unordered_map<eT, size_t> labelMap;
size_t curLabel = 0;
for (size_t i = 0; i < labelsIn.n_elem; ++i)
{
bool found = false;
for (size_t j = 0; j < curLabel; ++j)
// If labelsIn[i] is already in the map, use the existing label.
if (labelMap.count(labelsIn[i]) > 0)
{
// Is the label already in the list of labels we have seen?
if (labelsIn[i] == mapping[j])
{
labels[i] = j;
found = true;
break;
}
labels[i] = labelMap[labelsIn[i]];
}
// Do we need to add this new label?
if (!found)
else
{
mapping[curLabel] = labelsIn[i];
// If labelsIn[i] not there then add it to map.
labelMap[labelsIn[i]] = curLabel;
labels[i] = curLabel;
++curLabel;
}
}
// Resize mapping back down to necessary size.
mapping.resize(curLabel);
// Mapping array created with encoded labels.
for (auto it = labelMap.begin(); it != labelMap.end(); ++it)
{
mapping[it->second] = it->first;
}
}
/**
src/mlpack/core/dists/gamma_distribution.hpp
+4 -4
View File
@@ -6,7 +6,7 @@
* Implementation of a Gamma distribution of multidimensional data that fits
* gamma parameters (alpha, beta) to data.
* The fitting is done independently for each dataset dimension (row), based on
* the assumption each dimension is fully indepeendent.
* the assumption each dimension is fully independent.
*
* Based on "Estimating a Gamma Distribution" by Thomas P. Minka:
* research.microsoft.com/~minka/papers/minka-gamma.pdf
@@ -154,7 +154,7 @@ class GammaDistribution
* @param x The 1-dimensional observation.
* @param dim The dimension for which to calculate the probability.
*/
double Probability(double x, size_t dim) const;
double Probability(double x, const size_t dim) const;
/**
* This function returns the logarithm of the probability of a group of
@@ -179,12 +179,12 @@ class GammaDistribution
/**
* This function returns the logarithm of the probability of a single
* observation.
* observation.
*
* @param x The 1-dimensional observation.
* @param dim The dimension for which to calculate the probability.
*/
double LogProbability(double x, size_t dim) const;
double LogProbability(double x, const size_t dim) const;
/**
* This function returns an observation of this distribution.
src/mlpack/core/math/CMakeLists.txt
+2
View File
@@ -18,6 +18,8 @@ set(SOURCES
range_impl.hpp
round.hpp
shuffle_data.hpp
ccov.hpp
ccov_impl.hpp
)
# add directory name to sources
src/mlpack/core/math/ccov.hpp
+39
View File
@@ -0,0 +1,39 @@
/**
* @file ccov.hpp
* @author Ryan Curtin
* @author Conrad Sanderson
*
* ColumnCovariance(X) is same as cov(trans(X)) but without the cost
* of computing trans(X)
*
* mlpack is free software; you may redistribute it and/or modify it under the
* terms of the 3-clause BSD license. You should have received a copy of the
* 3-clause BSD license along with mlpack. If not, see
* http://www.opensource.org/licenses/BSD-3-Clause for more information.
*/
#ifndef MLPACK_CORE_MATH_CCOV_HPP
#define MLPACK_CORE_MATH_CCOV_HPP
#include <mlpack/prereqs.hpp>
namespace mlpack {
namespace math /** Miscellaneous math routines. */ {
template<typename eT>
inline
arma::Mat<eT>
ColumnCovariance(const arma::Mat<eT>& A, const size_t norm_type = 0);
template<typename T>
inline
arma::Mat< std::complex<T> >
ColumnCovariance(const arma::Mat< std::complex<T> >& A,
const size_t norm_type = 0);
} // namespace math
} // namespace mlpack
// Include implementation
#include "ccov_impl.hpp"
#endif // MLPACK_CORE_MATH_CCOV_HPP
src/mlpack/core/math/ccov_impl.hpp
+100
View File
@@ -0,0 +1,100 @@
/**
* @file ccov_impl.hpp
* @author Ryan Curtin
* @author Conrad Sanderson
*
* ColumnCovariance(X) is same as cov(trans(X)) but without the cost of computing trans(X)
*
* mlpack is free software; you may redistribute it and/or modify it under the
* terms of the 3-clause BSD license. You should have received a copy of the
* 3-clause BSD license along with mlpack. If not, see
* http://www.opensource.org/licenses/BSD-3-Clause for more information.
*/
#ifndef MLPACK_CORE_MATH_CCOV_IMPL_HPP
#define MLPACK_CORE_MATH_CCOV_IMPL_HPP
#include "ccov.hpp"
namespace mlpack {
namespace math /** Miscellaneous math routines. */ {
template<typename eT>
inline arma::Mat<eT> ColumnCovariance(const arma::Mat<eT>& x,
const size_t normType)
{
if (normType > 1)
{
Log::Fatal << "ColumnCovariance(): norm_type must be 0 or 1!" << std::endl;
}
arma::Mat<eT> out;
if (x.n_elem > 0)
{
const arma::Mat<eT>& xAlias = (x.n_cols == 1) ?
arma::Mat<eT>(const_cast<eT*>(x.memptr()), x.n_cols, x.n_rows, false,
false) :
arma::Mat<eT>(const_cast<eT*>(x.memptr()), x.n_rows, x.n_cols, false,
false);
const size_t n = xAlias.n_cols;
const eT normVal = (normType == 0) ? ((n > 1) ? eT(n - 1) : eT(1)) : eT(n);
const arma::Mat<eT> tmp = xAlias.each_col() - arma::mean(xAlias, 1);
out = tmp * tmp.t();
out /= normVal;
}
return out;
}
template<typename T>
inline arma::Mat<std::complex<T>> ColumnCovariance(
const arma::Mat<std::complex<T>>& x,
const size_t normType)
{
if (normType > 1)
{
Log::Fatal << "ColumnCovariance(): norm_type must be 0 or 1" << std::endl;
}
typedef typename std::complex<T> eT;
arma::Mat<eT> out;
if (x.is_vec())
{
if (x.n_rows == 1)
{
const arma::Mat<T> tmpMat = arma::var(arma::trans(x), normType);
out.set_size(1, 1);
out[0] = tmpMat[0];
}
else
{
const arma::Mat<T> tmpMat = arma::var(x, normType);
out.set_size(1, 1);
out[0] = tmpMat[0];
}
}
else
{
const size_t n = x.n_cols;
const eT normVal = (normType == 0) ?
((n > 1) ? eT(n - 1) : eT(1)) : eT(n);
const arma::Col<eT> acc = arma::sum(x, 1);
out = x * arma::trans(arma::conj(x));
out -= (acc * arma::trans(arma::conj(acc))) / eT(n);
out /= normVal;
}
return out;
}
} // namespace math
} // namespace mlpack
#endif // MLPACK_CORE_MATH_CCOV_IMPL_HPP
src/mlpack/core/math/lin_alg.cpp
+4 -4
View File
@@ -10,7 +10,7 @@
* http://www.opensource.org/licenses/BSD-3-Clause for more information.
*/
#include "lin_alg.hpp"
#include <mlpack/prereqs.hpp>
#include <mlpack/core.hpp>
#include <mlpack/core/math/random.hpp>
using namespace mlpack;
@@ -60,7 +60,7 @@ void mlpack::math::WhitenUsingSVD(const arma::mat& x,
arma::mat covX, u, v, invSMatrix, temp1;
arma::vec sVector;
covX = ccov(x);
covX = mlpack::math::ColumnCovariance(x);
svd(u, sVector, v, covX);
@@ -85,7 +85,7 @@ void mlpack::math::WhitenUsingEig(const arma::mat& x,
arma::vec eigenvalues;
// Get eigenvectors of covariance of input matrix.
eig_sym(eigenvalues, eigenvectors, ccov(x));
eig_sym(eigenvalues, eigenvectors, mlpack::math::ColumnCovariance(x));
// Generate diagonal matrix using 1 / sqrt(eigenvalues) for each value.
VectorPower(eigenvalues, -0.5);
@@ -135,7 +135,7 @@ void mlpack::math::Orthogonalize(const arma::mat& x, arma::mat& W)
// eigendecomposition of the matrix A.
arma::mat eigenvalues, eigenvectors;
arma::vec egval;
eig_sym(egval, eigenvectors, ccov(x));
eig_sym(egval, eigenvectors, mlpack::math::ColumnCovariance(x));
VectorPower(egval, -0.5);
eigenvalues.zeros(egval.n_elem, egval.n_elem);
src/mlpack/core/util/mlpack_main.hpp
+3
View File
@@ -177,6 +177,9 @@ PARAM_FLAG("copy_all_inputs", "If specified, all input parameters will be deep"
#define PRINT_CALL mlpack::bindings::markdown::ProgramCall
#define BINDING_IGNORE_CHECK mlpack::bindings::markdown::IgnoreCheck
// This doesn't actually matter for this binding type.
#define BINDING_MATRIX_TRANSPOSED true
namespace mlpack {
namespace util {
src/mlpack/core/util/version.hpp
+2 -2
View File
@@ -17,8 +17,8 @@
// The version of mlpack. If this is a git repository, this will be a version
// with higher number than the most recent release.
#define MLPACK_VERSION_MAJOR 3
#define MLPACK_VERSION_MINOR 0
#define MLPACK_VERSION_PATCH 5
#define MLPACK_VERSION_MINOR 1
#define MLPACK_VERSION_PATCH 1
// The name of the version (for use by --version).
namespace mlpack {
src/mlpack/methods/adaboost/adaboost.hpp
+14 -6
View File
@@ -106,9 +106,6 @@ class AdaBoost
*/
AdaBoost(const double tolerance = 1e-6);
// Return the value of ztProduct.
double ZtProduct() { return ztProduct; }
//! Get the tolerance for stopping the optimization during training.
double Tolerance() const { return tolerance; }
//! Modify the tolerance for stopping the optimization during training.
@@ -174,14 +171,25 @@ class AdaBoost
std::vector<WeakLearnerType> wl;
//! The weights corresponding to each weak learner.
std::vector<double> alpha;
//! To check for the bound for the Hamming loss.
double ztProduct;
}; // class AdaBoost
} // namespace adaboost
} // namespace mlpack
//! Set the serialization version of the adaboost class.
namespace boost {
namespace serialization {
template<typename WeakLearnerType, typename MatType>
struct version<mlpack::adaboost::AdaBoost<WeakLearnerType, MatType>>
{
BOOST_STATIC_CONSTANT(int, value = 1);
};
} // namespace serialization
} // namespace boost
// Include implementation.
#include "adaboost_impl.hpp"
#endif
src/mlpack/methods/adaboost/adaboost_impl.hpp
+9 -5
View File
@@ -56,8 +56,7 @@ AdaBoost<WeakLearnerType, MatType>::AdaBoost(
template<typename WeakLearnerType, typename MatType>
AdaBoost<WeakLearnerType, MatType>::AdaBoost(const double tolerance) :
numClasses(0),
tolerance(tolerance),
ztProduct(1.0)
tolerance(tolerance)
{
// Nothing to do.
}
@@ -83,7 +82,7 @@ double AdaBoost<WeakLearnerType, MatType>::Train(
// changing by less than the tolerance.
double rt, crt = 0.0, alphat = 0.0, zt;
ztProduct = 1.0;
double ztProduct = 1.0;
// To be used for prediction by the weak learner.
arma::Row<size_t> predictedLabels(labels.n_cols);
@@ -242,11 +241,16 @@ void AdaBoost<WeakLearnerType, MatType>::Classify(
template<typename WeakLearnerType, typename MatType>
template<typename Archive>
void AdaBoost<WeakLearnerType, MatType>::serialize(Archive& ar,
const unsigned int /* version */)
const unsigned int version)
{
ar & BOOST_SERIALIZATION_NVP(numClasses);
ar & BOOST_SERIALIZATION_NVP(tolerance);
ar & BOOST_SERIALIZATION_NVP(ztProduct);
if (version == 0 && Archive::is_loading::value)
{
// Load unused ztProduct double and forget it.
double tmpZtProduct = 0.0;
ar & BOOST_SERIALIZATION_NVP(tmpZtProduct);
}
ar & BOOST_SERIALIZATION_NVP(alpha);
// Now serialize each weak learner.
src/mlpack/methods/ann/brnn.hpp
+4 -4
View File
@@ -100,9 +100,9 @@ class BRNN
* @param optimizer Instantiated optimizer used to train the model.
*/
template<typename OptimizerType>
void Train(arma::cube predictors,
arma::cube responses,
OptimizerType& optimizer);
double Train(arma::cube predictors,
arma::cube responses,
OptimizerType& optimizer);
/**
* Train the bidirectional recurrent neural network on the given input data.
@@ -128,7 +128,7 @@ class BRNN
* @param responses Outputs results from input training variables.
*/
template<typename OptimizerType = ens::StandardSGD>
void Train(arma::cube predictors, arma::cube responses);
double Train(arma::cube predictors, arma::cube responses);
/**
* Predict the responses to a given set of predictors. The responses will
src/mlpack/methods/ann/brnn_impl.hpp
+4 -2
View File
@@ -64,7 +64,7 @@ template<typename OutputLayerType, typename MergeLayerType,
typename MergeOutputType, typename InitializationRuleType,
typename... CustomLayers>
template<typename OptimizerType>
void BRNN<OutputLayerType, MergeLayerType, MergeOutputType,
double BRNN<OutputLayerType, MergeLayerType, MergeOutputType,
InitializationRuleType, CustomLayers...>::Train(
arma::cube predictors,
arma::cube responses,
@@ -90,13 +90,14 @@ void BRNN<OutputLayerType, MergeLayerType, MergeOutputType,
Log::Info << "BRNN::BRNN(): final objective of trained model is " << out
<< "." << std::endl;
return out;
}
template<typename OutputLayerType, typename MergeLayerType,
typename MergeOutputType, typename InitializationRuleType,
typename... CustomLayers>
template<typename OptimizerType>
void BRNN<OutputLayerType, MergeLayerType, MergeOutputType,
double BRNN<OutputLayerType, MergeLayerType, MergeOutputType,
InitializationRuleType, CustomLayers...>::Train(
arma::cube predictors,
arma::cube responses)
@@ -121,6 +122,7 @@ void BRNN<OutputLayerType, MergeLayerType, MergeOutputType,
Log::Info << "BRNN::BRNN(): final objective of trained model is " << out
<< "." << std::endl;
return out;
}
template<typename OutputLayerType, typename MergeLayerType,
src/mlpack/methods/ann/layer/CMakeLists.txt
+2
View File
@@ -86,6 +86,8 @@ set(SOURCES
transposed_convolution_impl.hpp
vr_class_reward.hpp
vr_class_reward_impl.hpp
c_relu.hpp
c_relu_impl.hpp
)
# Add directory name to sources.
src/mlpack/methods/ann/layer/c_relu.hpp
+110
View File
@@ -0,0 +1,110 @@
/**
* @file c_relu_impl.hpp
* @author Jeffin Sam
*
* Implementation of CReLU layer.
*
* mlpack is free software; you may redistribute it and/or modify it under the
* terms of the 3-clause BSD license. You should have received a copy of the
* 3-clause BSD license along with mlpack. If not, see
* http://www.opensource.org/licenses/BSD-3-Clause for more information.
*/
#ifndef MLPACK_METHODS_ANN_LAYER_C_RELU_HPP
#define MLPACK_METHODS_ANN_LAYER_C_RELU_HPP
#include <mlpack/prereqs.hpp>
namespace mlpack {
namespace ann /** Artificial Neural Network. */ {
/**
*
* A concatenated ReLU has two outputs, one ReLU and one negative ReLU,
* concatenated together. In other words, for positive x it produces [x, 0],
* and for negative x it produces [0, x]. Because it has two outputs,
* CReLU doubles the output dimension.
*
* Note:
* The CReLU doubles the output size.
*
* For more information, see the following.
*
* @code
* @inproceedings{ICML2016,
* title = {Understanding and Improving Convolutional Neural Networks
* via Concatenated Rectified Linear Units},
* author = {LWenling Shang, Kihyuk Sohn, Diogo Almeida, Honglak Lee},
* year = {2016}
* }
* @endcode
*
* @tparam InputDataType Type of the input data (arma::colvec, arma::mat,
* arma::sp_mat or arma::cube).
* @tparam OutputDataType Type of the output data (arma::colvec, arma::mat,
* arma::sp_mat or arma::cube).
*/
template <
typename InputDataType = arma::mat,
typename OutputDataType = arma::mat
>
class CReLU
{
public:
/**
* Create the CReLU object.
*/
CReLU();
/**
* Ordinary feed forward pass of a neural network, evaluating the function
* f(x) by propagating the activity forward through f.
* Works only for 2D Tenosrs.
*
* @param input Input data used for evaluating the specified function.
* @param output Resulting output activation.
*/
template<typename InputType, typename OutputType>
void Forward(const InputType&& input, OutputType&& output);
/**
* Ordinary feed backward pass of a neural network, calculating the function
* f(x) by propagating x backwards through f. Using the results from the feed
* forward pass.
*
* @param input The propagated input activation.
* @param gy The backpropagated error.
* @param g The calculated gradient.
*/
template<typename DataType>
void Backward(const DataType&& input, DataType&& gy, DataType&& g);
//! Get the output parameter.
OutputDataType const& OutputParameter() const { return outputParameter; }
//! Modify the output parameter.
OutputDataType& OutputParameter() { return outputParameter; }
//! Get the delta.
OutputDataType const& Delta() const { return delta; }
//! Modify the delta.
OutputDataType& Delta() { return delta; }
/**
* Serialize the layer.
*/
template<typename Archive>
void serialize(Archive& /* ar */, const unsigned int /* version */);
private:
//! Locally-stored delta object.
OutputDataType delta;
//! Locally-stored output parameter object.
OutputDataType outputParameter;
}; // class CReLU
} // namespace ann
} // namespace mlpack
// Include implementation.
#include "c_relu_impl.hpp"
#endif
src/mlpack/methods/ann/layer/c_relu_impl.hpp
+59
View File
@@ -0,0 +1,59 @@
/**
* @file c_relu_impl.hpp
* @author Jeffin Sam
*
* Implementation of CReLU layer.
*
* mlpack is free software; you may redistribute it and/or modify it under the
* terms of the 3-clause BSD license. You should have received a copy of the
* 3-clause BSD license along with mlpack. If not, see
* http://www.opensource.org/licenses/BSD-3-Clause for more information.
*/
#ifndef MLPACK_METHODS_ANN_LAYER_C_RELU_IMPL_HPP
#define MLPACK_METHODS_ANN_LAYER_C_RELU_IMPL_HPP
// In case it hasn't yet been included.
#include "c_relu.hpp"
namespace mlpack {
namespace ann /** Artificial Neural Network. */ {
template<typename InputDataType, typename OutputDataType>
CReLU<InputDataType, OutputDataType>::CReLU()
{
// Nothing to do here.
}
template<typename InputDataType, typename OutputDataType>
template<typename InputType, typename OutputType>
void CReLU<InputDataType, OutputDataType>::Forward(
const InputType&& input, OutputType&& output)
{
output = arma::join_cols(arma::max(input, 0.0 * input), arma::max(
(-1 * input), 0.0 * input));
}
template<typename InputDataType, typename OutputDataType>
template<typename DataType>
void CReLU<InputDataType, OutputDataType>::Backward(
const DataType&& input, DataType&& gy, DataType&& g)
{
DataType temp;
temp = gy % (input >= 0.0);
g = temp.rows(0, (input.n_rows / 2 - 1)) - temp.rows(input.n_rows / 2,
(input.n_rows - 1));
}
template<typename InputDataType, typename OutputDataType>
template<typename Archive>
void CReLU<InputDataType, OutputDataType>::serialize(
Archive& /* ar */,
const unsigned int /* version */)
{
// Nothing to do here.
}
} // namespace ann
} // namespace mlpack
#endif
src/mlpack/methods/ann/layer/layer_types.hpp
+2
View File
@@ -28,6 +28,7 @@
#include <mlpack/methods/ann/layer/join.hpp>
#include <mlpack/methods/ann/layer/layer_norm.hpp>
#include <mlpack/methods/ann/layer/leaky_relu.hpp>
#include <mlpack/methods/ann/layer/c_relu.hpp>
#include <mlpack/methods/ann/layer/flexible_relu.hpp>
#include <mlpack/methods/ann/layer/log_softmax.hpp>
#include <mlpack/methods/ann/layer/lookup.hpp>
@@ -175,6 +176,7 @@ using LayerTypes = boost::variant<
Join<arma::mat, arma::mat>*,
LayerNorm<arma::mat, arma::mat>*,
LeakyReLU<arma::mat, arma::mat>*,
CReLU<arma::mat, arma::mat>*,
Linear<arma::mat, arma::mat>*,
LinearNoBias<arma::mat, arma::mat>*,
LogSoftMax<arma::mat, arma::mat>*,
src/mlpack/methods/ann/rnn_impl.hpp
+13 -1
View File
@@ -153,13 +153,25 @@ void RNN<OutputLayerType, InitializationRuleType, CustomLayers...>::Predict(
ResetDeterministic();
}
const size_t effectiveBatchSize = std::min(batchSize,
size_t(predictors.n_cols));
Forward(std::move(arma::mat(predictors.slice(0).colptr(0),
predictors.n_rows, effectiveBatchSize, false, true)));
arma::mat resultsTemp = boost::apply_visitor(outputParameterVisitor,
network.back());
outputSize = resultsTemp.n_rows;
results = arma::zeros<arma::cube>(outputSize, predictors.n_cols, rho);
results.slice(0).submat(0, 0, results.n_rows - 1,
effectiveBatchSize - 1) = resultsTemp;
// Process in accordance with the given batch size.
for (size_t begin = 0; begin < predictors.n_cols; begin += batchSize)
{
const size_t effectiveBatchSize = std::min(batchSize,
size_t(predictors.n_cols - begin));
for (size_t seqNum = 0; seqNum < rho; ++seqNum)
for (size_t seqNum = !begin; seqNum < rho; ++seqNum)
{
Forward(std::move(arma::mat(predictors.slice(seqNum).colptr(begin),
predictors.n_rows, effectiveBatchSize, false, true)));
src/mlpack/methods/reinforcement_learning/policy/greedy_policy.hpp
+6 -2
View File
@@ -1,6 +1,7 @@
/**
* @file greedy_policy.hpp
* @author Shangtong Zhang
* @author Abhinav Sagar
*
* This file is an implementation of epsilon greedy policy.
*
@@ -41,13 +42,16 @@ class GreedyPolicy
* @param annealInterval The steps during which the probability to explore
* will anneal.
* @param minEpsilon Epsilon will never be less than this value.
* @param decayRate How much to change the model in response to the
* estimated error each time the model weights are updated.
*/
GreedyPolicy(const double initialEpsilon,
const size_t annealInterval,
const double minEpsilon) :
const double minEpsilon,
const double decayRate = 1.0) :
epsilon(initialEpsilon),
minEpsilon(minEpsilon),
delta((initialEpsilon - minEpsilon) / annealInterval)
delta(((initialEpsilon - minEpsilon) * decayRate) / annealInterval)
{ /* Nothing to do here. */ }
/**
src/mlpack/methods/reinforcement_learning/q_learning.hpp
+10
View File
@@ -100,6 +100,16 @@ class QLearning
*/
const size_t& TotalSteps() const { return totalSteps; }
//! Modify the state of the agent.
StateType& State() { return state; }
//! Get the state of the agent.
const StateType& State() const { return state; }
//! Modify the environment in which the agent is.
EnvironmentType& Environment() { return environment; }
//! Get the environment in which the agent is.
const EnvironmentType& Environment() const { return environment; }
//! Modify the training mode / test mode indicator.
bool& Deterministic() { return deterministic; }
//! Get the indicator of training mode / test mode.
src/mlpack/tests/activation_functions_test.cpp
+30
View File
@@ -565,6 +565,36 @@ BOOST_AUTO_TEST_CASE(PReLUFunctionTest)
CheckPReLUGradientCorrect(activationData, desiredGradient);
}
/**
* Basic test of the CReLU function.
*/
BOOST_AUTO_TEST_CASE(CReLUFunctionTest)
{
const arma::colvec desiredActivations("0 3.2 4.5 0 \
1 0 2 0 2 0 0 \
100.2 0 1 0 0");
const arma::colvec desiredDerivatives("0 0 0 0 \
0 0 0 0");
CReLU<> crelu;
// Test the activation function using the entire vector as input.
arma::colvec activations;
crelu.Forward(std::move(activationData), std::move(activations));
arma::colvec derivatives;
// This error vector will be set to 1 to get the derivatives.
arma::colvec error = arma::ones<arma::colvec>(desiredActivations.n_elem);
crelu.Backward(std::move(desiredActivations), std::move(error),
std::move(derivatives));
for (size_t i = 0; i < activations.n_elem; i++)
{
BOOST_REQUIRE_CLOSE(activations.at(i), desiredActivations.at(i), 1e-3);
}
for (size_t i = 0; i < derivatives.n_elem; i++)
{
BOOST_REQUIRE_CLOSE(derivatives.at(i), desiredDerivatives.at(i), 1e-3);
}
}
/**
* Basic test of the swish function.
*/
src/mlpack/tests/adaboost_test.cpp
+33 -20
View File
@@ -51,7 +51,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundIris)
// Define parameters for AdaBoost.
size_t iterations = 100;
double tolerance = 1e-10;
AdaBoost<> a(inputData, labels.row(0), numClasses, p, iterations, tolerance);
AdaBoost<> a(tolerance);
double ztProduct = a.Train(inputData, labels.row(0), numClasses, p,
iterations, tolerance);
arma::Row<size_t> predictedLabels;
a.Classify(inputData, predictedLabels);
@@ -62,7 +64,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundIris)
countError++;
double hammingLoss = (double) countError / labels.n_cols;
BOOST_REQUIRE_LE(hammingLoss, a.ZtProduct());
// Check that ztProduct is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(ztProduct), true);
BOOST_REQUIRE_LE(hammingLoss, ztProduct);
}
/**
@@ -140,7 +144,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundVertebralColumn)
// Define parameters for AdaBoost.
size_t iterations = 50;
double tolerance = 1e-10;
AdaBoost<> a(inputData, labels.row(0), numClasses, p, iterations, tolerance);
AdaBoost<> a(tolerance);
double ztProduct = a.Train(inputData, labels.row(0), numClasses, p,
iterations, tolerance);
arma::Row<size_t> predictedLabels;
a.Classify(inputData, predictedLabels);
@@ -151,7 +157,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundVertebralColumn)
countError++;
double hammingLoss = (double) countError / labels.n_cols;
BOOST_REQUIRE_LE(hammingLoss, a.ZtProduct());
// Check that ztProduct is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(ztProduct), true);
BOOST_REQUIRE_LE(hammingLoss, ztProduct);
}
/**
@@ -227,7 +235,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundNonLinearSepData)
// Define parameters for AdaBoost.
size_t iterations = 50;
double tolerance = 1e-10;
AdaBoost<> a(inputData, labels.row(0), numClasses, p, iterations, tolerance);
AdaBoost<> a(tolerance);
double ztProduct = a.Train(inputData, labels.row(0), numClasses, p,
iterations, tolerance);
arma::Row<size_t> predictedLabels;
a.Classify(inputData, predictedLabels);
@@ -238,7 +248,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundNonLinearSepData)
countError++;
double hammingLoss = (double) countError / labels.n_cols;
BOOST_REQUIRE_LE(hammingLoss, a.ZtProduct());
// Check that ztProduct is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(ztProduct), true);
BOOST_REQUIRE_LE(hammingLoss, ztProduct);
}
/**
@@ -312,7 +324,8 @@ BOOST_AUTO_TEST_CASE(HammingLossIris_DS)
// Define parameters for AdaBoost.
size_t iterations = 50;
double tolerance = 1e-10;
AdaBoost<DecisionStump<>> a(inputData, labels.row(0), numClasses, ds,
AdaBoost<DecisionStump<>> a(tolerance);
double ztProduct = a.Train(inputData, labels.row(0), numClasses, ds,
iterations, tolerance);
arma::Row<size_t> predictedLabels;
@@ -324,7 +337,9 @@ BOOST_AUTO_TEST_CASE(HammingLossIris_DS)
countError++;
double hammingLoss = (double) countError / labels.n_cols;
BOOST_REQUIRE_LE(hammingLoss, a.ZtProduct());
// Check that ztProduct is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(ztProduct), true);
BOOST_REQUIRE_LE(hammingLoss, ztProduct);
}
/**
@@ -405,7 +420,8 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundVertebralColumn_DS)
size_t iterations = 50;
double tolerance = 1e-10;
AdaBoost<DecisionStump<>> a(inputData, labels.row(0), numClasses, ds,
AdaBoost<DecisionStump<>> a(tolerance);
double ztProduct = a.Train(inputData, labels.row(0), numClasses, ds,
iterations, tolerance);
arma::Row<size_t> predictedLabels;
@@ -417,7 +433,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundVertebralColumn_DS)
countError++;
double hammingLoss = (double) countError / labels.n_cols;
BOOST_REQUIRE_LE(hammingLoss, a.ZtProduct());
// Check that ztProduct is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(ztProduct), true);
BOOST_REQUIRE_LE(hammingLoss, ztProduct);
}
/**
@@ -494,7 +512,8 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundNonLinearSepData_DS)
size_t iterations = 50;
double tolerance = 1e-10;
AdaBoost<DecisionStump<> > a(inputData, labels.row(0), numClasses, ds,
AdaBoost<DecisionStump<>> a(tolerance);
double ztProduct = a.Train(inputData, labels.row(0), numClasses, ds,
iterations, tolerance);
arma::Row<size_t> predictedLabels;
@@ -506,7 +525,9 @@ BOOST_AUTO_TEST_CASE(HammingLossBoundNonLinearSepData_DS)
countError++;
double hammingLoss = (double) countError / labels.n_cols;
BOOST_REQUIRE_LE(hammingLoss, a.ZtProduct());
// Check that ztProduct is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(ztProduct), true);
BOOST_REQUIRE_LE(hammingLoss, ztProduct);
}
/**
@@ -805,10 +826,6 @@ BOOST_AUTO_TEST_CASE(PerceptronSerializationTest)
BOOST_REQUIRE_CLOSE(ab.Tolerance(), abText.Tolerance(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.Tolerance(), abBinary.Tolerance(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.ZtProduct(), abXml.ZtProduct(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.ZtProduct(), abText.ZtProduct(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.ZtProduct(), abBinary.ZtProduct(), 1e-5);
BOOST_REQUIRE_EQUAL(ab.WeakLearners(), abXml.WeakLearners());
BOOST_REQUIRE_EQUAL(ab.WeakLearners(), abText.WeakLearners());
BOOST_REQUIRE_EQUAL(ab.WeakLearners(), abBinary.WeakLearners());
@@ -862,10 +879,6 @@ BOOST_AUTO_TEST_CASE(DecisionStumpSerializationTest)
BOOST_REQUIRE_CLOSE(ab.Tolerance(), abText.Tolerance(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.Tolerance(), abBinary.Tolerance(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.ZtProduct(), abXml.ZtProduct(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.ZtProduct(), abText.ZtProduct(), 1e-5);
BOOST_REQUIRE_CLOSE(ab.ZtProduct(), abBinary.ZtProduct(), 1e-5);
BOOST_REQUIRE_EQUAL(ab.WeakLearners(), abXml.WeakLearners());
BOOST_REQUIRE_EQUAL(ab.WeakLearners(), abText.WeakLearners());
BOOST_REQUIRE_EQUAL(ab.WeakLearners(), abBinary.WeakLearners());
src/mlpack/tests/convolutional_network_test.cpp
+4 -1
View File
@@ -99,7 +99,10 @@ BOOST_AUTO_TEST_CASE(VanillaNetworkTest)
// Train for only 8 epochs.
ens::RMSProp opt(0.001, 1, 0.88, 1e-8, 8 * nPoints, -1);
model.Train(X, Y, opt);
double objVal = model.Train(X, Y, opt);
// Test that objective value returned by FFN::Train() is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
arma::mat predictionTemp;
model.Predict(X, predictionTemp);
src/mlpack/tests/dcgan_test.cpp
+4 -1
View File
@@ -126,7 +126,10 @@ BOOST_AUTO_TEST_CASE(DCGANMNISTTest)
discriminatorPreTrain, multiplier);
Log::Info << "Training..." << std::endl;
dcgan.Train(optimizer);
double objVal = dcgan.Train(optimizer);
// Test that objective value returned by GAN::Train() is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
// Generate samples
Log::Info << "Sampling..." << std::endl;
src/mlpack/tests/decision_stump_test.cpp
+32
View File
@@ -391,4 +391,36 @@ BOOST_AUTO_TEST_CASE(IntTest)
BOOST_CHECK_EQUAL(predictedLabels(0, 7), 2);
}
/**
* Test that DecisionStump::Train() returns finite gain.
*/
BOOST_AUTO_TEST_CASE(DecisionStumpTrainReturnEntropy)
{
const size_t numClasses = 2;
const size_t inpBucketSize = 2;
mat trainingData;
trainingData << -1 << 1 << -2 << 2 << -3 << 3;
// No need to normalize labels here.
Mat<size_t> labelsIn;
labelsIn << 0 << 1 << 0 << 1 << 0 << 1;
arma::Row<double> weights = arma::ones<arma::Row<double>>(labelsIn.n_elem);
// Train a simple decision stump without weights.
DecisionStump<> ds;
double gain = ds.Train(trainingData, labelsIn.row(0), numClasses,
inpBucketSize);
BOOST_REQUIRE_EQUAL(std::isfinite(gain), true);
// Train decision stump with weights.
DecisionStump<> wds;
gain = wds.Train(trainingData, labelsIn.row(0), weights, numClasses,
inpBucketSize);
BOOST_REQUIRE_EQUAL(std::isfinite(gain), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/decision_tree_test.cpp
+55 -3
View File
@@ -439,11 +439,11 @@ BOOST_AUTO_TEST_CASE(AllCategoricalSplitNoGainTest)
for (size_t i = 0; i < 300; i += 3)
{
values[i] = (i / 3) % 10;
values[i] = int(i / 3) % 10;
labels[i] = 0;
values[i + 1] = (i / 3) % 10;
values[i + 1] = int(i / 3) % 10;
labels[i + 1] = 1;
values[i + 2] = (i / 3) % 10;
values[i + 2] = int(i / 3) % 10;
labels[i + 2] = 2;
}
@@ -1126,4 +1126,56 @@ BOOST_AUTO_TEST_CASE(RegularisedDecisionTree)
BOOST_REQUIRE_GT(count, 0);
}
/**
* Test that DecisionTree::Train() returns finite entropy on numeric dataset.
*/
BOOST_AUTO_TEST_CASE(DecisionTreeNumericTrainReturnEntropy)
{
arma::mat dataset(10, 1000, arma::fill::randu);
arma::Row<size_t> labels(1000);
arma::rowvec weights(labels.n_elem);
weights.ones();
for (size_t i = 0; i < 1000; ++i)
labels[i] = i % 3; // 3 classes.
// Train a simpe tree on numeric dataset.
DecisionTree<> d(3);
double entropy = d.Train(dataset, labels, 3, 50);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
// Train a tree with weights on numeric dataset.
DecisionTree<> wd(3);
entropy = wd.Train(dataset, labels, 3, weights, 50);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
}
/**
* Test that DecisionTree::Train() returns finite entropy on categorical
* dataset.
*/
BOOST_AUTO_TEST_CASE(DecisionTreeCategoricalTrainReturnEntropy)
{
arma::mat d;
arma::Row<size_t> l;
data::DatasetInfo di;
MockCategoricalData(d, l, di);
arma::Row<double> weights = arma::ones<arma::Row<double>>(l.n_elem);
// Train a simple tree on categorical dataset.
DecisionTree<> dtree(5);
double entropy = dtree.Train(d, di, l, 5, 10);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
// Train a tree with weights on categorical dataset.
DecisionTree<> wdtree(5);
entropy = wdtree.Train(d, di, l, 5, weights, 10);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/distribution_test.cpp
+4 -4
View File
@@ -461,7 +461,7 @@ BOOST_AUTO_TEST_CASE(GaussianDistributionRandomTest)
// Now make sure that reflects the actual distribution.
arma::vec obsMean = arma::mean(obs, 1);
arma::mat obsCov = ccov(obs);
arma::mat obsCov = mlpack::math::ColumnCovariance(obs);
// 10% tolerance because this can be noisy.
BOOST_REQUIRE_CLOSE(obsMean[0], mean[0], 10.0);
@@ -496,7 +496,7 @@ BOOST_AUTO_TEST_CASE(GaussianDistributionTrainTest)
// Find actual mean and covariance of data.
arma::vec actualMean = arma::mean(observations, 1);
arma::mat actualCov = ccov(observations);
arma::mat actualCov = mlpack::math::ColumnCovariance(observations);
d.Train(observations);
@@ -1418,7 +1418,7 @@ BOOST_AUTO_TEST_CASE(DiagonalGaussianDistributionRandomTest)
// Make sure that reflects the actual distribution.
arma::vec obsMean = arma::mean(obs, 1);
arma::mat obsCov = arma::ccov(obs);
arma::mat obsCov = mlpack::math::ColumnCovariance(obs);
// 10% tolerance because this can be noisy.
BOOST_REQUIRE_CLOSE(obsMean(0), mean(0), 10.0);
@@ -1446,7 +1446,7 @@ BOOST_AUTO_TEST_CASE(DiagonalGaussianDistributionTrainTest)
// Calculate the actual mean and covariance of data using armadillo.
arma::vec actualMean = arma::mean(observations, 1);
arma::mat actualCov = arma::ccov(observations);
arma::mat actualCov = mlpack::math::ColumnCovariance(observations);
// Estimate the parameters.
d.Train(observations);
src/mlpack/tests/feedforward_network_test.cpp
+34
View File
@@ -579,4 +579,38 @@ BOOST_AUTO_TEST_CASE(PartialForwardTest)
CheckMatrices(output, arma::ones(10, 1) * 20);
}
/**
* Test that FFN::Train() returns finite objective value.
*/
BOOST_AUTO_TEST_CASE(FFNTrainReturnObjective)
{
// Load the dataset.
arma::mat trainData;
data::Load("thyroid_train.csv", trainData, true);
arma::mat trainLabels = trainData.row(trainData.n_rows - 1);
trainData.shed_row(trainData.n_rows - 1);
arma::mat testData;
data::Load("thyroid_test.csv", testData, true);
arma::mat testLabels = testData.row(testData.n_rows - 1);
testData.shed_row(testData.n_rows - 1);
// Vanilla neural net with logistic activation function.
// Because 92 percent of the patients are not hyperthyroid the neural
// network must be significantly better than 92%.
FFN<NegativeLogLikelihood<> > model;
model.Add<Linear<> >(trainData.n_rows, 8);
model.Add<SigmoidLayer<> >();
model.Add<Dropout<> >();
model.Add<Linear<> >(8, 3);
model.Add<LogSoftMax<> >();
ens::RMSProp opt(0.01, 32, 0.88, 1e-8, trainData.n_cols /* 1 epoch */, -1);
double objVal = model.Train(trainData, trainLabels, opt);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/gan_test.cpp
+3 -3
View File
@@ -173,7 +173,7 @@ BOOST_AUTO_TEST_CASE(GANMNISTTest)
<< trainData.n_cols << ")" << std::endl;
Log::Info << trainData.n_rows << "--------" << trainData.n_cols << std::endl;
// Create the Discriminator network
// Create the Discriminator network.
FFN<CrossEntropyError<> > discriminator;
discriminator.Add<Convolution<> >(1, dNumKernels, 5, 5, 1, 1, 2, 2, 28, 28);
discriminator.Add<ReLULayer<> >();
@@ -186,7 +186,7 @@ BOOST_AUTO_TEST_CASE(GANMNISTTest)
discriminator.Add<ReLULayer<> >();
discriminator.Add<Linear<> >(1024, 1);
// Create the Generator network
// Create the Generator network.
FFN<CrossEntropyError<> > generator;
generator.Add<Linear<> >(noiseDim, 3136);
generator.Add<BatchNorm<> >(3136);
@@ -217,7 +217,7 @@ BOOST_AUTO_TEST_CASE(GANMNISTTest)
Log::Info << "Training..." << std::endl;
gan.Train(optimizer);
// Generate samples
// Generate samples.
Log::Info << "Sampling..." << std::endl;
arma::mat noise(noiseDim, batchSize);
size_t dim = std::sqrt(trainData.n_rows);
src/mlpack/tests/gmm_test.cpp
+8 -4
View File
@@ -109,7 +109,8 @@ BOOST_AUTO_TEST_CASE(GMMTrainEMOneGaussian)
gmm.Train(data, 10);
arma::vec actualMean = arma::mean(data, 1);
arma::mat actualCovar = ccov(data, 1 /* biased estimator */);
arma::mat actualCovar = mlpack::math::ColumnCovariance(data,
1 /* biased estimator */);
// Check the model to see that it is correct.
BOOST_REQUIRE_LT(arma::norm(gmm.Component(0).Mean() - actualMean), 1e-5);
@@ -198,7 +199,8 @@ BOOST_AUTO_TEST_CASE(GMMTrainEMMultipleGaussians)
// Calculate the actual means and covariances because they will probably
// be different (this is easier to do before we shuffle the points).
means[i] = arma::mean(data.cols(point, point + counts[i] - 1), 1);
covars[i] = ccov(data.cols(point, point + counts[i] - 1), 1 /* biased */);
covars[i] = mlpack::math::ColumnCovariance(arma::mat(data.cols(point,
point + counts[i] - 1)), 1 /* biased */);
point += counts[i];
}
@@ -694,7 +696,8 @@ BOOST_AUTO_TEST_CASE(UseExistingModelTest)
// Calculate the actual means and covariances because they will probably
// be different (this is easier to do before we shuffle the points).
means[i] = arma::mean(data.cols(point, point + counts[i] - 1), 1);
covars[i] = ccov(data.cols(point, point + counts[i] - 1), 1 /* biased */);
covars[i] = mlpack::math::ColumnCovariance(arma::mat(data.cols(point,
point + counts[i] - 1)), 1 /* biased */);
point += counts[i];
}
@@ -855,7 +858,8 @@ BOOST_AUTO_TEST_CASE(DiagonalGMMTrainEMOneGaussian)
arma::vec actualMean = arma::mean(data, 1);
arma::vec actualCovar = arma::diagvec(
arma::ccov(data, 1 /* biased estimator */));
mlpack::math::ColumnCovariance(data,
1 /* biased estimator */));
// Check the model to see that it is correct.
CheckMatrices(gmm.Component(0).Mean(), actualMean);
src/mlpack/tests/hmm_test.cpp
+31 -1
View File
@@ -1229,6 +1229,35 @@ BOOST_AUTO_TEST_CASE(DiscreteHMMLoadSaveTest)
hmm2.Emission()[j].Probabilities()[i], 1e-3);
}
/**
* Test that HMM::Train() returns finite log-likelihood.
*/
BOOST_AUTO_TEST_CASE(HMMTrainReturnLogLikelihood)
{
HMM<DiscreteDistribution> hmm(1, 2); // 1 state, 2 emissions.
// Randomize the emission matrix.
hmm.Emission()[0].Probabilities() = arma::randu<arma::vec>(2);
hmm.Emission()[0].Probabilities() /= accu(hmm.Emission()[0].Probabilities());
std::vector<arma::mat> observations;
observations.push_back("0 1 0 1 0 1 0 1 0 1 0 1");
observations.push_back("0 0 0 0 0 0 1 1 1 1 1 1");
observations.push_back("1 1 1 1 1 1 0 0 0 0 0 0");
observations.push_back("1 1 1 0 0 0 1 1 1 0 0 0");
observations.push_back("0 0 1 1 0 0 0 0 1 1 1 1");
observations.push_back("1 1 1 0 0 0 1 1 1 0 0 0");
observations.push_back("0 1 0 1 0 1 0 1 0 1 0 1");
observations.push_back("0 0 0 0 0 0 1 1 1 1 1 1");
observations.push_back("1 1 1 1 1 0 1 0 0 0 0 0");
observations.push_back("1 1 1 0 0 1 0 1 1 0 0 0");
observations.push_back("0 0 1 1 0 0 0 1 0 1 1 1");
observations.push_back("1 1 1 0 0 1 0 1 1 0 0 0");
double loglik = hmm.Train(observations);
BOOST_REQUIRE_EQUAL(std::isfinite(loglik), true);
}
/********************************************/
/** DiagonalGMM Hidden Markov Models Tests **/
/********************************************/
@@ -1381,7 +1410,8 @@ BOOST_AUTO_TEST_CASE(DiagonalGMMHMMOneGaussianOneStateTrainingTest)
// Generate the ground truth values.
arma::vec actualMean = arma::mean(observations[0], 1);
arma::vec actualCovar = arma::diagvec(
arma::ccov(observations[0], 1 /* biased estimator */));
mlpack::math::ColumnCovariance(observations[0],
1 /* biased estimator */));
// Check the model to see that it is correct.
CheckMatrices(hmm.Emission()[0].Component(0).Mean(), actualMean);
src/mlpack/tests/lars_test.cpp
+45
View File
@@ -352,4 +352,49 @@ BOOST_AUTO_TEST_CASE(TrainingConstructorWithNonDefaultsTest)
BOOST_REQUIRE_CLOSE(beta[i], lars2.Beta()[i], 1e-5);
}
/**
* Test that LARS::Train() returns finite correlation value.
*/
BOOST_AUTO_TEST_CASE(LARSTrainReturnCorrelation)
{
arma::mat X;
arma::mat Y;
data::Load("lars_dependent_x.csv", X);
data::Load("lars_dependent_y.csv", Y);
arma::rowvec y = Y.row(0);
double lambda1 = 0.1;
double lambda2 = 0.1;
// Test with Cholesky decomposition and with lasso.
LARS lars1(true, lambda1, 0.0);
arma::vec betaOpt1;
double maxCorr = lars1.Train(X, y, betaOpt1);
BOOST_REQUIRE_EQUAL(std::isfinite(maxCorr), true);
// Test without Cholesky decomposition and with lasso.
LARS lars2(false, lambda1, 0.0);
arma::vec betaOpt2;
maxCorr = lars2.Train(X, y, betaOpt2);
BOOST_REQUIRE_EQUAL(std::isfinite(maxCorr), true);
// Test with Cholesky decomposition and with elasticnet.
LARS lars3(true, lambda1, lambda2);
arma::vec betaOpt3;
maxCorr = lars3.Train(X, y, betaOpt3);
BOOST_REQUIRE_EQUAL(std::isfinite(maxCorr), true);
// Test without Cholesky decomposition and with elasticnet.
LARS lars4(false, lambda1, lambda2);
arma::vec betaOpt4;
maxCorr = lars4.Train(X, y, betaOpt4);
BOOST_REQUIRE_EQUAL(std::isfinite(maxCorr), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/lin_alg_test.cpp
+2 -2
View File
@@ -89,7 +89,7 @@ BOOST_AUTO_TEST_CASE(TestWhitenUsingEig)
Center(tmp, tmp_centered);
WhitenUsingEig(tmp_centered, whitened, whitening_matrix);
mat newcov = ccov(whitened);
mat newcov = mlpack::math::ColumnCovariance(whitened);
for (int row = 0; row < 5; row++)
{
for (int col = 0; col < 5; col++)
@@ -118,7 +118,7 @@ BOOST_AUTO_TEST_CASE(TestOrthogonalize)
Orthogonalize(tmp, orth);
// test orthogonality
mat test = ccov(orth);
mat test = mlpack::math::ColumnCovariance(orth);
double ival = test(0, 0);
for (size_t row = 0; row < test.n_rows; row++)
{
src/mlpack/tests/linear_regression_test.cpp
+44
View File
@@ -224,4 +224,48 @@ BOOST_AUTO_TEST_CASE(LinearRegressionTest)
binaryLr.Parameters());
}
/**
* Test that LinearRegression::Train() returns finite OLS error.
*/
BOOST_AUTO_TEST_CASE(LinearRegressionTrainReturnObjective)
{
arma::mat predictors(3, 10);
arma::mat points(3, 10);
// Responses is the "correct" value for each point in predictors and points.
arma::rowvec responses(10);
// The values we get back when we predict for points.
arma::rowvec predictions(10);
// We'll randomly select some coefficients for the linear response.
arma::vec coeffs;
coeffs.randu(4);
// Now generate each point.
for (size_t row = 0; row < 3; row++)
predictors.row(row) = arma::linspace<arma::rowvec>(0, 9, 10);
points = predictors;
// Now add a small amount of noise to each point.
for (size_t elem = 0; elem < points.n_elem; elem++)
{
// Max added noise is 0.02.
points[elem] += math::Random() / 50.0;
predictors[elem] += math::Random() / 50.0;
}
// Generate responses.
for (size_t elem = 0; elem < responses.n_elem; elem++)
responses[elem] = coeffs[0] +
dot(coeffs.rows(1, 3), arma::ones<arma::rowvec>(3) * elem);
// Initialize and predict.
LinearRegression lr;
double error = lr.Train(predictors, responses);
BOOST_REQUIRE_EQUAL(std::isfinite(error), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/local_coordinate_coding_test.cpp
+25
View File
@@ -165,4 +165,29 @@ BOOST_AUTO_TEST_CASE(SerializationTest)
BOOST_REQUIRE_EQUAL(lcc.MaxIterations(), lccBinary.MaxIterations());
}
/**
* Test that LocalCoordinateCoding::Train() returns finite final objective
* value.
*/
BOOST_AUTO_TEST_CASE(LocalCoordinateCodingTrainReturnObjective)
{
double lambda1 = 0.1;
uword nAtoms = 10;
mat X;
X.load("mnist_first250_training_4s_and_9s.arm");
uword nPoints = X.n_cols;
// Normalize each point since these are images.
for (uword i = 0; i < nPoints; i++)
{
X.col(i) /= norm(X.col(i), 2);
}
LocalCoordinateCoding lcc(nAtoms, lambda1, 10);
double objVal = lcc.Train(X);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/logistic_regression_test.cpp
+40
View File
@@ -962,4 +962,44 @@ BOOST_AUTO_TEST_CASE(ClassifyProbabilitiesTest)
}
}
/**
* Test that LogisticRegression::Train() returns finite final objective
* value.
*/
BOOST_AUTO_TEST_CASE(LogisticRegressionTrainReturnObjective)
{
// Very simple fake dataset.
arma::mat data("1 2 3;"
"1 2 3");
arma::Row<size_t> responses("1 1 0");
// Check with L_BFGS optimizer.
LogisticRegression<> lr1(data.n_rows, 0.5);
double objVal = lr1.Train<ens::L_BFGS>(data, responses);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
// Check with a pre-defined L_BFGS optimizer.
LogisticRegression<> lr2(data.n_rows, 0.5);
ens::L_BFGS lbfgsOpt;
objVal = lr2.Train(data, responses, lbfgsOpt);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
// Check with SGD optimizer.
LogisticRegression<> lr3(data.n_rows, 0.5);
objVal = lr3.Train<ens::StandardSGD>(data, responses);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
// Check with pre-defined SGD optimizer.
LogisticRegression<> lr4(data.n_rows, 0.0005);
ens::StandardSGD sgdOpt;
sgdOpt.StepSize() = 0.15;
sgdOpt.Tolerance() = 1e-75;
objVal = lr4.Train(data, responses, sgdOpt);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/q_learning_test.cpp
+4 -4
View File
@@ -49,7 +49,7 @@ BOOST_AUTO_TEST_CASE(CartPoleWithDQN)
model.Add<Linear<>>(128, 2);
// Set up the policy and replay method.
GreedyPolicy<CartPole> policy(1.0, 1000, 0.1);
GreedyPolicy<CartPole> policy(1.0, 1000, 0.1, 0.99);
RandomReplay<CartPole> replayMethod(10, 10000);
TrainingConfig config;
@@ -122,7 +122,7 @@ BOOST_AUTO_TEST_CASE(CartPoleWithDoubleDQN)
model.Add<Linear<>>(20, 2);
// Set up the policy and replay method.
GreedyPolicy<CartPole> policy(1.0, 1000, 0.1);
GreedyPolicy<CartPole> policy(1.0, 1000, 0.1, 0.99);
RandomReplay<CartPole> replayMethod(10, 10000);
TrainingConfig config;
@@ -191,7 +191,7 @@ BOOST_AUTO_TEST_CASE(AcrobotWithDQN)
model.Add<Linear<>>(32, 3);
// Set up the policy and replay method.
GreedyPolicy<Acrobot> policy(1.0, 1000, 0.1);
GreedyPolicy<Acrobot> policy(1.0, 1000, 0.1, 0.99);
RandomReplay<Acrobot> replayMethod(20, 10000);
TrainingConfig config;
@@ -268,7 +268,7 @@ BOOST_AUTO_TEST_CASE(MountainCarWithDQN)
model.Add<Linear<>>(32, 3);
// Set up the policy and replay method.
GreedyPolicy<MountainCar> policy(1.0, 1000, 0.1);
GreedyPolicy<MountainCar> policy(1.0, 1000, 0.1, 0.99);
RandomReplay<MountainCar> replayMethod(20, 10000);
TrainingConfig config;
src/mlpack/tests/random_forest_test.cpp
+90 -2
View File
@@ -281,8 +281,8 @@ BOOST_AUTO_TEST_CASE(WeightedCategoricalLearningTest)
arma::Row<size_t> fullLabels = arma::join_rows(trainingLabels, randomLabels);
// Build a random forest and a decision tree.
RandomForest<> rf(fullData, di, fullLabels, 5, 15 /* 15 trees */, 5);
DecisionTree<> dt(fullData, di, fullLabels, 5, 5);
RandomForest<> rf(fullData, di, fullLabels, 5, weights, 15 /* 15 trees */, 5);
DecisionTree<> dt(fullData, di, fullLabels, 5, weights, 5);
// Get performance statistics on test data.
arma::Row<size_t> rfPredictions;
@@ -395,4 +395,92 @@ BOOST_AUTO_TEST_CASE(SerializationTest)
binaryProbabilities);
}
/**
* Test that RandomForest::Train() returns finite average entropy on numeric
* dataset.
*/
BOOST_AUTO_TEST_CASE(RandomForestNumericTrainReturnEntropy)
{
arma::mat dataset;
arma::Row<size_t> labels;
data::Load("vc2.csv", dataset);
data::Load("vc2_labels.txt", labels);
// Add some noise.
arma::mat noise(dataset.n_rows, 1000, arma::fill::randu);
arma::Row<size_t> noiseLabels(1000);
for (size_t i = 0; i < noiseLabels.n_elem; ++i)
noiseLabels[i] = math::RandInt(3); // Random label.
// Concatenate data matrices.
arma::mat data = arma::join_rows(dataset, noise);
arma::Row<size_t> fullLabels = arma::join_rows(labels, noiseLabels);
// Now set weights.
arma::rowvec weights(dataset.n_cols + 1000);
for (size_t i = 0; i < dataset.n_cols; ++i)
weights[i] = math::Random(0.9, 1.0);
for (size_t i = dataset.n_cols; i < dataset.n_cols + 1000; ++i)
weights[i] = math::Random(0.0, 0.01); // Low weights for false points.
// Test random forest on unweighted numeric dataset.
RandomForest<GiniGain, RandomDimensionSelect> rf;
double entropy = rf.Train(dataset, labels, 3, 10, 5);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
// Test random forest on weighted numeric dataset.
RandomForest<GiniGain, RandomDimensionSelect> wrf;
entropy = wrf.Train(dataset, labels, 3, weights, 10, 5);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
}
/**
* Test that RandomForest::Train() returns finite average entropy on categorical
* dataset.
*/
BOOST_AUTO_TEST_CASE(RandomForestCategoricalTrainReturnEntropy)
{
arma::mat d;
arma::Row<size_t> l;
data::DatasetInfo di;
MockCategoricalData(d, l, di);
// Now create random points.
arma::mat randomNoise(4, 2000);
arma::Row<size_t> randomLabels(2000);
for (size_t i = 0; i < 2000; ++i)
{
randomNoise(0, i) = math::Random();
randomNoise(1, i) = math::Random();
randomNoise(2, i) = math::RandInt(4);
randomNoise(3, i) = math::RandInt(2);
randomLabels[i] = math::RandInt(5);
}
// Generate weights.
arma::rowvec weights(6000);
for (size_t i = 0; i < 4000; ++i)
weights[i] = math::Random(0.9, 1.0);
for (size_t i = 4000; i < 6000; ++i)
weights[i] = math::Random(0.0, 0.001);
arma::mat fullData = arma::join_rows(d, randomNoise);
arma::Row<size_t> fullLabels = arma::join_rows(l, randomLabels);
// Test random forest on unweighted categorical dataset.
RandomForest<> rf;
double entropy = rf.Train(fullData, di, fullLabels, 5, 15 /* 15 trees */, 5);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
// Test random forest on weighted categorical dataset.
RandomForest<> wrf;
entropy = wrf.Train(fullData, di, fullLabels, 5, weights, 15 /* 15 trees */,
5);
BOOST_REQUIRE_EQUAL(std::isfinite(entropy), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/rbm_network_test.cpp
+9 -2
View File
@@ -81,7 +81,10 @@ BOOST_AUTO_TEST_CASE(BinaryRBMClassificationTest)
model.HiddenBias().ones();
// Test the reset function.
model.Train(msgd);
double objVal = model.Train(msgd);
// Test that objective value returned by RBM::Train() is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
for (size_t i = 0; i < trainData.n_cols; i++)
{
@@ -179,7 +182,11 @@ BOOST_AUTO_TEST_CASE(ssRBMClassificationTest)
modelssRBM.VisiblePenalty().fill(5);
modelssRBM.SpikeBias().fill(1);
modelssRBM.Train(msgd);
double objVal = modelssRBM.Train(msgd);
// Test that objective value returned by RBM::Train() is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
for (size_t i = 0; i < trainData.n_cols; i++)
{
modelssRBM.HiddenMean(std::move(trainData.col(i)),
src/mlpack/tests/recurrent_network_test.cpp
+95
View File
@@ -1318,4 +1318,99 @@ BOOST_AUTO_TEST_CASE(MultiTimestepTest)
BOOST_REQUIRE_LE(err, 0.025);
}
/**
* Test that RNN::Train() returns finite objective value.
*/
BOOST_AUTO_TEST_CASE(RNNTrainReturnObjective)
{
const size_t rho = 10;
// Generate 12 (2 * 6) noisy sines. A single sine contains rho
// points/features.
arma::cube input;
arma::mat labelsTemp;
GenerateNoisySines(input, labelsTemp, rho, 6);
arma::cube labels = arma::zeros<arma::cube>(1, labelsTemp.n_cols, rho);
for (size_t i = 0; i < labelsTemp.n_cols; ++i)
{
const int value = arma::as_scalar(arma::find(
arma::max(labelsTemp.col(i)) == labelsTemp.col(i), 1)) + 1;
labels.tube(0, i).fill(value);
}
/**
* Construct a network with 1 input unit, 4 hidden units and 10 output
* units. The hidden layer is connected to itself. The network structure
* looks like:
*
* Input Hidden Output
* Layer(1) Layer(4) Layer(10)
* +-----+ +-----+ +-----+
* | | | | | |
* | +------>| +------>| |
* | | ..>| | | |
* +-----+ . +--+--+ +-----+
* . .
* . .
* .......
*/
Add<> add(4);
Linear<> lookup(1, 4);
SigmoidLayer<> sigmoidLayer;
Linear<> linear(4, 4);
Recurrent<>* recurrent = new Recurrent<>(add, lookup, linear,
sigmoidLayer, rho);
RNN<> model(rho);
model.Add<IdentityLayer<> >();
model.Add(recurrent);
model.Add<Linear<> >(4, 10);
model.Add<LogSoftMax<> >();
StandardSGD opt(0.1, 1, input.n_cols /* 1 epoch */, -100);
double objVal = model.Train(input, labels, opt);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
}
/**
* Test that BRNN::Train() returns finite objective value.
*/
BOOST_AUTO_TEST_CASE(BRNNTrainReturnObjective)
{
const size_t rho = 10;
arma::cube input;
arma::mat labelsTemp;
GenerateNoisySines(input, labelsTemp, rho, 6);
arma::cube labels = arma::zeros<arma::cube>(1, labelsTemp.n_cols, rho);
for (size_t i = 0; i < labelsTemp.n_cols; ++i)
{
const int value = arma::as_scalar(arma::find(
arma::max(labelsTemp.col(i)) == labelsTemp.col(i), 1)) + 1;
labels.tube(0, i).fill(value);
}
Add<> add(4);
Linear<> lookup(1, 4);
SigmoidLayer<> sigmoidLayer;
Linear<> linear(4, 4);
Recurrent<>* recurrent = new Recurrent<>(
add, lookup, linear, sigmoidLayer, rho);
BRNN<> model(rho);
model.Add<IdentityLayer<> >();
model.Add(recurrent);
model.Add<Linear<> >(4, 5);
StandardSGD opt(0.1, 1, 500 * input.n_cols, -100);
double objVal = model.Train(input, labels, opt);
BOOST_TEST_CHECKPOINT("Training over");
// Test that BRNN::Train() returns finite objective value.
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/reward_clipping_test.cpp
+1 -1
View File
@@ -67,7 +67,7 @@ BOOST_AUTO_TEST_CASE(RewardClippedAcrobotWithDQN)
model.Add<Linear<>>(32, 3);
// Set up the policy and replay method.
GreedyPolicy<RewardClipping<Acrobot>> policy(1.0, 1000, 0.1);
GreedyPolicy<RewardClipping<Acrobot>> policy(1.0, 1000, 0.1, 0.99);
RandomReplay<RewardClipping<Acrobot>> replayMethod(20, 10000);
// Set up Acrobot task and reward clipping wrapper
src/mlpack/tests/rl_components_test.cpp
+2 -2
View File
@@ -50,7 +50,7 @@ BOOST_AUTO_TEST_CASE(SimplePendulumTest)
}
/**
* Constructs a Continuous MountainCar instance and check if the main rountine
* Constructs a Continuous MountainCar instance and check if the main rountine
* works as it should be.
*/
BOOST_AUTO_TEST_CASE(SimpleContinuousMountainCarTest)
@@ -172,7 +172,7 @@ BOOST_AUTO_TEST_CASE(RandomReplayTest)
*/
BOOST_AUTO_TEST_CASE(GreedyPolicyTest)
{
GreedyPolicy<CartPole> policy(1.0, 10, 0.0);
GreedyPolicy<CartPole> policy(1.0, 10, 0.0, 0.99);
for (size_t i = 0; i < 15; ++i)
policy.Anneal();
BOOST_REQUIRE_CLOSE(0.0, policy.Epsilon(), 1e-5);
src/mlpack/tests/sparse_coding_test.cpp
+23
View File
@@ -189,5 +189,28 @@ BOOST_AUTO_TEST_CASE(SerializationTest)
BOOST_REQUIRE_CLOSE(sc.NewtonTolerance(), scBinary.NewtonTolerance(), 1e-5);
}
/**
* Test that SparseCoding::Train() returns finite final objective value.
*/
BOOST_AUTO_TEST_CASE(SparseCodingTrainReturnObjective)
{
const double tol = 1e-6;
double lambda1 = 0.1;
uword nAtoms = 25;
mat X;
X.load("mnist_first250_training_4s_and_9s.arm");
uword nPoints = X.n_cols;
// Normalize each point since these are images.
for (uword i = 0; i < nPoints; ++i)
X.col(i) /= norm(X.col(i), 2);
SparseCoding sc(nAtoms, lambda1, 0.0, 0, 0.01, tol);
double objVal = sc.Train(X);
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
}
BOOST_AUTO_TEST_SUITE_END();
src/mlpack/tests/wgan_test.cpp
+8 -2
View File
@@ -127,7 +127,10 @@ BOOST_AUTO_TEST_CASE(WGANMNISTTest)
discriminatorPreTrain, multiplier, clippingParameter);
Log::Info << "Training..." << std::endl;
wgan.Train(optimizer);
double objVal = wgan.Train(optimizer);
// Test that objective value returned by GAN::Train() is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
// Generate samples
Log::Info << "Sampling..." << std::endl;
@@ -255,7 +258,10 @@ BOOST_AUTO_TEST_CASE(WGANGPMNISTTest)
lambda);
Log::Info << "Training..." << std::endl;
wganGP.Train(optimizer);
double objVal = wganGP.Train(optimizer);
// Test that objective value returned by GAN::Train() is finite.
BOOST_REQUIRE_EQUAL(std::isfinite(objVal), true);
// Generate samples
Log::Info << "Sampling..." << std::endl;