Applied more style improvements

This commit is contained in:
Sourabh Varshney
2017-12-13 16:19:25 +05:30
parent 3633cd3cac
commit fc45099367
3 changed files with 18 additions and 14 deletions
+9 -8
View File
@@ -2,7 +2,7 @@
* @file nadam.hpp
* @author Sourabh Varshney
*
* Nadam optimizer. Nadam is an optimizer that combines the effect of Adam and
* Nadam optimizer. Nadam is an optimizer that combines the effect of Adam and
* NAG to the gradient descent to improve its Performance.
*
* mlpack is free software; you may redistribute it and/or modify it under the
@@ -51,11 +51,12 @@ namespace optimization {
*
* NumFunctions() should return the number of functions (\f$n\f$), and in the
* other two functions, the parameter i refers to which individual function (or
* gradient) is being evaluated. So, for the case of a data-dependent function,
* such as NCA (see mlpack::nca::NCA), NumFunctions() should return the number
* of points in the dataset, and Evaluate(coordinates, 0) will evaluate the
* objective function on the first point in the dataset (presumably, the dataset
* is held internally in the DecomposableFunctionType).
* gradient) is being evaluated. So, for the case of a data-dependent
* function, such as NCA (see mlpack::nca::NCA), NumFunctions() should return
* the number of points in the dataset, and Evaluate(coordinates, 0) will
* evaluate the objective function on the first point in the dataset
* (presumably, the dataset is held internally in the
* DecomposableFunctionType).
*
* @tparam UpdateRule Nadam optimizer update rule to be used.
*/
@@ -68,8 +69,8 @@ class NadamType
* defaults here are not necessarily good for the given problem, so it is
* suggested that the values used be tailored to the task at hand. The
* maximum number of iterations refers to the maximum number of points that
* are processed (i.e., one iteration equals one point; one iteration does not
* equal one pass over the dataset).
* are processed (i.e., one iteration equals one point; one iteration does
* not equal one pass over the dataset).
*
* @param stepSize Step size for each iteration.
* @param batchSize Number of points to process in a single step.
@@ -43,15 +43,15 @@ class NadamUpdate
* parameter.
* @param beta1 The smoothing parameter.
*/
NadamUpdate(const double epsilon = 1e-8,const double beta1 = 0.9)
NadamUpdate(const double epsilon = 1e-8,const double beta1 = 0.9)
:epsilon(epsilon),beta1(beta1),iteration(0)
{
// Nothing to do.
}
/**
* The Initialize method is called by SGD Optimizer method before the start of
* the iteration update process.
* The Initialize method is called by SGD Optimizer method before the start
* of the iteration update process.
*
* @param rows Number of rows in the gradient matrix.
* @param cols Number of columns in the gradient matrix.
@@ -69,7 +69,8 @@ class NadamUpdate
* @param stepSize Step size to be used for the given iteration.
* @param gradient The gradient matrix.
*/
void Update(arma::mat& iterate,const double stepSize,const arma::mat& gradient)
void Update(arma::mat& iterate, const double stepSize,
const arma::mat& gradient)
{
// Increment the iteration counter variable.
++iteration;
+4 -2
View File
@@ -52,8 +52,10 @@ BOOST_AUTO_TEST_CASE(SimpleNadamTestFunction)
BOOST_AUTO_TEST_CASE(NadamLogisticRegressionTest)
{
// Generate a two-Gaussian dataset.
GaussianDistribution g1(arma::vec("1.0 1.0 1.0"), arma::eye<arma::mat>(3, 3));
GaussianDistribution g2(arma::vec("9.0 9.0 9.0"), arma::eye<arma::mat>(3, 3));
GaussianDistribution g1(arma::vec("1.0 1.0 1.0"),
arma::eye<arma::mat>(3, 3));
GaussianDistribution g2(arma::vec("9.0 9.0 9.0"),
arma::eye<arma::mat>(3, 3));
arma::mat data(3, 1000);
arma::Row<size_t> responses(1000);