178 lines
17 KiB
HTML
178 lines
17 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
|
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
|
|
<meta name="generator" content="Doxygen 1.8.11"/>
|
|
<title>mlpack: Alternating Matrix Factorization tutorial.</title>
|
|
<link href="tabs.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="jquery.js"></script>
|
|
<script type="text/javascript" src="dynsections.js"></script>
|
|
<link href="search/search.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="search/searchdata.js"></script>
|
|
<script type="text/javascript" src="search/search.js"></script>
|
|
<script type="text/javascript">
|
|
$(document).ready(function() { init_search(); });
|
|
</script>
|
|
<link href="doxygen.css" rel="stylesheet" type="text/css" />
|
|
<link href="extra-stylesheet.css" rel="stylesheet" type="text/css"/>
|
|
</head>
|
|
<body>
|
|
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
|
<div id="titlearea">
|
|
<table cellspacing="0" cellpadding="0">
|
|
<tbody>
|
|
<tr style="height: 56px;">
|
|
<td id="projectalign" style="padding-left: 0.5em;">
|
|
<div id="projectname">mlpack
|
|
 <span id="projectnumber">master</span>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<!-- end header part -->
|
|
<!-- Generated by Doxygen 1.8.11 -->
|
|
<script type="text/javascript">
|
|
var searchBox = new SearchBox("searchBox", "search",false,'Search');
|
|
</script>
|
|
<div id="navrow1" class="tabs">
|
|
<ul class="tablist">
|
|
<li><a href="index.html"><span>Main Page</span></a></li>
|
|
<li class="current"><a href="pages.html"><span>Related Pages</span></a></li>
|
|
<li><a href="namespaces.html"><span>Namespaces</span></a></li>
|
|
<li><a href="annotated.html"><span>Classes</span></a></li>
|
|
<li><a href="files.html"><span>Files</span></a></li>
|
|
<li>
|
|
<div id="MSearchBox" class="MSearchBoxInactive">
|
|
<span class="left">
|
|
<img id="MSearchSelect" src="search/mag_sel.png"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
alt=""/>
|
|
<input type="text" id="MSearchField" value="Search" accesskey="S"
|
|
onfocus="searchBox.OnSearchFieldFocus(true)"
|
|
onblur="searchBox.OnSearchFieldFocus(false)"
|
|
onkeyup="searchBox.OnSearchFieldChange(event)"/>
|
|
</span><span class="right">
|
|
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
|
|
</span>
|
|
</div>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
<!-- window showing the filter options -->
|
|
<div id="MSearchSelectWindow"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
|
</div>
|
|
|
|
<!-- iframe showing the search results (closed by default) -->
|
|
<div id="MSearchResultsWindow">
|
|
<iframe src="javascript:void(0)" frameborder="0"
|
|
name="MSearchResults" id="MSearchResults">
|
|
</iframe>
|
|
</div>
|
|
|
|
</div><!-- top -->
|
|
<div class="header">
|
|
<div class="headertitle">
|
|
<div class="title">Alternating Matrix Factorization tutorial. </div> </div>
|
|
</div><!--header-->
|
|
<div class="contents">
|
|
<div class="textblock"><h1><a class="anchor" id="intro_amftut"></a>
|
|
Introduction</h1>
|
|
<p>Alternating Matrix Factorization</p>
|
|
<p>Alternating matrix factorization decomposes matrx V in the form <img class="formulaInl" alt="$ V \approx WH $" src="form_58.png"/> where W is called the basis matrix and H is called the encoding matrix. V is taken to be of size n x m and the obtained W is n x r and H is r x m. The size r is called the rank of the factorization. Factorization is done by alternately calculating W and H respectively while holding the other matrix constant.</p>
|
|
<p><b>mlpack</b> provides:</p>
|
|
<ul>
|
|
<li>a <a class="el" href="amftutorial.html#amf_amftut">simple C++ interface</a> to perform Alternating Matrix Factorization</li>
|
|
</ul>
|
|
<h1><a class="anchor" id="toc_amftut"></a>
|
|
Table of Contents</h1>
|
|
<p>A list of all the sections this tutorial contains.</p>
|
|
<ul>
|
|
<li><a class="el" href="amftutorial.html#intro_amftut">Introduction</a></li>
|
|
<li><a class="el" href="amftutorial.html#toc_amftut">Table of Contents</a></li>
|
|
<li><a class="el" href="amftutorial.html#amf_amftut">The 'AMF' class</a><ul>
|
|
<li><a class="el" href="amftutorial.html#t_policy_amftut">Using different termination policies</a></li>
|
|
<li><a class="el" href="amftutorial.html#init_rule_amftut">Using different initialization policies</a></li>
|
|
<li><a class="el" href="amftutorial.html#update_rule_amftut">Using different update rules</a></li>
|
|
<li><a class="el" href="amftutorial.html#nmf_amftut">Using Non-Negative Matrix Factorization with AMF</a></li>
|
|
<li><a class="el" href="amftutorial.html#svd_amftut">Using Singular Value Decomposition with AMF</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a class="el" href="amftutorial.html#further_doc_amftut">Further documentation</a></li>
|
|
</ul>
|
|
<h1><a class="anchor" id="amf_amftut"></a>
|
|
The 'AMF' class</h1>
|
|
<p>The AMF class is templatized with 3 parameters; the first contains the policy used to determine when the algorithm has converged; the second contains the initialization rule for the W and H matrix; the last contains the update rule to be used during each iteration. This templatization allows the user to try various update rules, initialization rules, and termination policies (including ones not supplied with mlpack) for factorization.</p>
|
|
<p>The class provides the following method that performs factorization </p><div class="fragment"><div class="line"><span class="keyword">template</span><<span class="keyword">typename</span> MatType> <span class="keywordtype">double</span> Apply(<span class="keyword">const</span> MatType& V,</div><div class="line"> <span class="keyword">const</span> <span class="keywordtype">size_t</span> r,</div><div class="line"> arma::mat& W,</div><div class="line"> arma::mat& H);</div></div><!-- fragment --><h2><a class="anchor" id="t_policy_amftut"></a>
|
|
Using different termination policies</h2>
|
|
<p>The AMF implementation comes with different termination policies to support many implemented algorithms. Every termination policy implements the following method which returns the status of convergence. </p><div class="fragment"><div class="line"><span class="keywordtype">bool</span> IsConverged(arma::mat& W, arma::mat& H)</div></div><!-- fragment --><p>list of all the termination policies</p>
|
|
<ul>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1SimpleResidueTermination.html">mlpack::amf::SimpleResidueTermination</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1SimpleToleranceTermination.html">mlpack::amf::SimpleToleranceTermination</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1ValidationRMSETermination.html">mlpack::amf::ValidationRMSETermination</a></li>
|
|
</ul>
|
|
<p>In SimpleResidueTermination, termination decision depends on two factors, value of residue and number of iteration. If the current value of residue drops below the threshold or the number of iterations goes beyond the threshold, positive termination signal is passed to AMF.</p>
|
|
<p>In SimpleToleranceTermination, termination criterion is met when increase in residue value drops below the given tolerance. To accommodate spikes, certain number of successive residue drops are accepted. Secondary termination criterion terminates algorithm when iteration count goes beyond the threshold.</p>
|
|
<p>ValidationRMSETermination divids the data into 2 sets, training set and validation set. Entries of validation set are nullifed in the input matrix. Termination criterion is met when increase in validation set RMSe value drops below the given tolerance. To accommodate spikes certain number of successive validation RMSE drops are accepted. This upper imit on successive drops can be adjusted with reverseStepCount. Secondary termination criterion terminates algorithm when iteration count goes above the threshold. Though this termination policy is better measure of convergence than the above 2 termination policies, it may cause a overhead in performance.</p>
|
|
<p>On the other hand <a class="el" href="classmlpack_1_1amf_1_1CompleteIncrementalTermination.html">CompleteIncrementalTermination</a> and <a class="el" href="classmlpack_1_1amf_1_1IncompleteIncrementalTermination.html">mlpack::amf::IncompleteIncrementalTermination</a> are just wrapper classes for other termination policies. These policies are used when AMF is applied with <a class="el" href="classmlpack_1_1amf_1_1SVDCompleteIncrementalLearning.html">SVDCompleteIncrementalLearning</a> and <a class="el" href="classmlpack_1_1amf_1_1SVDIncompleteIncrementalLearning.html">SVDIncompleteIncrementalLearning</a> respectively.</p>
|
|
<h2><a class="anchor" id="init_rule_amftut"></a>
|
|
Using different initialization policies</h2>
|
|
<p>The AMF class comes with 2 initialization policies</p><ul>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1RandomInitialization.html">RandomInitialization</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1RandomAcolInitialization.html">RandomAcolInitialization</a></li>
|
|
</ul>
|
|
<p>RandomInitialization initializes matrices W and H with random uniform distribution while RandomAcolInitialization initializes the W matrix by averaging p randomly chosen columns of V. In case of RandomAcolInitialization, p is a template parameter.</p>
|
|
<p>To implement their own initialization policy, users need to define the following function in their class. </p><div class="fragment"><div class="line"><span class="keyword">template</span><<span class="keyword">typename</span> MatType></div><div class="line"><span class="keyword">inline</span> <span class="keyword">static</span> <span class="keywordtype">void</span> Initialize(<span class="keyword">const</span> MatType& V,</div><div class="line"> <span class="keyword">const</span> <span class="keywordtype">size_t</span> r,</div><div class="line"> arma::mat& W,</div><div class="line"> arma::mat& H)</div></div><!-- fragment --><h2><a class="anchor" id="update_rule_amftut"></a>
|
|
Using different update rules</h2>
|
|
<p>AMF supports following update rules</p><ul>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1NMFALSUpdate.html">AMFALSUpdate</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1NMFMultiplicativeDistanceUpdate.html">NMFMultiplicativeDistanceUpdate</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1NMFMultiplicativeDivergenceUpdate.html">NMFMultiplicativeDivergenceUpdate</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1SVDBatchLearning.html">SVDBatchLearning</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1SVDIncompleteIncrementalLearning.html">SVDIncompleteIncrementalLearning</a></li>
|
|
<li><a class="el" href="classmlpack_1_1amf_1_1SVDCompleteIncrementalLearning.html">SVDCompleteIncrementalLearning</a></li>
|
|
</ul>
|
|
<p>Non-Negative Matrix factorization can be achieved with NMFALSUpdate, NMFMultiplicativeDivergenceUpdate or NMFMultiplicativeDivergenceUpdate. NMFALSUpdate implements simple Alternating Least Square optimization while the other rules implement algorithms given in paper 'Algorithms for Non-negative Matrix Factorization'.</p>
|
|
<p>The remaining update rules perform Singular Value Decomposition of matrix V. This SVD factorization is optimized for the use by Collaborative Filtering. This use of SVD factorizers for Collaborative Filtering is described in the paper 'A Guide to singular Value Decomposition' by Chih-Chao Ma. For further details about the algorithms refer to the respective class documentation.</p>
|
|
<h2><a class="anchor" id="nmf_amftut"></a>
|
|
Using Non-Negative Matrix Factorization with AMF</h2>
|
|
<p>The use of AMF for Non-Negative Matrix factorization is simple. The AMF module defines <a class="el" href="namespacemlpack_1_1amf.html#a28055143d86e284abd74bd150ec38c52">NMFALSFactorizer</a> which can be used directly without knowing the internal structure of AMF. For example -</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <iostream></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="core_8hpp.html">mlpack/core.hpp</a>></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="amf_8hpp.html">mlpack/methods/amf/amf.hpp</a>></span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacestd.html">std</a>;</div><div class="line"><span class="keyword">using namespace </span>arma;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1amf.html">mlpack::amf</a>;</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main()</div><div class="line">{</div><div class="line"> <a class="code" href="classmlpack_1_1amf_1_1AMF.html">NMFALSFactorizer</a> nmf;</div><div class="line"> mat W, H;</div><div class="line"> mat V = randu<mat>(100, 100);</div><div class="line"> <span class="keywordtype">double</span> residue = nmf.<a class="code" href="classmlpack_1_1amf_1_1AMF.html#a0d9ccd51c9d59f363cfc726e93778d4e">Apply</a>(V, W, H);</div><div class="line"> <span class="keywordflow">return</span> 1;</div><div class="line">}</div></div><!-- fragment --><p>NMFALSFactorizer uses SimpleResidueTermination which is most preferred with Non-Negative Matrix factorizers. Initialization of W and H in NMFALSFactorizer is random. The Apply function returns the residue obtained by comparing the constructed matrix W * H with the original matrix V.</p>
|
|
<h2><a class="anchor" id="svd_amftut"></a>
|
|
Using Singular Value Decomposition with AMF</h2>
|
|
<p>AMF implementation supports following SVD factorizers</p><ul>
|
|
<li><a class="el" href="namespacemlpack_1_1amf.html#a0a89e9140ca599a10ead8ba2dacbc38b">SVDBatchFactorizer</a></li>
|
|
<li><a class="el" href="namespacemlpack_1_1amf.html#a2c91f0f4499dbc56a5d31de8fea0f199">SparseSVDBatchFactorizer</a></li>
|
|
<li><a class="el" href="namespacemlpack_1_1amf.html#adbb3b940b48c9f2996a825d66ef1a302">SVDIncompleteIncrementalFactorizer</a></li>
|
|
<li><a class="el" href="namespacemlpack_1_1amf.html#a58095f1a71997e80942cb56f09230fc6">SparseSVDIncompleteIncrementalFactorizer</a></li>
|
|
<li><a class="el" href="namespacemlpack_1_1amf.html#a2cd6a9e37e9e0942a335fcc60f853722">SVDCompleteIncrementalFactorizer</a></li>
|
|
<li><a class="el" href="namespacemlpack_1_1amf.html#a5b82f9723a02ddca746cdc19e588708a">SparseSVDCompleteIncrementalFactorizer</a></li>
|
|
</ul>
|
|
<p>The sparse version of factorizers can be used with Armadillo's sparse matrix support. These specialized implementations boost runtime performance when the matrix to be factorized is relatively sparse.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="core_8hpp.html">mlpack/core.hpp</a>></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="amf_8hpp.html">mlpack/methods/amf/amf.hpp</a>></span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacestd.html">std</a>;</div><div class="line"><span class="keyword">using namespace </span>arma;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1amf.html">mlpack::amf</a>;</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main()</div><div class="line">{</div><div class="line"> sp_mat V = randu<sp_mat>(100,100);</div><div class="line"> mat W, H;</div><div class="line"></div><div class="line"> <a class="code" href="classmlpack_1_1amf_1_1AMF.html">SparseSVDBatchFactorizer</a> svd;</div><div class="line"> <span class="keywordtype">double</span> residue = svd.<a class="code" href="classmlpack_1_1amf_1_1AMF.html#a0d9ccd51c9d59f363cfc726e93778d4e">Apply</a>(V, W, H);</div><div class="line">}</div></div><!-- fragment --><h1><a class="anchor" id="further_doc_amftut"></a>
|
|
Further documentation</h1>
|
|
<p>For further documentation on the AMF class, consult the <a class="el" href="classmlpack_1_1amf_1_1AMF.html">complete API documentation</a>. </p>
|
|
</div></div><!-- contents -->
|
|
<!-- start footer part -->
|
|
<hr class="footer"/><address class="footer"><small>
|
|
Generated by  <a href="http://www.doxygen.org/index.html">
|
|
<img class="footer" src="doxygen.png" alt="doxygen"/>
|
|
</a> 1.8.11
|
|
</small></address>
|
|
</body>
|
|
<script type="text/javascript">
|
|
var x = document.getElementsByClassName("formulaDsp");
|
|
var i;
|
|
for (i = 0; i < x.length; i++)
|
|
{
|
|
x[i].width /= 4;
|
|
}
|
|
</script>
|
|
</html>
|