156 lines
16 KiB
HTML
156 lines
16 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
|
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
|
|
<meta name="generator" content="Doxygen 1.8.11"/>
|
|
<title>mlpack: The KernelType policy in mlpack</title>
|
|
<link href="tabs.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="jquery.js"></script>
|
|
<script type="text/javascript" src="dynsections.js"></script>
|
|
<link href="search/search.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="search/searchdata.js"></script>
|
|
<script type="text/javascript" src="search/search.js"></script>
|
|
<script type="text/javascript">
|
|
$(document).ready(function() { init_search(); });
|
|
</script>
|
|
<link href="doxygen.css" rel="stylesheet" type="text/css" />
|
|
<link href="extra-stylesheet.css" rel="stylesheet" type="text/css"/>
|
|
</head>
|
|
<body>
|
|
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
|
<div id="titlearea">
|
|
<table cellspacing="0" cellpadding="0">
|
|
<tbody>
|
|
<tr style="height: 56px;">
|
|
<td id="projectalign" style="padding-left: 0.5em;">
|
|
<div id="projectname">mlpack
|
|
 <span id="projectnumber">master</span>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<!-- end header part -->
|
|
<!-- Generated by Doxygen 1.8.11 -->
|
|
<script type="text/javascript">
|
|
var searchBox = new SearchBox("searchBox", "search",false,'Search');
|
|
</script>
|
|
<div id="navrow1" class="tabs">
|
|
<ul class="tablist">
|
|
<li><a href="index.html"><span>Main Page</span></a></li>
|
|
<li class="current"><a href="pages.html"><span>Related Pages</span></a></li>
|
|
<li><a href="namespaces.html"><span>Namespaces</span></a></li>
|
|
<li><a href="annotated.html"><span>Classes</span></a></li>
|
|
<li><a href="files.html"><span>Files</span></a></li>
|
|
<li>
|
|
<div id="MSearchBox" class="MSearchBoxInactive">
|
|
<span class="left">
|
|
<img id="MSearchSelect" src="search/mag_sel.png"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
alt=""/>
|
|
<input type="text" id="MSearchField" value="Search" accesskey="S"
|
|
onfocus="searchBox.OnSearchFieldFocus(true)"
|
|
onblur="searchBox.OnSearchFieldFocus(false)"
|
|
onkeyup="searchBox.OnSearchFieldChange(event)"/>
|
|
</span><span class="right">
|
|
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
|
|
</span>
|
|
</div>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
<!-- window showing the filter options -->
|
|
<div id="MSearchSelectWindow"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
|
</div>
|
|
|
|
<!-- iframe showing the search results (closed by default) -->
|
|
<div id="MSearchResultsWindow">
|
|
<iframe src="javascript:void(0)" frameborder="0"
|
|
name="MSearchResults" id="MSearchResults">
|
|
</iframe>
|
|
</div>
|
|
|
|
</div><!-- top -->
|
|
<div class="header">
|
|
<div class="headertitle">
|
|
<div class="title">The KernelType policy in mlpack </div> </div>
|
|
</div><!--header-->
|
|
<div class="contents">
|
|
<div class="textblock"><h1><a class="anchor" id="kerneltoc"></a>
|
|
Table of Contents</h1>
|
|
<ul>
|
|
<li><a class="el" href="kernels.html#kerneltype">Introduction to the KernelType policy</a></li>
|
|
<li><a class="el" href="kernels.html#kerneltraits">The KernelTraits trait class</a></li>
|
|
<li><a class="el" href="kernels.html#kernellist">List of kernels and classes that use a <code>KernelType</code> </a></li>
|
|
</ul>
|
|
<h1><a class="anchor" id="kerneltype"></a>
|
|
Introduction to the KernelType policy</h1>
|
|
<p>`Kernel methods' make up a large class of machine learning techniques. Each of these methods is characterized by its dependence on a <b>kernel</b> <b>function</b>. In rough terms, a kernel function is a general notion of similarity between two points, with its value large when objects are similar and its value small when objects are dissimilar (note that this is not the only interpretation of what a kernel is).</p>
|
|
<p>A kernel (or `Mercer kernel') <img class="formulaInl" alt="$\mathcal{K}(\cdot, \cdot)$" src="form_156.png"/> takes two objects as input and returns some sort of similarity value. The specific details and properties of kernels are outside the scope of this documentation; for a better introduction to kernels and kernel methods, there are numerous better resources available, including http://www.eric-kim.net/eric-kim-net/posts/1/kernel_trick.html "Eric Kim's
|
|
tutorial".</p>
|
|
<p>mlpack implements a number of kernel methods and, accordingly, each of these methods allows arbitrary kernels to be used via the <code>KernelType</code> template parameter. Like the <a class="el" href="metrics.html">MetricType policy</a>, the requirements are quite simple: a class implementing the <code>KernelType</code> policy must have</p>
|
|
<ul>
|
|
<li>an <code>Evaluate()</code> function</li>
|
|
<li>a default constructor</li>
|
|
</ul>
|
|
<p>The signature of the <code>Evaluate()</code> function is straightforward:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">template</span><<span class="keyword">typename</span> VecTypeA, <span class="keyword">typename</span> VecTypeB></div><div class="line"><span class="keywordtype">double</span> Evaluate(<span class="keyword">const</span> VecTypeA& a, <span class="keyword">const</span> VecTypeB& b);</div></div><!-- fragment --><p>The function takes two vector arguments, <code>a</code> and <code>b</code>, and returns a <code>double</code> that is the evaluation of the kernel between the two arguments. So, for a particular kernel <img class="formulaInl" alt="$\mathcal{K}(\cdot, \cdot)$" src="form_156.png"/>, the <code>Evaluate()</code> function should return <img class="formulaInl" alt="$\mathcal{K}(a, b)$" src="form_157.png"/>.</p>
|
|
<p>The arguments <code>a</code> and <code>b</code>, of types <code>VecTypeA</code> and <code>VecTypeB</code>, respectively, will be an Armadillo-like vector type (usually <code>arma::vec</code>, <code>arma::sp_vec</code>, or similar). In general it should be valid to assume that <code>VecTypeA</code> is a class with the same API as <code>arma::vec</code>.</p>
|
|
<p>Note that for kernels that do not hold any state, the <code>Evaluate()</code> method can be marked as <code>static</code>.</p>
|
|
<p>Overall, the <code>KernelType</code> template policy is quite simple (much like the <a class="el" href="metrics.html">MetricType policy</a>). Below is an example kernel class, which outputs <code>1</code> if the vectors are close and <code>0</code> otherwise.</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">class </span>ExampleKernel</div><div class="line">{</div><div class="line"> <span class="comment">// Default constructor is required.</span></div><div class="line"> ExampleKernel() { }</div><div class="line"></div><div class="line"> <span class="comment">// The example kernel holds no state, so we can mark Evaluate() as static.</span></div><div class="line"> <span class="keyword">template</span><<span class="keyword">typename</span> VecTypeA, <span class="keyword">typename</span> VecTypeB></div><div class="line"> <span class="keyword">static</span> <span class="keywordtype">double</span> Evaluate(<span class="keyword">const</span> VecTypeA& a, <span class="keyword">const</span> VecTypeB& b)</div><div class="line"> {</div><div class="line"> <span class="comment">// Get how far apart the vectors are (using the Euclidean distance).</span></div><div class="line"> <span class="keyword">const</span> <span class="keywordtype">double</span> distance = arma::norm(a - b);</div><div class="line"></div><div class="line"> <span class="keywordflow">if</span> (distance < 0.05) <span class="comment">// Less than 0.05 distance is "close".</span></div><div class="line"> <span class="keywordflow">return</span> 1;</div><div class="line"> <span class="keywordflow">else</span></div><div class="line"> <span class="keywordflow">return</span> 0;</div><div class="line"> }</div><div class="line">};</div></div><!-- fragment --><p>Then, this kernel may be easily used inside of mlpack algorithms. For instance, the code below runs kernel PCA (<code><a class="el" href="classmlpack_1_1kpca_1_1KernelPCA.html" title="This class performs kernel principal components analysis (Kernel PCA), for a given kernel...">mlpack::kpca::KernelPCA</a></code>) on a random dataset using the <code>ExampleKernel</code>. The results are saved to a file called <code>results.csv</code>. (Note that this is simply an example to demonstrate usage, and this example kernel isn't actually likely to be useful in practice.)</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="core_8hpp.html">mlpack/core.hpp</a>></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="kernel__pca_8hpp.html">mlpack/methods/kernel_pca/kernel_pca.hpp</a>></span></div><div class="line"><span class="preprocessor">#include "<a class="code" href="example__kernel_8hpp.html">example_kernel.hpp</a>"</span> <span class="comment">// Contains the ExampleKernel class.</span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack.html">mlpack</a>;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1kpca.html">mlpack::kpca</a>;</div><div class="line"><span class="keyword">using namespace </span>arma;</div><div class="line"></div><div class="line"><span class="keywordtype">int</span> main()</div><div class="line">{</div><div class="line"> <span class="comment">// Generate the random dataset; 10 dimensions, 5000 points.</span></div><div class="line"> mat dataset = randu<mat>(10, 5000);</div><div class="line"></div><div class="line"> <span class="comment">// Instantiate the KernelPCA object with the ExampleKernel kernel type.</span></div><div class="line"> <a class="code" href="classmlpack_1_1kpca_1_1KernelPCA.html">KernelPCA<ExampleKernel></a> kpca;</div><div class="line"></div><div class="line"> <span class="comment">// The dataset will be transformed using kernel PCA with the example kernel to</span></div><div class="line"> <span class="comment">// contain only 2 dimensions.</span></div><div class="line"> kpca.<a class="code" href="classmlpack_1_1kpca_1_1KernelPCA.html#a879bf80f5e30c27dd1d8acec37a39b70">Apply</a>(dataset, 2);</div><div class="line"></div><div class="line"> <span class="comment">// Save the results to 'results.csv'.</span></div><div class="line"> <a class="code" href="namespacemlpack_1_1data.html#a16bc794b14db0ab126595b19a32a5bc0">data::Save</a>(dataset, <span class="stringliteral">"results.csv"</span>);</div><div class="line">}</div></div><!-- fragment --><h1><a class="anchor" id="kerneltraits"></a>
|
|
The KernelTraits trait class</h1>
|
|
<p>Some algorithms that use kernels can specialize if the kernel fulfills some certain conditions. An example of a condition might be that the kernel is shift-invariant or that the kernel is normalized. In the case of fast max-kernel search (<a class="el" href="classmlpack_1_1fastmks_1_1FastMKS.html" title="An implementation of fast exact max-kernel search. ">mlpack::fastmks::FastMKS</a>), the computation can be accelerated if the kernel is normalized. For this reason, the <code>KernelTraits</code> trait class exists. This allows a kernel to specify via a <code>const</code> <code>static</code> <code>bool</code> when these types of conditions are satisfied. <b>Note that a KernelTraits class is not required,</b> but may be helpful.</p>
|
|
<p>The <code>KernelTraits</code> trait class is a template class that takes a <code>KernelType</code> as a parameter, and exposes <code>const</code> <code>static</code> <code>bool</code> values that depend on the kernel. Setting these values is achieved by specialization. The code below provides an example, specializing <code>KernelTraits</code> for the <code>ExampleKernel</code> from earlier:</p>
|
|
<div class="fragment"><div class="line"><span class="keyword">template</span><></div><div class="line"><span class="keyword">class </span>KernelTraits<ExampleKernel></div><div class="line">{</div><div class="line"> <span class="keyword">public</span>:</div><div class="line"> <span class="keyword">const</span> <span class="keyword">static</span> <span class="keywordtype">bool</span> IsNormalized = <span class="keyword">true</span>;</div><div class="line">};</div></div><!-- fragment --><p>At this time, there is only one kernel trait that is used in mlpack code:</p>
|
|
<ul>
|
|
<li><code>IsNormalized</code> (defaults to <code>false</code>): if <img class="formulaInl" alt="$ K(x, x) = 1 \; \forall x $" src="form_158.png"/>, then the kernel is normalized and this should be set to true.</li>
|
|
</ul>
|
|
<h1><a class="anchor" id="kernellist"></a>
|
|
List of kernels and classes that use a \c KernelType</h1>
|
|
<p>mlpack comes with a number of pre-written kernels that satisfy the <code>KernelType</code> policy:</p>
|
|
<ul>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1LinearKernel.html" title="The simple linear kernel (dot product). ">mlpack::kernel::LinearKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1ExampleKernel.html" title="An example kernel function. ">mlpack::kernel::ExampleKernel</a> – an example kernel with more documentation</li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1GaussianKernel.html" title="The standard Gaussian kernel. ">mlpack::kernel::GaussianKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1HyperbolicTangentKernel.html" title="Hyperbolic tangent kernel. ">mlpack::kernel::HyperbolicTangentKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1EpanechnikovKernel.html" title="The Epanechnikov kernel, defined as. ">mlpack::kernel::EpanechnikovKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1CosineDistance.html" title="The cosine distance (or cosine similarity). ">mlpack::kernel::CosineDistance</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1LaplacianKernel.html" title="The standard Laplacian kernel. ">mlpack::kernel::LaplacianKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1PolynomialKernel.html" title="The simple polynomial kernel. ">mlpack::kernel::PolynomialKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1TriangularKernel.html" title="The trivially simple triangular kernel, defined by. ">mlpack::kernel::TriangularKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1SphericalKernel.html" title="The spherical kernel, which is 1 when the distance between the two argument points is less than or eq...">mlpack::kernel::SphericalKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1PSpectrumStringKernel.html" title="The p-spectrum string kernel. ">mlpack::kernel::PSpectrumStringKernel</a> – operates on strings, not vectors</li>
|
|
</ul>
|
|
<p>These kernels (or a custom kernel) may be used in a variety of mlpack methods:</p>
|
|
<ul>
|
|
<li><a class="el" href="classmlpack_1_1kpca_1_1KernelPCA.html" title="This class performs kernel principal components analysis (Kernel PCA), for a given kernel...">mlpack::kpca::KernelPCA</a> - kernel principal components analysis</li>
|
|
<li><a class="el" href="classmlpack_1_1fastmks_1_1FastMKS.html" title="An implementation of fast exact max-kernel search. ">mlpack::fastmks::FastMKS</a> - fast max-kernel search</li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1NystroemMethod.html">mlpack::kernel::NystroemMethod</a> - the Nystroem method for sampling</li>
|
|
<li><a class="el" href="classmlpack_1_1metric_1_1IPMetric.html" title="The inner product metric, IPMetric, takes a given Mercer kernel (KernelType), and when Evaluate() is ...">mlpack::metric::IPMetric</a> - a metric built on a kernel </li>
|
|
</ul>
|
|
</div></div><!-- contents -->
|
|
<!-- start footer part -->
|
|
<hr class="footer"/><address class="footer"><small>
|
|
Generated by  <a href="http://www.doxygen.org/index.html">
|
|
<img class="footer" src="doxygen.png" alt="doxygen"/>
|
|
</a> 1.8.11
|
|
</small></address>
|
|
</body>
|
|
<script type="text/javascript">
|
|
var x = document.getElementsByClassName("formulaDsp");
|
|
var i;
|
|
for (i = 0; i < x.length; i++)
|
|
{
|
|
x[i].width /= 4;
|
|
}
|
|
</script>
|
|
</html>
|