247 lines
41 KiB
HTML
247 lines
41 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
|
|
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
|
|
<meta name="generator" content="Doxygen 1.8.11"/>
|
|
<title>mlpack: Fast max-kernel search tutorial (fastmks)</title>
|
|
<link href="tabs.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="jquery.js"></script>
|
|
<script type="text/javascript" src="dynsections.js"></script>
|
|
<link href="search/search.css" rel="stylesheet" type="text/css"/>
|
|
<script type="text/javascript" src="search/searchdata.js"></script>
|
|
<script type="text/javascript" src="search/search.js"></script>
|
|
<script type="text/javascript">
|
|
$(document).ready(function() { init_search(); });
|
|
</script>
|
|
<link href="doxygen.css" rel="stylesheet" type="text/css" />
|
|
<link href="extra-stylesheet.css" rel="stylesheet" type="text/css"/>
|
|
</head>
|
|
<body>
|
|
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
|
|
<div id="titlearea">
|
|
<table cellspacing="0" cellpadding="0">
|
|
<tbody>
|
|
<tr style="height: 56px;">
|
|
<td id="projectalign" style="padding-left: 0.5em;">
|
|
<div id="projectname">mlpack
|
|
 <span id="projectnumber">master</span>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<!-- end header part -->
|
|
<!-- Generated by Doxygen 1.8.11 -->
|
|
<script type="text/javascript">
|
|
var searchBox = new SearchBox("searchBox", "search",false,'Search');
|
|
</script>
|
|
<div id="navrow1" class="tabs">
|
|
<ul class="tablist">
|
|
<li><a href="index.html"><span>Main Page</span></a></li>
|
|
<li class="current"><a href="pages.html"><span>Related Pages</span></a></li>
|
|
<li><a href="namespaces.html"><span>Namespaces</span></a></li>
|
|
<li><a href="annotated.html"><span>Classes</span></a></li>
|
|
<li><a href="files.html"><span>Files</span></a></li>
|
|
<li>
|
|
<div id="MSearchBox" class="MSearchBoxInactive">
|
|
<span class="left">
|
|
<img id="MSearchSelect" src="search/mag_sel.png"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
alt=""/>
|
|
<input type="text" id="MSearchField" value="Search" accesskey="S"
|
|
onfocus="searchBox.OnSearchFieldFocus(true)"
|
|
onblur="searchBox.OnSearchFieldFocus(false)"
|
|
onkeyup="searchBox.OnSearchFieldChange(event)"/>
|
|
</span><span class="right">
|
|
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
|
|
</span>
|
|
</div>
|
|
</li>
|
|
</ul>
|
|
</div>
|
|
<!-- window showing the filter options -->
|
|
<div id="MSearchSelectWindow"
|
|
onmouseover="return searchBox.OnSearchSelectShow()"
|
|
onmouseout="return searchBox.OnSearchSelectHide()"
|
|
onkeydown="return searchBox.OnSearchSelectKey(event)">
|
|
</div>
|
|
|
|
<!-- iframe showing the search results (closed by default) -->
|
|
<div id="MSearchResultsWindow">
|
|
<iframe src="javascript:void(0)" frameborder="0"
|
|
name="MSearchResults" id="MSearchResults">
|
|
</iframe>
|
|
</div>
|
|
|
|
</div><!-- top -->
|
|
<div class="header">
|
|
<div class="headertitle">
|
|
<div class="title">Fast max-kernel search tutorial (fastmks) </div> </div>
|
|
</div><!--header-->
|
|
<div class="contents">
|
|
<div class="textblock"><h1><a class="anchor" id="intro_fmkstut"></a>
|
|
Introduction</h1>
|
|
<p>The FastMKS algorithm (fast exact max-kernel search) is a recent algorithm proposed in the following papers:</p>
|
|
<div class="fragment"><div class="line">@inproceedings{curtin2013fast,</div><div class="line"> title={Fast Exact Max-Kernel Search},</div><div class="line"> author={Curtin, Ryan R. and Ram, Parikshit and Gray, Alexander G.},</div><div class="line"> booktitle={Proceedings of the 2013 SIAM International Conference on Data</div><div class="line"> Mining (SDM <span class="stringliteral">'13)},</span></div><div class="line"><span class="stringliteral"> year={2013},</span></div><div class="line"><span class="stringliteral"> pages={1--9}</span></div><div class="line"><span class="stringliteral">}</span></div><div class="line"><span class="stringliteral"></span></div><div class="line"><span class="stringliteral">@article{curtin2014dual,</span></div><div class="line"><span class="stringliteral"> author = {Curtin, Ryan R. and Ram, Parikshit},</span></div><div class="line"><span class="stringliteral"> title = {Dual-tree fast exact max-kernel search},</span></div><div class="line"><span class="stringliteral"> journal = {Statistical Analysis and Data Mining},</span></div><div class="line"><span class="stringliteral"> volume = {7},</span></div><div class="line"><span class="stringliteral"> number = {4},</span></div><div class="line"><span class="stringliteral"> publisher = {Wiley Subscription Services, Inc., A Wiley Company},</span></div><div class="line"><span class="stringliteral"> issn = {1932-1872},</span></div><div class="line"><span class="stringliteral"> url = {http://dx.doi.org/10.1002/sam.11218},</span></div><div class="line"><span class="stringliteral"> doi = {10.1002/sam.11218},</span></div><div class="line"><span class="stringliteral"> pages = {229--253},</span></div><div class="line"><span class="stringliteral"> year = {2014},</span></div><div class="line"><span class="stringliteral">}</span></div></div><!-- fragment --><p>Given a set of query points <img class="formulaInl" alt="$Q$" src="form_102.png"/> and a set of reference points <img class="formulaInl" alt="$R$" src="form_101.png"/>, the FastMKS algorithm is a fast dual-tree (or single-tree) algorithm which finds</p>
|
|
<p class="formulaDsp">
|
|
<img class="formulaDsp" alt="\[ \arg\max_{p_r \in R} K(p_q, p_r) \]" src="form_122.png"/>
|
|
</p>
|
|
<p>for all points <img class="formulaInl" alt="$p_q \in Q$" src="form_123.png"/> and for some Mercer kernel <img class="formulaInl" alt="$K(\cdot, \cdot)$" src="form_124.png"/>. A Mercer kernel is a kernel that is positive semidefinite; these are the classes of kernels that can be used with the kernel trick. In short, the positive semidefiniteness of a Mercer kernel means that any kernel matrix (or Gram matrix) created on a dataset must be positive semidefinite.</p>
|
|
<p>The FastMKS algorithm builds trees on the datasets <img class="formulaInl" alt="$Q$" src="form_102.png"/> and <img class="formulaInl" alt="$R$" src="form_101.png"/> in such a way that explicit representation of the points in the kernel space is unnecessary, by using cover trees (<a class="el" href="classmlpack_1_1tree_1_1CoverTree.html">mlpack::tree::CoverTree</a>). This allows the algorithm to be run, for instance, on string kernels, where there is no sensible explicit representation. The <b>mlpack</b> implementation allows any type of tree that does not require an explicit representation to be used. For more details, see the paper.</p>
|
|
<p>At the time of this writing there is no other fast algorithm for exact max-kernel search. <b>mlpack</b> implements both single-tree and dual-tree fast max-kernel search.</p>
|
|
<p><b>mlpack</b> provides:</p>
|
|
<ul>
|
|
<li>a <a class="el" href="fmkstutorial.html#cli_fmkstut">simple command-line executable</a> to run FastMKS</li>
|
|
<li>a <a class="el" href="fmkstutorial.html#fastmks_fmkstut">C++ interface</a> to run FastMKS</li>
|
|
</ul>
|
|
<h1><a class="anchor" id="toc_fmkstut"></a>
|
|
Table of Contents</h1>
|
|
<p>A list of all the sections this tutorial contains.</p>
|
|
<ul>
|
|
<li><a class="el" href="fmkstutorial.html#intro_fmkstut">Introduction</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#toc_fmkstut">Table of Contents</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#cli_fmkstut">Command-line FastMKS (mlpack_fastmks)</a><ul>
|
|
<li><a class="el" href="fmkstutorial.html#cli_ex1_fmkstut">FastMKS with a linear kernel on one dataset</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#cli_ex2_fmkstut">FastMKS on a reference and query dataset</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#cli_ex3_fmkstut">FastMKS with a different kernel</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#cli_ex4_fmkstut">Using single-tree search or naive search</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#cli_ex5_fmkstut">Parameters for alternate kernels</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#cli_ex6_fmkstut">Saving a FastMKS model/tree</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#cli_ex7_fmkstut">Loading a FastMKS model for further searches</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a class="el" href="fmkstutorial.html#fastmks_fmkstut">The 'FastMKS' class</a><ul>
|
|
<li><a class="el" href="fmkstutorial.html#fastmks_ex1_fmkstut">FastMKS on one dataset</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#fastmks_ex2_fmkstut">FastMKS with a query and reference dataset</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#fastmks_ex3_fmkstut">FastMKS with an initialized kernel</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#fastmks_ex4_fmkstut">FastMKS with an already-created tree</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a class="el" href="fmkstutorial.html#writing_kernel_fmkstut">Writing a custom kernel for FastMKS</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#custom_tree_fmkstut">Using other tree types for FastMKS</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#objects_fmkstut">Running FastMKS on objects</a></li>
|
|
<li><a class="el" href="fmkstutorial.html#further_doc_fmkstut">Further documentation</a></li>
|
|
</ul>
|
|
<h1><a class="anchor" id="cli_fmkstut"></a>
|
|
Command-line FastMKS (mlpack_fastmks)</h1>
|
|
<p><b>mlpack</b> provides a command-line program, <code>mlpack_fastmks</code>, which is used to perform FastMKS on a given query and reference dataset. It supports numerous different types of kernels:</p>
|
|
<ul>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1LinearKernel.html">linear kernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1PolynomialKernel.html">polynomial kernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1CosineDistance.html">cosine distance</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1GaussianKernel.html">Gaussian kernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1EpanechnikovKernel.html">Epanechnikov kernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1TriangularKernel.html">triangular kernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1HyperbolicTangentKernel.html">hyperbolic tangent kernel</a></li>
|
|
</ul>
|
|
<p>Note that when a shift-invariant kernel is used, the results will be the same as nearest neighbor search, so <a class="el" href="nstutorial.html">KNN</a> may be a better option. A shift-invariant kernel is a kernel that depends only on the distance between the two input points. The <a class="el" href="classmlpack_1_1kernel_1_1GaussianKernel.html">Gaussian kernel</a>, <a class="el" href="classmlpack_1_1kernel_1_1EpanechnikovKernel.html">Epanechnikov kernel</a>, and <a class="el" href="classmlpack_1_1kernel_1_1TriangularKernel.html">triangular kernel</a> are instances of shift-invariant kernels. The paper contains more details on this situation. The <code>mlpack_fastmks</code> executable still provides these kernels as options, though.</p>
|
|
<p>The following examples detail usage of the <code>mlpack_fastmks</code> program. Note that you can get documentation on all the possible parameters by typing:</p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks --help</div></div><!-- fragment --><h2><a class="anchor" id="cli_ex1_fmkstut"></a>
|
|
FastMKS with a linear kernel on one dataset</h2>
|
|
<p>If only one dataset is specified (with <code>-r</code> or <code>–reference_file</code>), the reference dataset is taken to be both the query and reference datasets. The example below finds the 4 maximum kernels of each point in dataset.csv, using the default linear kernel.</p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks -r dataset.csv -k 4 -v -p products.csv -i indices.csv</div></div><!-- fragment --><p>When the operation completes, the values of the kernels are saved in products.csv and the indices of the points which give the maximum kernels are saved in indices.csv.</p>
|
|
<div class="fragment"><div class="line">$ head indices.csv</div><div class="line">762,910,863,890</div><div class="line">762,910,426,568</div><div class="line">910,762,863,426</div><div class="line">762,910,863,426</div><div class="line">863,910,614,762</div><div class="line">762,863,910,614</div><div class="line">762,910,488,568</div><div class="line">762,910,863,426</div><div class="line">910,762,863,426</div><div class="line">863,762,910,614</div></div><!-- fragment --><div class="fragment"><div class="line">$ head products.csv</div><div class="line">1.6221652894e+00,1.5998743443e+00,1.5898890769e+00,1.5406789753e+00</div><div class="line">1.3387953449e+00,1.3317349486e+00,1.2966613184e+00,1.2774493620e+00</div><div class="line">1.6386110476e+00,1.6332029753e+00,1.5952629124e+00,1.5887195330e+00</div><div class="line">1.0917545803e+00,1.0820878726e+00,1.0668992636e+00,1.0419838050e+00</div><div class="line">1.2272441028e+00,1.2169643942e+00,1.2104597963e+00,1.2067780154e+00</div><div class="line">1.5720962456e+00,1.5618504956e+00,1.5609069923e+00,1.5235605095e+00</div><div class="line">1.3655478674e+00,1.3548593212e+00,1.3311547298e+00,1.3250728881e+00</div><div class="line">2.0119149744e+00,2.0043668067e+00,1.9847289214e+00,1.9298280046e+00</div><div class="line">1.1586923205e+00,1.1494586097e+00,1.1274872962e+00,1.1248172766e+00</div><div class="line">4.4789820372e-01,4.4618539778e-01,4.4200024852e-01,4.3989721792e-01</div></div><!-- fragment --><p>We can see in this example that for point 0, the point with maximum kernel value is point 762, with a kernel value of 1.622165. For point 3, the point with third largest kernel value is point 863, with a kernel value of 1.0669.</p>
|
|
<h2><a class="anchor" id="cli_ex2_fmkstut"></a>
|
|
FastMKS on a reference and query dataset</h2>
|
|
<p>The query points may be different than the reference points. To specify a different query set, the <code>-q</code> (or <code>–query_file</code>) option is used, as in the example below.</p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks -q query_set.csv -r reference_set.csv -k 5 -i indices.csv \</div><div class="line">> -p products.csv</div></div><!-- fragment --><h2><a class="anchor" id="cli_ex3_fmkstut"></a>
|
|
FastMKS with a different kernel</h2>
|
|
<p>The <code>mlpack_fastmks</code> program offers more than just the linear kernel. Valid options are <code>'linear'</code>, <code>'polynomial'</code>, <code>'cosine'</code>, <code>'gaussian'</code>, <code>'epanechnikov'</code>, <code>'triangular'</code> and <code>'hyptan'</code> (the hyperbolic tangent kernel). Note that the hyperbolic tangent kernel is provably not a Mercer kernel but is positive semidefinite on most datasets and is commonly used as a kernel. Note also that the Gaussian kernel and other shift-invariant kernels give the same results as nearest neighbor search (see <a class="el" href="nstutorial.html">NeighborSearch tutorial (k-nearest-neighbors)</a>).</p>
|
|
<p>The kernel to use is specified with the <code>-K</code> (or <code>–kernel</code>) option. The example below uses the cosine similarity as a kernel.</p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks -r dataset.csv -k 5 -K cosine -i indices.csv -p products.csv -v</div></div><!-- fragment --><h2><a class="anchor" id="cli_ex4_fmkstut"></a>
|
|
Using single-tree search or naive search</h2>
|
|
<p>In some cases, it may be useful to not use the dual-tree FastMKS algorithm. Instead you can specify the <code>–single</code> option, indicating that a tree should be built only on the reference set, and then the queries should be processed in a linear scan (instead of in a tree). Alternately, the <code>-N</code> (or <code>–naive</code>) option makes the program not build trees at all and instead use brute-force search to find the solutions.</p>
|
|
<p>The example below uses single-tree search on two datasets with the linear kernel.</p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks -q query_set.csv -r reference_set.csv --single -k 5 \</div><div class="line">> -p products.csv -i indices.csv -K linear</div></div><!-- fragment --><p>The example below uses naive search on one dataset.</p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks -r reference_set.csv -k 5 -N -p products.csv -i indices.csv</div></div><!-- fragment --><h2><a class="anchor" id="cli_ex5_fmkstut"></a>
|
|
Parameters for alternate kernels</h2>
|
|
<p>Many of the alternate kernel choices have parameters which can be chosen; these are detailed in this section.</p>
|
|
<ul>
|
|
<li><b><code>-w</code> </b>(<code>–bandwidth</code>): this sets the bandwidth of the kernel, and is applicable to the <code>'gaussian'</code>, <code>'epanechnikov'</code>, and <code>'triangular'</code> kernels. This is the "spread" of the kernel.</li>
|
|
<li><b><code>-d</code> </b>(<code>–degree</code>): this sets the degree of the polynomial kernel (the power to which the result is raised). It is only applicable to the <code>'polynomial'</code> kernel.</li>
|
|
<li><b><code>-o</code> </b>(<code>–offset</code>): this sets the offset of the kernel, for the <code>'polynomial'</code> and <code>'hyptan'</code> kernel. See <a class="el" href="classmlpack_1_1kernel_1_1PolynomialKernel.html">the polynomial kernel documentation</a> and <a class="el" href="classmlpack_1_1kernel_1_1HyperbolicTangentKernel.html">the hyperbolic tangent kernel documentation</a> for more information.</li>
|
|
<li><b><code>-s</code> </b>(<code>–scale</code>): this sets the scale of the kernel, and is only applicable to the <code>'hyptan'</code> kernel. See <a class="el" href="classmlpack_1_1kernel_1_1HyperbolicTangentKernel.html">the hyperbolic tangent kernel documentation</a> for more information.</li>
|
|
</ul>
|
|
<h2><a class="anchor" id="cli_ex6_fmkstut"></a>
|
|
Saving a FastMKS model/tree</h2>
|
|
<p>The <code>mlpack_fastmks</code> program also supports saving a model built on a reference dataset (this model includes the tree, the kernel, and the search parameters). The <code>–output_model_file</code> or <code>-M</code> option allows one to save these parameters to disk for later usage. An example is below:</p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks -r reference_set.csv -K cosine -M fastmks_model.xml</div></div><!-- fragment --><p>This example builds a tree on the dataset in <code>reference_set.csv</code> using the cosine similarity kernel, and saves the resulting model to <code>fastmks_model.xml</code>. This model may then be used in later calls to the <code>mlpack_fastmks</code> program.</p>
|
|
<h2><a class="anchor" id="cli_ex7_fmkstut"></a>
|
|
Loading a FastMKS model for further searches</h2>
|
|
<p>Supposing that a FastMKS model has been saved with the <code>–output_model_file</code> or <code>-M</code> parameter, that model can then be later loaded in subsequent calls to the <code>mlpack_fastmks</code> program, using the <code>–input_model_file</code> or <code>-m</code> option. For instance, with a model saved in <code>fastmks_model.xml</code> and a query set in <code>query_set.csv</code>, we can find 3 max-kernel candidates, saving to <code>indices.csv</code> and <code>kernels.csv:</code> </p>
|
|
<div class="fragment"><div class="line">$ mlpack_fastmks -m fastmks_model.xml -k 3 -i indices.csv -p kernels.csv</div></div><!-- fragment --><p>Loading a model as opposed to building a model is advantageous because the reference tree is already built. So, among other situations, this could be useful in the setting where many different query sets (or many different values of k) will be used.</p>
|
|
<p>Note that the kernel cannot be changed in a saved model without rebuilding the model entirely.</p>
|
|
<h1><a class="anchor" id="fastmks_fmkstut"></a>
|
|
The 'FastMKS' class</h1>
|
|
<p>The <code>FastMKS<></code> class offers a simple API for use within C++ applications, and allows further flexibility in kernel choice and tree type choice. However, <code>FastMKS<></code> has no default template parameter for the kernel type – that must be manually specified. Choices that <b>mlpack</b> provides include:</p>
|
|
<ul>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1LinearKernel.html">mlpack::kernel::LinearKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1PolynomialKernel.html">mlpack::kernel::PolynomialKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1CosineDistance.html">mlpack::kernel::CosineDistance</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1GaussianKernel.html">mlpack::kernel::GaussianKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1EpanechnikovKernel.html">mlpack::kernel::EpanechnikovKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1TriangularKernel.html">mlpack::kernel::TriangularKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1HyperbolicTangentKernel.html">mlpack::kernel::HyperbolicTangentKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1LaplacianKernel.html">mlpack::kernel::LaplacianKernel</a></li>
|
|
<li><a class="el" href="classmlpack_1_1kernel_1_1PSpectrumStringKernel.html">mlpack::kernel::PSpectrumStringKernel</a></li>
|
|
</ul>
|
|
<p>The following examples use kernels from that list. Writing your own kernel is detailed in <a class="el" href="fmkstutorial.html#writing_kernel_fmkstut">the next section</a>. Remember that when you are using the C++ interface, the data matrices must be column-major. See <a class="el" href="matrices.html">Matrices in mlpack</a> for more information.</p>
|
|
<h2><a class="anchor" id="fastmks_ex1_fmkstut"></a>
|
|
FastMKS on one dataset</h2>
|
|
<p>Given only a reference dataset, the following code will run FastMKS with k set to 5.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="fastmks_8hpp.html">mlpack/methods/fastmks/fastmks.hpp</a>></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="linear__kernel_8hpp.html">mlpack/core/kernels/linear_kernel.hpp</a>></span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1fastmks.html">mlpack::fastmks</a>;</div><div class="line"></div><div class="line"><span class="comment">// The reference dataset, which is column-major.</span></div><div class="line"><span class="keyword">extern</span> arma::mat data;</div><div class="line"></div><div class="line"><span class="comment">// This will initialize the FastMKS object with the linear kernel with default</span></div><div class="line"><span class="comment">// options: K(x, y) = x^T y. The tree is built in the constructor.</span></div><div class="line"><a class="code" href="classmlpack_1_1fastmks_1_1FastMKS.html">FastMKS<LinearKernel></a> f(data);</div><div class="line"></div><div class="line"><span class="comment">// The results will be stored in these matrices.</span></div><div class="line">arma::Mat<size_t> indices;</div><div class="line">arma::mat products;</div><div class="line"></div><div class="line"><span class="comment">// Run FastMKS.</span></div><div class="line">f.Search(5, indices, products);</div></div><!-- fragment --><h2><a class="anchor" id="fastmks_ex2_fmkstut"></a>
|
|
FastMKS with a query and reference dataset</h2>
|
|
<p>In this setting we have both a query and reference dataset. We search for 10 maximum kernels.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="fastmks_8hpp.html">mlpack/methods/fastmks/fastmks.hpp</a>></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="triangular__kernel_8hpp.html">mlpack/core/kernels/triangular_kernel.hpp</a>></span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1fastmks.html">mlpack::fastmks</a>;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1kernel.html">mlpack::kernel</a>;</div><div class="line"></div><div class="line"><span class="comment">// The reference and query datasets, which are column-major.</span></div><div class="line"><span class="keyword">extern</span> arma::mat referenceData;</div><div class="line"><span class="keyword">extern</span> arma::mat queryData;</div><div class="line"></div><div class="line"><span class="comment">// This will initialize the FastMKS object with the triangular kernel with</span></div><div class="line"><span class="comment">// default options (bandwidth of 1). The reference tree is built in the</span></div><div class="line"><span class="comment">// constructor.</span></div><div class="line"><a class="code" href="classmlpack_1_1fastmks_1_1FastMKS.html">FastMKS<TriangularKernel></a> f(referenceData);</div><div class="line"></div><div class="line"><span class="comment">// The results will be stored in these matrices.</span></div><div class="line">arma::Mat<size_t> indices;</div><div class="line">arma::mat products;</div><div class="line"></div><div class="line"><span class="comment">// Run FastMKS. The query tree is built during the call to Search().</span></div><div class="line">f.Search(queryData, 10, indices, products);</div></div><!-- fragment --><h2><a class="anchor" id="fastmks_ex3_fmkstut"></a>
|
|
FastMKS with an initialized kernel</h2>
|
|
<p>Often, kernels have parameters which need to be specified. <code>FastMKS<></code> has constructors which take initialized kernels. Note that temporary kernels cannot be passed as an argument. The example below initializes a <code>PolynomialKernel</code> object and then runs FastMKS with a query and reference dataset.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="fastmks_8hpp.html">mlpack/methods/fastmks/fastmks.hpp</a>></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="polynomial__kernel_8hpp.html">mlpack/core/kernels/polynomial_kernel.hpp</a>></span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1fastmks.html">mlpack::fastmks</a>;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1kernel.html">mlpack::kernel</a>;</div><div class="line"></div><div class="line"><span class="comment">// The reference and query datasets, which are column-major.</span></div><div class="line"><span class="keyword">extern</span> arma::mat referenceData;</div><div class="line"><span class="keyword">extern</span> arma::mat queryData;</div><div class="line"></div><div class="line"><span class="comment">// Initialize the polynomial kernel with degree of 3 and offset of 2.5.</span></div><div class="line"><a class="code" href="classmlpack_1_1kernel_1_1PolynomialKernel.html">PolynomialKernel</a> pk(3.0, 2.5);</div><div class="line"></div><div class="line"><span class="comment">// Create the FastMKS object with the initialized kernel.</span></div><div class="line"><a class="code" href="classmlpack_1_1fastmks_1_1FastMKS.html">FastMKS<PolynomialKernel></a> f(referenceData, pk);</div><div class="line"></div><div class="line"><span class="comment">// The results will be stored in these matrices.</span></div><div class="line">arma::Mat<size_t> indices;</div><div class="line">arma::mat products;</div><div class="line"></div><div class="line"><span class="comment">// Run FastMKS.</span></div><div class="line">f.Search(queryData, 10, indices, products);</div></div><!-- fragment --><p>The syntax for running FastMKS with one dataset and an initialized kernel is very similar:</p>
|
|
<div class="fragment"><div class="line">f.Search(10, indices, products);</div></div><!-- fragment --><h2><a class="anchor" id="fastmks_ex4_fmkstut"></a>
|
|
FastMKS with an already-created tree</h2>
|
|
<p>By default, <code>FastMKS<></code> uses the cover tree datastructure (see <a class="el" href="classmlpack_1_1tree_1_1CoverTree.html">mlpack::tree::CoverTree</a>). Sometimes, it is useful to modify the parameters of the cover tree. In this scenario, a tree must be built outside of the constructor, and then passed to the appropriate <code>FastMKS<></code> constructor. An example on just a reference dataset is shown below, where the base of the cover tree is modified.</p>
|
|
<p>We also use an instantiated kernel, but because we are building our own tree, we must use <a class="el" href="classmlpack_1_1metric_1_1IPMetric.html">IPMetric</a> so that our tree is built on the metric induced by our kernel function.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="fastmks_8hpp.html">mlpack/methods/fastmks/fastmks.hpp</a>></span></div><div class="line"><span class="preprocessor">#include <<a class="code" href="polynomial__kernel_8hpp.html">mlpack/core/kernels/polynomial_kernel.hpp</a>></span></div><div class="line"></div><div class="line"><span class="comment">// The reference dataset, which is column-major.</span></div><div class="line"><span class="keyword">extern</span> arma::mat data;</div><div class="line"></div><div class="line"><span class="comment">// Initialize the polynomial kernel with a degree of 4 and offset of 2.0.</span></div><div class="line">PolynomialKernel pk(4.0, 2.0);</div><div class="line"></div><div class="line"><span class="comment">// Create the metric induced by this kernel (because a kernel is not a metric</span></div><div class="line"><span class="comment">// and we can't build a tree on a kernel alone).</span></div><div class="line">IPMetric<PolynomialKernel> metric(pk);</div><div class="line"></div><div class="line"><span class="comment">// Now build a tree on the reference dataset using the instantiated metric and</span></div><div class="line"><span class="comment">// the custom base of 1.5 (default is 1.3). We have to be sure to use the right</span></div><div class="line"><span class="comment">// type here -- FastMKS needs the FastMKSStat object as the tree's</span></div><div class="line"><span class="comment">// StatisticType.</span></div><div class="line"><span class="keyword">typedef</span> tree::CoverTree<IPMetric<PolynomialKernel>, tree::FirstPointIsRoot,</div><div class="line"> FastMKSStat> TreeType; <span class="comment">// Convenience typedef.</span></div><div class="line">TreeType* tree = <span class="keyword">new</span> TreeType(data, metric, 1.5);</div><div class="line"></div><div class="line"><span class="comment">// Now initialize FastMKS with that statistic. We don't need to specify the</span></div><div class="line"><span class="comment">// TreeType template parameter since we are still using the default. We don't</span></div><div class="line"><span class="comment">// need to pass the kernel because that is contained in the tree.</span></div><div class="line">FastMKS<PolynomialKernel> f(tree);</div><div class="line"></div><div class="line"><span class="comment">// The results will be stored in these matrices.</span></div><div class="line">arma::Mat<size_t> indices;</div><div class="line">arma::mat products;</div><div class="line"></div><div class="line"><span class="comment">// Run FastMKS.</span></div><div class="line">f.Search(10, indices, products);</div></div><!-- fragment --><p>The syntax is similar for the case where different query and reference datasets are given; but trees for both need to be built in the manner specified above. Be sure to build both trees using the same metric (or at least a metric with the exact same parameters).</p>
|
|
<div class="fragment"><div class="line">f.Search(queryTree, 10, indices, products);</div></div><!-- fragment --><h1><a class="anchor" id="writing_kernel_fmkstut"></a>
|
|
Writing a custom kernel for FastMKS</h1>
|
|
<p>While <b>mlpack</b> provides some number of kernels in the <a class="el" href="namespacemlpack_1_1kernel.html" title="Kernel functions. ">mlpack::kernel</a> namespace, it is easy to create a custom kernel. To satisfy the KernelType policy, a class must implement the following methods:</p>
|
|
<div class="fragment"><div class="line"><span class="comment">// Empty constructor is required.</span></div><div class="line">KernelType();</div><div class="line"></div><div class="line"><span class="comment">// Evaluate the kernel between two points.</span></div><div class="line"><span class="keyword">template</span><<span class="keyword">typename</span> VecType></div><div class="line"><span class="keywordtype">double</span> Evaluate(<span class="keyword">const</span> VecType& a, <span class="keyword">const</span> VecType& b);</div></div><!-- fragment --><p>The template parameter <code>VecType</code> is helpful (but not necessary) so that the kernel can be used with both sparse and dense matrices (<code>arma::sp_mat</code> and <code>arma::mat</code>).</p>
|
|
<h1><a class="anchor" id="custom_tree_fmkstut"></a>
|
|
Using other tree types for FastMKS</h1>
|
|
<p>The use of the cover tree (see <a class="el" href="classmlpack_1_1tree_1_1CoverTree.html">CoverTree</a>) is not necessary for FastMKS, although it is the default tree type. A different type of tree can be specified with the TreeType template parameter. However, the tree type is required to have <a class="el" href="classmlpack_1_1fastmks_1_1FastMKSStat.html">FastMKSStat</a> as the StatisticType, and for FastMKS to work, the tree must be built only on kernel evaluations (or distance evaluations in the kernel space via <a class="el" href="classmlpack_1_1metric_1_1IPMetric.html">IPMetric::Evaluate()</a>).</p>
|
|
<p>Below is an example where a custom tree class, <code>CustomTree</code>, is used as the tree type for FastMKS. In this example FastMKS is only run on one dataset.</p>
|
|
<div class="fragment"><div class="line"><span class="preprocessor">#include <<a class="code" href="fastmks_8hpp.html">mlpack/methods/fastmks/fastmks.hpp</a>></span></div><div class="line"><span class="preprocessor">#include "custom_tree.hpp"</span></div><div class="line"></div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1fastmks.html">mlpack::fastmks</a>;</div><div class="line"><span class="keyword">using namespace </span><a class="code" href="namespacemlpack_1_1tree.html">mlpack::tree</a>;</div><div class="line"></div><div class="line"><span class="comment">// The dataset that FastMKS will be run on.</span></div><div class="line"><span class="keyword">extern</span> arma::mat data;</div><div class="line"></div><div class="line"><span class="comment">// The custom tree type. We'll assume that the first template parameter is the</span></div><div class="line"><span class="comment">// statistic type.</span></div><div class="line"><span class="keyword">typedef</span> CustomTree<FastMKSStat> TreeType;</div><div class="line"></div><div class="line"><span class="comment">// The FastMKS constructor will create the tree.</span></div><div class="line"><a class="code" href="classmlpack_1_1fastmks_1_1FastMKS.html">FastMKS<LinearKernel, arma::mat, TreeType></a> f(data);</div><div class="line"></div><div class="line"><span class="comment">// These will hold the results.</span></div><div class="line">arma::Mat<size_t> indices;</div><div class="line">arma::mat products;</div><div class="line"></div><div class="line"><span class="comment">// Run FastMKS.</span></div><div class="line">f.Search(5, indices, products);</div></div><!-- fragment --><h1><a class="anchor" id="objects_fmkstut"></a>
|
|
Running FastMKS on objects</h1>
|
|
<p>FastMKS has a lot of utility on objects which are not representable in some sort of metric space. These objects might be strings, graphs, models, or other objects. For these types of objects, questions based on distance don't really make sense. One good example is with strings. The question "how far is 'dog'
|
|
from 'Taki Inoue'?" simply doesn't make sense. We can't have a centroid of the terms 'Fritz', 'E28', and 'popsicle'.</p>
|
|
<p>However, what we can do is define some sort of kernel on these objects. These kernels generally correspond to some similarity measure, with one example being the p-spectrum string kernel (see <a class="el" href="classmlpack_1_1kernel_1_1PSpectrumStringKernel.html">mlpack::kernel::PSpectrumStringKernel</a>). Using that, we can say "how similar is 'dog' to 'Taki Inoue'?" and get an actual numerical result by evaluating K('dog', 'Taki Inoue') (where K is our p-spectrum string kernel).</p>
|
|
<p>The only requirement on these kernels is that they are positive definite kernels (or Mercer kernels). For more information on those details, refer to the FastMKS paper.</p>
|
|
<p>Remember that FastMKS is a tree-based method. But trees like the binary space tree require centroids – and as we said earlier, centroids often don't make sense with these types of objects. Therefore, we need a type of tree which is built <b>exclusively</b> on points in the dataset – those are points which we can evaluate our kernel function on. The cover tree is one example of a type of tree satisfying this condition; its construction will only call the kernel function on two points that are in the dataset.</p>
|
|
<p>But, we have one more problem. The <code>CoverTree</code> class is built on <code>arma::mat</code> objects (dense matrices). Our objects, however, are not necessarily representable in a column of a matrix. To use the example we have been using, strings cannot be represented easily in a matrix because they may all have different lengths.</p>
|
|
<p>The way to work around this problem is to create a "fake" data matrix which simply holds indices to objects. A good example of how to do this is detailed in the documentation for the <a class="el" href="classmlpack_1_1kernel_1_1PSpectrumStringKernel.html">PSpectrumStringKernel</a>.</p>
|
|
<p>In short, the trick is to make each data matrix one-dimensional and containing linear indices:</p>
|
|
<div class="fragment"><div class="line">arma::mat data = <span class="stringliteral">"0 1 2 3 4 5 6 7 8"</span>;</div></div><!-- fragment --><p>Then, when <code>Evaluate()</code> is called on the kernel function, the parameters will be two one-dimensional vectors that simply contain indices to objects. The example below details the process a little better:</p>
|
|
<div class="fragment"><div class="line"><span class="comment">// This function evaluates the kernel on two Objects (in this example, its</span></div><div class="line"><span class="comment">// implementation is not important; the only important thing is that the</span></div><div class="line"><span class="comment">// function exists).</span></div><div class="line"><span class="keywordtype">double</span> ObjectKernel::Evaluate(<span class="keyword">const</span> Object& a, <span class="keyword">const</span> Object& b) <span class="keyword">const</span>;</div><div class="line"></div><div class="line"><span class="keyword">template</span><<span class="keyword">typename</span> VecType></div><div class="line"><span class="keywordtype">double</span> ObjectKernel::Evaluate(<span class="keyword">const</span> VecType& a, <span class="keyword">const</span> VecType& b)<span class="keyword"> const</span></div><div class="line"><span class="keyword"></span>{</div><div class="line"> <span class="comment">// Extract the indices from the vectors.</span></div><div class="line"> <span class="keyword">const</span> <span class="keywordtype">size_t</span> indexA = size_t(a[0]);</div><div class="line"> <span class="keyword">const</span> <span class="keywordtype">size_t</span> indexB = size_t(b[0]);</div><div class="line"></div><div class="line"> <span class="comment">// Assume that 'objects' is an array (or std::vector or other container)</span></div><div class="line"> <span class="comment">// holding Objects.</span></div><div class="line"> <span class="keyword">const</span> Object& objectA = objects[indexA];</div><div class="line"> <span class="keyword">const</span> Object& objectB = objects[indexB];</div><div class="line"></div><div class="line"> <span class="comment">// Now call the function that does the actual evaluation on the objects and</span></div><div class="line"> <span class="comment">// return its result.</span></div><div class="line"> <span class="keywordflow">return</span> Evaluate(objectA, objectB);</div><div class="line">}</div></div><!-- fragment --><p>As written earlier, the documentation for <a class="el" href="classmlpack_1_1kernel_1_1PSpectrumStringKernel.html">PSpectrumStringKernel</a> is a good place to consult for further reference on this. That kernel uses two dimensional indices; one dimension represents the index of the string, and the other represents whether it is referring to the query set or the reference set. If your kernel is meant to work on separate query and reference sets, that strategy should be considered.</p>
|
|
<h1><a class="anchor" id="further_doc_fmkstut"></a>
|
|
Further documentation</h1>
|
|
<p>For further documentation on the FastMKS class, consult the <a class="el" href="classmlpack_1_1fastmks_1_1FastMKS.html">complete API documentation</a>. </p>
|
|
</div></div><!-- contents -->
|
|
<!-- start footer part -->
|
|
<hr class="footer"/><address class="footer"><small>
|
|
Generated by  <a href="http://www.doxygen.org/index.html">
|
|
<img class="footer" src="doxygen.png" alt="doxygen"/>
|
|
</a> 1.8.11
|
|
</small></address>
|
|
</body>
|
|
<script type="text/javascript">
|
|
var x = document.getElementsByClassName("formulaDsp");
|
|
var i;
|
|
for (i = 0; i < x.length; i++)
|
|
{
|
|
x[i].width /= 4;
|
|
}
|
|
</script>
|
|
</html>
|