apaam/mlpack
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge pull request #3808 from rcurtin/doc-vptree

Browse Source 
Document `VPTree`
This commit is contained in:
Ryan Curtin
2024-10-22 17:37:41 -04:00
committed by GitHub
parent 94f5e1ea04 48333f9110
commit ca27fa52f7
9 changed files with 1066 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
HISTORY.md
+2
View File
@@ -13,6 +13,8 @@ _????-??-??_
* Fix compilation if only including `mlpack/methods/kde/kde_model.hpp` (#3800).
* Fix serialization and `MinDistance()` bugs with `HollowBallBound` (#3808).
## mlpack 4.5.0
_2024-09-17_
doc/img/hollowballbound.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 71 KiB

doc/sidebar.html
+5
View File
@@ -96,6 +96,11 @@ when the sidebar is built for each page.
<code>MeanSplitBallTree</code>
</a>
</li>
<li>
<a href="LINKROOTuser/core/trees/vptree.html">
<code>VPTree</code>
</a>
</li>
<li>
<a href="LINKROOTuser/core/trees/binary_space_tree.html">
<code>BinarySpaceTree</code>
doc/user/core/distributions.md
+1 -1
View File
@@ -849,7 +849,7 @@ regression model's prediction on `x`.
This class is meant to be used with mlpack's
[HMM](/src/mlpack/methods/hmm/hmm.hpp) class for the task of
[HMM regression (pdf)](https://conservancy.umn.edu/bitstream/handle/11299/2532/1195.pdf).
[HMM regression (pdf)](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=93a56eb64e77ac83404fddfd0036e95a742fcee6).
### Constructors
doc/user/core/trees/binary_space_tree.md
+416 -5
View File
@@ -413,7 +413,7 @@ class uses a hyperrectangle bound. An example `HRectBound` is shown below; the
bound is the smallest rectangle that encloses all of the points.
<center>
<img src="../../../img/hrectbound.png" width="50%" alt="hyperrectangle bound enclosing points">
<img src="../../../img/hrectbound.png" width="450" alt="hyperrectangle bound enclosing points">
</center>
mlpack supplies several drop-in `BoundType` classes, and it is also possible to
@@ -423,6 +423,8 @@ write a custom `BoundType` for use with `BinarySpaceTree`:
points in the smallest possible hyperrectangle
* [`BallBound`](#ballbound): ball bound, encloses the descendant points in the
ball with the smallest possible radius
* [`HollowBallBound`](#hollowballbound): hollow ball bound, equivalent to a
ball bound with a ball subtracted from it.
* [Custom `BoundType`s](#custom-boundtypes): implement a fully custom
`BoundType`
@@ -577,7 +579,7 @@ operations with data points or other bounds.
Once an `HRectBound` has been successfully created and set to the desired
bounding hyperrectangle, there are a number of functions that can bound the
distance between a `HRectBound` and other objects.
distance between an `HRectBound` and other objects.
* `b.Contains(point)`
* `b.Contains(bound)`
@@ -776,7 +778,7 @@ std::cout << "Distance between Manhattan distance HRectBound and "
// point.
arma::fmat floatData(3, 25, arma::fill::randu);
mlpack::HRectBound<mlpack::ChebyshevDistance, float> cb;
cb |= floatData; // This will set the bound to [2.0, 3.0] in every dimension.
cb |= floatData;
// Note the use of arma::fvec to represent a point, since ElemType is float.
const mlpack::RangeType<float> r3 = cb.RangeDistance(arma::fvec("1.5 1.5 4.0"));
std::cout << "Distance between Chebyshev distance HRectBound and "
@@ -840,8 +842,8 @@ Different constructor forms can be used to specify different template parameters
any points at all).
***Note***: these constructors provide an empty bound; be sure to
[grow](#growing-and-shrinking-the-bound) the bound or
[directly modify the bound](#accessing-and-modifying-properties-of-the-bound)
[grow](#growing-the-bound) the bound or
[directly modify the bound](#accessing-and-modifying-properties-of-the-bound-1)
before using it!
---
@@ -1078,6 +1080,368 @@ std::cout << "Distance between Chebyshev distance BallBound and "
---
### `HollowBallBound`
The `HollowBallBound` class represents a bounding shape that is an
arbitrary-dimensional ball bound with another smaller ball subtracted from its
inside. A `HollowBallBound` consists of a center point, an outer radius, and a
secondary center point and inner radius. An example `HollowBallBound` is shown
below in two dimensions; shaded area represents area held within the bound.
<center>
<img src="../../../img/hollowballbound.png" width="350" alt="hollow ball bound">
</center>
`HollowBallBound` is used directly by the [`VPTree`](vptree.md) class.
---
#### Constructors
`HollowBallBound` allows configurable behavior via its two template parameters:
```
HollowBallBound<DistanceType, ElemType>
```
Different constructor forms can be used to specify different template parameters
(and thus different bound behavior).
* `b = HollowBallBound(dimensionality)`
- Construct a `HollowBallBound` with the given `dimensionality`.
- The bound will be empty with invalid centers and radii (e.g., `b` will not
contain any points at all).
- The bound will use the [Euclidean distance](../distances.md#lmetric) for
distance computation, and will expect data to have elements with type
`double`.
* `b = HollowBallBound<DistanceType, ElemType>(dimensionality)`
- Construct a `HollowBallBound` with the given `dimensionality` that will use
the given `DistanceType` class to compute distances, and expect data to
have elements with type `ElemType`.
- `ElemType` should generally be `double` or `float`.
***Note***: these constructors provide an empty bound; be sure to
[grow](#growing-the-bound-1) the bound or
[directly modify the bound](#accessing-and-modifying-properties-of-the-bound-2)
before using it!
---
* `b = HollowBallBound(innerRadius, outerRadius, center)`
- Construct a `HollowBallBound` with the given `innerRadius` for the inner
ball, `outerRadius` for the outer ball, and `center`.
- Both the inner and outer ball are centered at `center`.
- `innerRadius` and `outerRadius` should have type `double`.
- `center` should have type `arma::vec`.
- The bound will use the [Euclidean distance](../distances.md#lmetric) for
distance computation, and will expect data to have elements with type
`double`.
* `b = HollowBallBound<DistanceType, ElemType>(innerRadius, outerRadius, center)`
- Construct a `HollowBallBound` with the given `innerRadius` for the inner
ball, `outerRadius` for the outer ball, and `center`.
- Both the inner and outer ball are centered at `center`.
- `innerRadius` and `outerRadius` should have type `ElemType`.
- `center` should be a vector with element type `ElemType` (e.g.
`arma::Col<ElemType>`).
- The bound will use the given `DistanceType` class to compute distances, and
expect data to have elements with type `ElemType`.
---
#### Accessing and modifying properties of the bound
The individual bounds associated with each dimension of a `HollowBallBound` can
be accessed and modified.
* `b.Dim()` will return a `size_t` indicating the dimensionality of the bound.
* `b.Center()` returns an `arma::vec&` containing the center of the outer ball.
Its elements can be directly modified.
* `b.HollowCenter()` returns an `arma::vec&` containing the center of the inner
ball. Its elements can be directly modified.
- It is possible that `b.HollowCenter()` is outside of the outer ball!
* `b.OuterRadius()` will return a `double` that is the radius of the outer
ball.
- `b.OuterRadius() = r` will set the radius of the outer ball to `r`.
* `b.InnerRadius()` will return a `double` that is the radius of the inner
ball.
- `b.InnerRadius() = r` will set the radius of the inner ball to `r`.
- It is possible that `b.InnerRadius() > b.OuterRadius()`, and this implies
that the hollow center is outside the outer ball (otherwise the bound is
empty).
* `b[dim]` will return a [`Range`](../math.md#range) object representing the
extents of the bound in dimension `dim`.
- The range is defined as
`[b.Center()[dim] - b.OuterRadius(), b.Center()[dim] + b.OuterRadius()]`.
- ***Note:*** this returns the maximum extents of the bound and does not
consider the inner (hollow) ball.
* `b.Diameter()` returns the diameter of the ball. This is always equal to
`2 * b.OuterRadius()`.
* `b.MinWidth()` returns the minimum width of the bound in any dimension as a
`double`. This is always equal to `b.Diameter()`.
* `b.Distance()` returns either a
[`EuclideanDistance`](../distances.md#lmetric) distance metric object, or a
`DistanceType` if a custom `DistanceType` has been specified in the
constructor.
* `b.Center(center)` will store the center of the `HollowBallBound` in the
vector `center`. `center` should be of type `arma::vec`.
* `b.MinWidth()` returns the minimum width of the bound in any dimension as a
`double`. This value is cached and no computation is performed when calling
`b.MinWidth()`. If the bound is empty, `0` is returned.
* `b.Distance()` returns either a
[`EuclideanDistance`](../distances.md#lmetric) distance metric object, or a
`DistanceType` if a custom `DistanceType` has been specified in the
constructor.
* `b.Center(center)` will compute the center of the `HollowBallBound` (e.g. the
vector with elements equal to the midpoint of `b` in each dimension) and
store it in the vector `center`. `center` should be of type `arma::vec`.
* `b.Volume()` computes the volume of the hyperrectangle specified by `b`. The
volume is returned as a `double`.
* `b.Diameter()` computes the longest diagonal of the hyperrectangle specified
by `b`.
* A `HollowBallBound` can be serialized with
[`data::Save()` and `data::Load()`](../../load_save.md#mlpack-objects).
***Note:*** if a custom `ElemType` was specified in the constructor, then:
* `b[dim]` will return a `RangeType<ElemType>`;
* `b.OuterRadius()`, `b.InnerRadius()`, `b.MinWidth()`, and `b.Diameter()` will
return `ElemType`;
* `b.Center()` and `b.HollowCenter()` will return `arma::Col<ElemType>&`; and
* `b.Center(center)` expects `center` to be of type `arma::Col<ElemType>`.
---
#### Growing the bound
The `HollowBallBound` uses the logical `|=` to grow the bound to include points
or other bounds.
* `b |= data` expands `b` so the outer ball includes all of the data points in
`data`, shrinking the inner ball as necessary. `data` should be a
[column-major `arma::mat`](../../matrices.md#representing-data-in-mlpack).
The expansion operation is minimal, so `b` is not expanded any more than
necessary.
- The bound is grown using [Jack Ritter's bounding sphere
algorithm](https://en.wikipedia.org/wiki/Bounding_sphere#Ritter's_bounding_sphere),
which may move the center of the bound as it iteratively adds points to the
bound. (The hollow center is not moved.)
- If the bound is empty, the centers are initialized to the first point of
`data`.
- If the bound is not empty, then `data` is expected to have dimensionality
that matches `b.Dim()`.
* `b |= bound` expands `b` to include all of the volume included in `bound`.
The center points will not be modified.
- The outer ball's radius will be expanded to include the outer balls of both
`b` and `bound`.
- The inner (hollow) ball's radius will be shrunk to be the intersection of
the inner balls of `b` and `bound`. (This may result in `b.InnerRadius()`
being 0.)
***Notes:***
- The growth operation does not grow the inner (hollow) ball. Properties
related to the inner ball should be set manually with `b.HollowCenter()` and
`b.InnerRadius()`.
- If a custom `ElemType` was specified, then any `data` argument should be a
matrix with that `ElemType` (e.g. `arma::Mat<ElemType>`).
---
#### Bounding distances to other objects
Once a `HollowBallBound` has been successfully created and set to the desired
bounding balls, there are a number of functions that can bound the
distance between a `HollowBallBound` and other objects.
* `b.Contains(point)`
* `b.Contains(bound)`
- Return a `bool` indicating whether or not `b` contains the given `point`
(an `arma::vec`) or another `bound` (an `HRectBound`).
- When passing another `bound`, `true` will be returned if `bound` even
partially overlaps with `b`.
* `b.MinDistance(point)`
* `b.MinDistance(bound)`
- Return a `double` whose value is the minimum possible distance between `b`
and either a `point` (an `arma::vec`) or another `bound` (a
`HollowBallBound`).
- The minimum distance between `b` and another point or bound is the length
of the shortest possible line that can connect the other point or bound to
`b`.
- If `point` or `bound` are contained in `b`, then the returned distance is
0.
* `b.MaxDistance(point)`
* `b.MaxDistance(bound)`
- Return a `double` whose value is the maximum possible distance between `b`
and either a `point` (an `arma::vec`) or another `bound` (a
`HollowBallBound`).
- The maximum distance between `b` and a given `point` is the furthest
possible distance between `point` and any possible point falling within the
bounding hyperrectangle of `b`.
- The maximum distance between `b` and another `bound` is the furthest
possible distance between any possible point falling within the bounding
hyperrectangle of `b`, and any possible point falling within the bounding
hyperrectangle of `bound`.
- Note that this definition means that even if `b.Contains(point)` or
`b.Contains(bound)` is `true`, the maximum distance may be greater than
`0`.
* `b.RangeDistance(point)`
* `b.RangeDistance(bound)`
- Compute the minimum and maximum distance between `b` and `point` or
`bound`, returning the result as a [`Range`](../math.md#range) object.
- This is more efficient than calling `b.MinDistance()` and
`b.MaxDistance()`.
***Note:*** if a custom `DistanceType` and `ElemType` were specified in the
constructor, then all distances will be computed with respect to the specified
`DistanceType` and all return values will either be `ElemType` or
[`RangeType<ElemType>`](../math.md#range) (except for `Contains()`, which will
still return a `bool`).
---
#### Example usage
```c++
// Create a hollow ball bound in 3 dimensions whose outer ball is the unit ball
// and whose inner ball is the ball with radius 0.5 centered at the origin.
// The bounding range for all three dimensions is [0.0, 1.0].
mlpack::HollowBallBound b(0.5, 1.0, arma::vec(3));
std::cout << "Hollow unit ball bound created manually:" << std::endl;
std::cout << " - Center: " << b.Center().t();
std::cout << " - Outer radius: " << b.OuterRadius() << "." << std::endl;
std::cout << " - Hollow center: " << b.HollowCenter().t();
std::cout << " - Inner radius: " << b.InnerRadius() << "." << std::endl;
for (size_t i = 0; i < 3; ++i)
{
std::cout << " - Dimension " << i << " extents: [" << b[i].Lo() << ", "
<< b[i].Hi() << "]." << std::endl;
}
std::cout << std::endl;
// Create a small dataset of 5 points.
arma::mat dataset(3, 5);
dataset.col(0) = arma::vec("2.0 2.0 2.0");
dataset.col(1) = arma::vec("2.5 2.5 2.5");
dataset.col(2) = arma::vec("3.0 2.0 3.0");
dataset.col(3) = arma::vec("2.0 3.0 2.0");
dataset.col(4) = arma::vec("3.0 3.0 3.0");
// If we simply build a HollowBallBound to enclose those points, the hollow part
// of the ball is unmodified and remains empty.
mlpack::HollowBallBound b2(3);
b2 |= dataset;
std::cout << "Hollow ball bound on points with only `operator|=()`:"
<< std::endl;
std::cout << " - Center: " << b2.Center().t();
std::cout << " - Outer radius: " << b2.OuterRadius() << "." << std::endl;
std::cout << " - Hollow center: " << b2.HollowCenter().t();
std::cout << " - Inner radius: " << b2.InnerRadius() << "." << std::endl;
std::cout << std::endl;
// On the other hand, if we initialize a HollowBallBound to a non-empty bound,
// then `operator|=()` will shrink the hollow ball as necessary.
//
// We initialize this ball bound to a "slice" with radii [3.6, 3.7].
mlpack::HollowBallBound b3(3.6, 3.7, arma::vec(3));
b3 |= dataset;
std::cout << "Hollow ball bound on points with pre-initialization and "
<< "`operator|=()`:" << std::endl;
std::cout << " - Center: " << b3.Center().t();
std::cout << " - Outer radius: " << b3.OuterRadius() << "." << std::endl;
std::cout << " - Hollow center: " << b3.HollowCenter().t();
std::cout << " - Inner radius: " << b3.InnerRadius() << "." << std::endl;
std::cout << std::endl;
// Manually create a hollow ball bound whose hollow center is different than the
// outer ball's center.
mlpack::HollowBallBound b4(3);
b4.OuterRadius() = 3.0;
b4.InnerRadius() = 1.5;
b4.Center() = arma::vec(3);
b4.HollowCenter() = arma::vec("1.0 1.0 1.0");
// Compute the minimum distance between a point inside the hollow unit ball's
// outer ball.
const double d1 = b.MinDistance(arma::vec("0.9 0.9 0.9"));
std::cout << "Minimum distance between hollow unit ball bound and [0.9, 0.9, "
<< "0.9]: " << d1 << "." << std::endl;
// Compute the minimum distance between a point inside the hollow unit ball's
// inner ball (so the point is not contained in the bound---it is within the
// hollow section).
const double d2 = b.MinDistance(arma::vec("0.0 0.0 0.0"));
std::cout << "Minimum distance between hollow unit ball bound and [0.0, 0.0, "
<< "0.0]: " << d2 << "." << std::endl;
std::cout << std::endl;
// Use Contains(). In this case, the 'else' will be taken.
if (b.Contains(arma::vec("1.5 1.5 1.5")))
{
std::cout << "Hollow unit ball bound contains [1.5, 1.5, 1.5]." << std::endl;
}
else
{
std::cout << "Hollow unit ball bound does not contain [1.5, 1.5, 1.5]."
<< std::endl;
}
std::cout << std::endl;
// Compute the maximum distance between a point inside the unit ball and the
// unit hollow ball bound.
const double d3 = b4.MaxDistance(arma::vec("0.1 0.1 0.1"));
std::cout << "Maximum distance between hollow unit ball bound and [0.1, 0.1, "
<< "0.1]: " << d3 << "." << std::endl;
// Compute the minimum and maximum distances between the hollow unit ball bound
// and the bound built on data points.
const mlpack::Range r = b.RangeDistance(b3);
std::cout << "Distances between hollow unit ball bound and second hollow "
<< "dataset bound: [" << r.Lo() << ", " << r.Hi() << "]." << std::endl;
// Create a bound using the Manhattan (L1) distance and compute the minimum and
// maximum distance to a point.
mlpack::HollowBallBound<mlpack::ManhattanDistance> mb(2.0, 5.0, arma::vec(3));
const mlpack::Range r2 = mb.RangeDistance(arma::vec("1.5 1.5 4.0"));
std::cout << "Distance between Manhattan distance HollowBallBound and "
<< "[1.5, 1.5, 4.0]: [" << r2.Lo() << ", " << r2.Hi() << "]." << std::endl;
// Create a bound using the Chebyshev (L-inf) distance, using random 32-bit
// floating point elements, and compute the minimum and maximum distance to a
// point.
arma::fmat floatData(3, 25, arma::fill::randu);
mlpack::HollowBallBound<mlpack::ChebyshevDistance, float> cb;
cb |= floatData;
// Note the use of arma::fvec to represent a point, since ElemType is float.
const mlpack::RangeType<float> r3 = cb.RangeDistance(arma::fvec("1.5 1.5 4.0"));
std::cout << "Distance between Chebyshev distance HollowBallBound and "
<< "[1.5, 1.5, 4.0]: [" << r3.Lo() << ", " << r3.Hi() << "]." << std::endl;
```
---
### Custom `BoundType`s
The `BinarySpaceTree` class allows an arbitrary `BoundType` template parameter
@@ -1261,6 +1625,8 @@ to write a fully custom split:
with maximum width
* [`MeanSplit`](#meansplit): splits on the mean value of the points in the
dimension with maximum width
* [`VantagePointSplit`](#vantagepointsplit): split by selecting a 'vantage
point' and then split points into 'near' and 'far' sets
* [Custom `SplitType`s](#custom-splittypes): implement a fully custom
`SplitType` class
@@ -1309,6 +1675,51 @@ task*.
For implementation details, see
[the source code](/src/mlpack/core/tree/binary_space_tree/mean_split_impl.hpp).
### `VantagePointSplit`
The `VantagePointSplit` class is a splitting strategy that can be used by
[`BinarySpaceTree`](#binaryspacetree). It is the default strategy for splitting
[`VPTree`s](vptree.md), and is detailed in
[the paper](https://www.mlpack.org/papers/uhlmann91.pdf).
Due to the nature of the split, ***`VantagePointSplit` should always be used
with the [`HollowBallBound`](#hollowballbound)***.
The splitting strategy for the `VantagePointSplit` class is, given a set of
points:
* Select a vantage point from a sample of 100 random candidate points (or use
the full set if there are fewer than 100 points):
- Compute the distances between each candidate point and 100 additional
random samples (or the full set if there are fewer than 100 points).
- Select the vantage point as the candidate with maximum average distance to
the additional random samples.
* Compute a boundary distance `mu` that is the median distance between the
vantage point and its random samples.
* Points with distance less than `mu` from the vantage point will go to the
left child.
* Points with distance greater than `mu` from the vantage point will go to the
right child.
The `VantagePointSplit` class has three template parameters:
```
VantagePointSplit<BoundType, MatType, MaxNumSamples = 100>
```
If a custom number of samples `S` is desired, the easiest way to specify is via
a template typedef:
```
template<typename BoundType, typename MatType>
using MyVantagePointSplit = VantagePointSplit<BoundType, MatType, S>;
```
Then, `MyVantagePointSplit` can be used directly with `BinarySpaceTree` as a
`SplitType`.
For implementation details, see
[the source code](/src/mlpack/core/tree/binary_space_tree/vantage_point_split_impl.hpp).
### Custom `SplitType`s
Custom split strategies for a binary space tree can be implemented via the
doc/user/core/trees/vptree.md
+634
View File
@@ -0,0 +1,634 @@
# `VPTree`
<!-- TODO: link to knn.md once it's done -->
The `VPTree` class represents a `k`-dimensional vantage point tree,
and is a well-known data structure for efficient distance operations (such as
nearest neighbor search) in low dimensions---typically less than 100. The
vantage point tree is also known as the 'metric tree'.
A vantage point tree is a binary tree where each node selects a 'vantage
point', and child nodes partition points into those that are nearer to the
vantage point and those that are further from it. `VPTree` supports arbitrary
[distance metrics](../distances.md) that are not
[`LMetric`](../distances.md#lmetric), making it more flexible than `KDTree`.
mlpack's `VPTree` implementation supports three template parameters for
configurable behavior, and implements all the functionality required by the
[TreeType API](../../../developer/trees.md#the-treetype-api), plus some
additional functionality specific to vantage point trees.
* [Template parameters](#template-parameters)
* [Constructors](#constructors)
* [Basic tree properties](#basic-tree-properties)
* [Bounding distances with the tree](#bounding-distances-with-the-tree)
* [Tree traversals](#tree-traversals)
* [Example usage](#example-usage)
## See also
<!-- TODO: add links to all distance-based algorithms and other trees? -->
* [Vantage point tree on Wikipedia](https://en.wikipedia.org/wiki/Vantage-point_tree)
* [`BinarySpaceTree`](binary_space_tree.md)
* [Binary space partitioning on Wikipedia](https://dl.acm.org/doi/pdf/10.1145/361002.361007)
* [Data structures and algorithms for nearest neighbor search in general metric spaces (pdf)](https://dl.acm.org/doi/pdf/10.5555/313559.313789)
* [Satisfying General Proximity/Similarity Queries with Metric Trees (pdf)](https://www.mlpack.org/papers/uhlmann91.pdf)
* [Tree-Independent Dual-Tree Algorithms (pdf)](https://www.ratml.org/pub/pdf/2013tree.pdf)
## Template parameters
In accordance with the [TreeType
API](../../../developer/trees.md#template-parameters-required-by-the-treetype-policy)
(see also [this more detailed section](../../../developer/trees.md#template-parameters)),
the `VPTree` class takes three template parameters:
```
VPTree<DistanceType, StatisticType, MatType>
```
* `DistanceType`: the [distance metric](../distances.md) to use for distance
computations. By default, this is
[`EuclideanDistance`](../distances.md#lmetric).
* [`StatisticType`](binary_space_tree.md#statistictype): this holds auxiliary
information in each tree node. By default,
[`EmptyStatistic`](binary_space_tree.md#emptystatistic) is used, which holds
no information.
* `MatType`: the type of matrix used to represent points. Must be a type
matching the [Armadillo API](../../matrices.md). By default, `arma::mat` is
used, but other types such as `arma::fmat` or similar will work just fine.
The `VPTree` class itself is a convenience typedef of the generic
[`BinarySpaceTree`](binary_space_tree.md) class, using the
[`HollowBallBound`](binary_space_tree.md#hollowballbound) class as the bounding
structure, and using the
[`VantagePointSplit`](binary_space_tree.md#vantagepointsplit) splitting strategy
for construction, which splits points into those that are nearer and further
from a 'vantage point'.
## Constructors
`VPTree`s are efficiently constructed by permuting points in a dataset in a
quicksort-like algorithm. However, this means that the ordering of points in
the tree's dataset (accessed with `node.Dataset()`) after construction may be
different.
---
* `node = VPTree(data, maxLeafSize=20)`
* `node = VPTree(data, oldFromNew, maxLeafSize=20)`
* `node = VPTree(data, oldFromNew, newFromOld, maxLeafSize=20)`
- Construct a `VPTree` on the given `data`, using `maxLeafSize` as the
maximum number of points held in a leaf.
- By default, `data` is copied. Avoid a copy by using `std::move()` (e.g.
`std::move(data)`); when doing this, `data` will be set to an empty matrix.
- Optionally, construct mappings from old points to new points. `oldFromNew`
and `newFromOld` will have length `data.n_cols`, and:
* `oldFromNew[i]` indicates that point `i` in the tree's dataset was
originally point `oldFromNew[i]` in `data`; that is,
`node.Dataset().col(i)` is the point `data.col(oldFromNew[i])`.
* `newFromOld[i]` indicates that point `i` in `data` is now point
`newFromOld[i]` in the tree's dataset; that is,
`node.Dataset().col(newFromOld[i])` is the point `data.col(i)`.
---
* `node = VPTree<DistanceType, StatisticType, MatType>(data, maxLeafSize=20)`
* `node = VPTree<DistanceType, StatisticType, MatType>(data, oldFromNew, maxLeafSize=20)`
* `node = VPTree<DistanceType, StatisticType, MatType>(data, oldFromNew, newFromOld, maxLeafSize=20)`
- Construct a `VPTree` on the given `data`, using custom template parameters
to control the behavior of the tree, using `maxLeafSize` as the maximum
number of points held in a leaf.
- By default, `data` is copied. Avoid a copy by using `std::move()` (e.g.
`std::move(data)`); when doing this, `data` will be set to an empty matrix.
- Optionally, construct mappings from old points to new points. `oldFromNew`
and `newFromOld` will have length `data.n_cols`, and:
* `oldFromNew[i]` indicates that point `i` in the tree's dataset was
originally point `oldFromNew[i]` in `data`; that is,
`node.Dataset().col(i)` is the point `data.col(oldFromNew[i])`.
* `newFromOld[i]` indicates that point `i` in `data` is now point
`newFromOld[i]` in the tree's dataset; that is,
`node.Dataset().col(newFromOld[i])` is the point `data.col(i)`.
---
* `node = VPTree()`
- Construct an empty vantage point tree with no children and no points.
---
***Notes:***
- The name `node` is used here for `VPTree` objects instead of `tree`, because
each `VPTree` object is a single node in the tree. The constructor returns
the node that is the root of the tree.
- Inserting individual points or removing individual points from a `VPTree` is
not supported, because this generally results in a vantage point tree with
very loose bounding balls. It is better to simply build a new `VPTree` on
the modified dataset. For trees that support individual insertion and
deletions, see the `RectangleTree` class and all its variants (e.g. `RTree`,
`RStarTree`, etc.).
- See also the
[developer documentation on tree constructors](../../../developer/trees.md#constructors-and-destructors).
<!-- TODO: add links to RectangleTree above when it is documented -->
---
### Constructor parameters:
| **name** | **type** | **description** | **default** |
|----------|----------|-----------------|-------------|
| `data` | [`arma::mat`](../../matrices.md) | [Column-major](../../matrices.md#representing-data-in-mlpack) matrix to build the tree on. Pass with `std::move(data)` to avoid copying the matrix. | _(N/A)_ |
| `maxLeafSize` | `size_t` | Maximum number of points to store in each leaf. | `20` |
| `oldFromNew` | `std::vector<size_t>` | Mappings from points in `node.Dataset()` to points in `data`. | _(N/A)_ |
| `newFromOld` | `std::vector<size_t>` | Mappings from points in `data` to points in `node.Dataset()`. | _(N/A)_ |
## Basic tree properties
Once a `VPTree` object is constructed, various properties of the tree can be
accessed or inspected. Many of these functions are required by the [TreeType
API](../../../developer/trees.md#the-treetype-api).
### Navigating the tree
* `node.NumChildren()` returns the number of children in `node`. This is
either `2` if `node` has children, or `0` if `node` is a leaf.
* `node.IsLeaf()` returns a `bool` indicating whether or not `node` is a leaf.
* `node.Child(i)` returns a `VPTree&` that is the `i`th child.
- `i` must be `0` or `1`.
- This function should only be called if `node.NumChildren()` is not `0`
(e.g. if `node` is not a leaf). Note that this returns a valid `VPTree&`
that can itself be used just like the root node of the tree!
- `node.Left()` and `node.Right()` are convenience functions specific to
`VPTree` that will return `VPTree*` (pointers) to the left and right
children, respectively, or `NULL` if `node` has no children.
* `node.Parent()` will return a `VPTree*` that points to the parent of `node`,
or `NULL` if `node` is the root of the `VPTree`.
---
### Accessing members of a tree
* `node.Bound()` will return an
[`HollowBallBound&`](binary_space_tree.md#hollowballbound) object that
represents the hollow bounding ball of `node`. This structure encloses all
the descendant points of `node`.
* `node.Stat()` will return an `EmptyStatistic&` (or a `StatisticType&` if a
[custom `StatisticType`](#template-parameters) was specified as a template
parameter) holding the statistics of the node that were computed during tree
construction.
* `node.Distance()` will return a
[`EuclideanDistance&`](../distances.md#lmetric) (or a `DistanceType&` if a
[custom `DistanceType`](#template-parameters) was specified as a template
parameter).
See also the
[developer documentation](../../../developer/trees.md#basic-tree-functionality)
for basic tree functionality in mlpack.
---
### Accessing data held in a tree
* `node.Dataset()` will return a `const arma::mat&` that is the dataset the
tree was built on. Note that this is a permuted version of the `data` matrix
passed to the constructor.
- If a [custom `MatType`](#template-parameters) is being used, the return
type will be `const MatType&` instead of `const arma::mat&`.
* `node.NumPoints()` returns a `size_t` indicating the number of points held
directly in `node`.
- If `node` is not a leaf, this will return `0`, as `VPTree` only holds
points directly in its leaves.
- If `node` is a leaf, then the number of points will be less than or equal
to the `maxLeafSize` that was specified when the tree was constructed.
* `node.Point(i)` returns a `size_t` indicating the index of the `i`'th point
in `node.Dataset()`.
- `i` must be in the range `[0, node.NumPoints() - 1]` (inclusive).
- `node` must be a leaf (as non-leaves do not hold any points).
- The `i`'th point in `node` can then be accessed as
`node.Dataset().col(node.Point(i))`.
- In a `VPTree`, because of the permutation of points done [during
construction](#constructors), point indices are contiguous:
`node.Point(i + j)` is the same as `node.Point(i) + j` for valid `i` and
`j`.
- Accessing the actual `i`'th point itself can be done with, e.g.,
`node.Dataset().col(node.Point(i))`.
* `node.NumDescendants()` returns a `size_t` indicating the number of points
held in all descendant leaves of `node`.
- If `node` is the root of the tree, then `node.NumDescendants()` will be
equal to `node.Dataset().n_cols`.
* `node.Descendant(i)` returns a `size_t` indicating the index of the `i`'th
descendant point in `node.Dataset()`.
- `i` must be in the range `[0, node.NumDescendants() - 1]` (inclusive).
- `node` does not need to be a leaf.
- The `i`'th descendant point in `node` can then be accessed as
`node.Dataset().col(node.Descendant(i))`.
- In a `VPTree`, because of the permutation of points done [during
construction](#constructors), point indices are contiguous:
`node.Descendant(i + j)` is the same as `node.Descendant(i) + j` for valid
`i` and `j`.
- Accessing the actual `i`'th descendant itself can be done with, e.g.,
`node.Dataset().col(node.Descendant(i))`.
* `node.Begin()` returns a `size_t` indicating the index of the first
descendant point of `node`.
- This is equivalent to `node.Descendant(0)`.
* `node.Count()` returns a `size_t` indicating the number of descendant points of `node`.
- This is equivalent to `node.NumDescendants()`.
---
### Accessing computed bound quantities of a tree
The following quantities are cached for each node in a `VPTree`, and so
accessing them does not require any computation.
* `node.FurthestPointDistance()` returns a `double` representing the distance
between the center of the hollow bounding ball of `node` and the furthest
point held by `node`.
- If `node` is not a leaf, this returns 0 (because `node` does not hold any
points).
* `node.FurthestDescendantDistance()` returns a `double` representing the
distance between the center of the hollow bounding ball of `node` and the
furthest descendant point held by `node`.
* `node.MinimumBoundDistance()` returns a `double` representing minimum
possible distance from the center of the node to any edge of the
hollow ball bound.
- This quantity is equivalent to `node.Bound().OuterRadius()`.
* `node.ParentDistance()` returns a `double` representing the distance between
the center of the hollow bounding ball of `node` and the center of the
hollow bounding ball of its parent.
- If `node` is the root of the tree, `0` is returned.
***Notes:***
- If a [custom `MatType`](#template-parameters) was specified when constructing
the `VPTree`, then the return type of each method is the element type of the
given `MatType` instead of `double`. (e.g., if `MatType` is `arma::fmat`,
then the return type is `float`.)
- For more details on each bound quantity, see the
[developer documentation](../../../developer/trees.md#complex-tree-functionality-and-bounds)
on bound quantities for trees.
---
### Other functionality
* `node.Center(center)` computes the center of the hollow bounding ball of
`node` and stores it in `center`.
- `center` should be of type `arma::vec&`. (If a [custom
`MatType`](#template-parameters) was specified when constructing the
`VPTree`, the type is instead the column vector type for the given
`MatType`; e.g., `arma::fvec&` when `MatType` is `arma::fmat`.)
- `center` will be set to have size equivalent to the dimensionality of the
dataset held by `node`.
- This is equivalent to calling `node.Bound().Center(center)`.
* A `VPTree` can be serialized with
[`data::Save()` and `data::Load()`](../../load_save.md#mlpack-objects).
## Bounding distances with the tree
The primary use of trees in mlpack is bounding distances to points or other tree
nodes. The following functions can be used for these tasks.
* `node.GetNearestChild(point)`
* `node.GetFurthestChild(point)`
- Return a `size_t` indicating the index of the child (`0` for left, `1` for
right) that is closest to (or furthest from) `point`, with respect
to the `MinDistance()` (or `MaxDistance()`) function.
- If there is a tie, `0` (the left child) is returned.
- If `node` is a leaf, `0` is returned.
- `point` should be of type `arma::vec`. (If a [custom
`MatType`](#template-parameters) was specified when constructing the
`VPTree`, the type is instead the column vector type for the given
`MatType`; e.g., `arma::fvec` when `MatType` is `arma::fmat`.)
* `node.GetNearestChild(other)`
* `node.GetFurthestChild(other)`
- Return a `size_t` indicating the index of the child (`0` for left, `1` for
right) that is closest to (or furthest from) the `VPTree` node `other`,
with respect to the `MinDistance()` (or `MaxDistance()`) function.
- If there is a tie, `2` (an invalid index) is returned. ***Note that this
behavior differs from the version above that takes a point.***
- If `node` is a leaf, `0` is returned.
---
* `node.MinDistance(point)`
* `node.MinDistance(other)`
- Return a `double` indicating the minimum possible distance between `node`
and `point`, or the `VPTree` node `other`.
- This is equivalent to the minimum possible distance between any point
contained in the hollow bounding ball of `node` and `point`, or between
any point contained in the hollow bounding ball of `node` and any point
contained in the hollow bounding ball of `other`.
- `point` should be of type `arma::vec`. (If a [custom
`MatType`](#template-parameters) was specified when constructing the
`VPTree`, the type is instead the column vector type for the given
`MatType`, and the return type is the element type of `MatType`; e.g.,
`point` should be `arma::fvec` when `MatType` is `arma::fmat`, and the
returned distance is `float`).
* `node.MaxDistance(point)`
* `node.MaxDistance(other)`
- Return a `double` indicating the maximum possible distance between `node`
and `point`, or the `VPTree` node `other`.
- This is equivalent to the maximum possible distance between any point
contained in the hollow bounding ball of `node` and `point`, or between
any point contained in the hollow bounding ball of `node` and any point
contained in the hollow bounding ball of `other`.
- `point` should be of type `arma::vec`. (If a [custom
`MatType`](#template-parameters) was specified when constructing the
`VPTree`, the type is instead the column vector type for the given
`MatType`, and the return type is the element type of `MatType`; e.g.,
`point` should be `arma::fvec` when `MatType` is `arma::fmat`, and the
returned distance is `float`).
* `node.RangeDistance(point)`
* `node.RangeDistance(other)`
- Return a [`Range`](../math.md#range) whose lower bound is
`node.MinDistance(point)` or `node.MinDistance(other)`, and whose upper
bound is `node.MaxDistance(point)` or `node.MaxDistance(other)`.
- `point` should be of type `arma::vec`. (If a
[custom `MatType`](#template-parameters) was specified when constructing
the `VPTree`, the type is instead the column vector type for the given
`MatType`, and the return type is a `RangeType` with element type the same
as `MatType`; e.g., `point` should be `arma::fvec` when `MatType` is
`arma::fmat`, and the returned type is
[`RangeType<float>`](../math.md#range)).
### Tree traversals
Like every mlpack tree, the `VPTree` class provides a [single-tree and dual-tree
traversal](../../../developer/trees.md#traversals) that can be paired with a
[`RuleType` class](../../../developer/trees.md#rules) to implement a single-tree
or dual-tree algorithm.
* `VPTree::SingleTreeTraverser`
- Implements a depth-first single-tree traverser.
* `VPTree::DualTreeTraverser`
- Implements a dual-depth-first dual-tree traverser.
In addition to those two classes, which are required by the
[`TreeType` policy](../../../developer/trees.md), an additional traverser is
available:
* `VPTree::BreadthFirstDualTreeTraverser`
- Implements a dual-breadth-first dual-tree traverser.
- ***Note:*** this traverser is not useful for all tasks; because the
`VPTree` only holds points in the leaves, this means that no base cases
(e.g. comparisons between points) will be called until *all* pairs of
intermediate nodes have been scored!
## Example usage
Build a `VPTree` on the `cloud` dataset and print basic statistics about the
tree.
```c++
// See https://datasets.mlpack.org/cloud.csv.
arma::mat dataset;
mlpack::data::Load("cloud.csv", dataset, true);
// Build the vp-tree with a leaf size of 10. (This means that nodes are split
// until they contain 10 or fewer points.)
//
// The std::move() means that `dataset` will be empty after this call, and no
// data will be copied during tree building.
//
// Note that the '<>' isn't necessary if C++20 is being used (e.g.
// `mlpack::VPTree tree(...)` will work fine in C++20 or newer).
mlpack::VPTree<> tree(std::move(dataset));
// Print the bounding ball of the root node. (There will be no hollow ball.)
std::cout << "Bounding ball of root node:" << std::endl;
std::cout << " - Center: " << tree.Bound().Center().t();
std::cout << " - Outer radius: " << tree.Bound().OuterRadius() << "."
<< std::endl;
std::cout << " - Hollow center: " << tree.Bound().HollowCenter().t();
std::cout << " - Inner radius: " << tree.Bound().InnerRadius() << "."
<< std::endl;
std::cout << std::endl;
// Print the bounding ball of the right child. (This will have a hollow ball.)
std::cout << "Bounding ball of right child: " << std::endl;
std::cout << " - Center: " << tree.Right()->Bound().Center().t();
std::cout << " - Outer radius: " << tree.Right()->Bound().OuterRadius() << "."
<< std::endl;
std::cout << " - Hollow center: " << tree.Right()->Bound().HollowCenter().t();
std::cout << " - Inner radius: " << tree.Right()->Bound().InnerRadius() << "."
<< std::endl;
std::cout << " - Distance between centers: " <<
mlpack::EuclideanDistance::Evaluate(tree.Right()->Bound().Center(),
tree.Right()->Bound().HollowCenter()) << "." << std::endl;
std::cout << std::endl;
// Print the number of descendant points of the root, and of each of its
// children.
std::cout << "Descendant points of root: "
<< tree.NumDescendants() << "." << std::endl;
std::cout << "Descendant points of left child: "
<< tree.Left()->NumDescendants() << "." << std::endl;
std::cout << "Descendant points of right child: "
<< tree.Right()->NumDescendants() << "." << std::endl;
std::cout << std::endl;
// Print the center of the vp-tree.
arma::vec center;
tree.Center(center);
std::cout << "Center of vp-tree: " << center.t();
```
---
Build two `VPTree`s on subsets of the corel dataset and compute various
bounding quantities.
```c++
// See https://datasets.mlpack.org/corel-histogram.csv.
arma::mat dataset;
mlpack::data::Load("corel-histogram.csv", dataset, true);
// Build vp-trees on the first half and the second half of points.
mlpack::VPTree<> tree1(dataset.cols(0, dataset.n_cols / 2));
mlpack::VPTree<> tree2(dataset.cols(dataset.n_cols / 2 + 1,
dataset.n_cols - 1));
// Compute the maximum distance between the trees.
std::cout << "Maximum distance between tree root nodes: "
<< tree1.MaxDistance(tree2) << "." << std::endl;
// Get the leftmost grandchild of the first tree's root---if it exists.
if (!tree1.IsLeaf() && !tree1.Child(0).IsLeaf())
{
mlpack::VPTree<>& node1 = tree1.Child(0).Child(0);
// Get the rightmost grandchild of the second tree's root---if it exists.
if (!tree2.IsLeaf() && !tree2.Child(1).IsLeaf())
{
mlpack::VPTree<>& node2 = tree2.Child(1).Child(1);
// Print the minimum and maximum distance between the nodes.
mlpack::Range dists = node1.RangeDistance(node2);
std::cout << "Possible distances between two grandchild nodes: ["
<< dists.Lo() << ", " << dists.Hi() << "]." << std::endl;
// Print the minimum distance between the first node and the first
// descendant point of the second node.
const size_t descendantIndex = node2.Descendant(0);
const double descendantMinDist =
node1.MinDistance(node2.Dataset().col(descendantIndex));
std::cout << "Minimum distance between grandchild node and descendant "
<< "point: " << descendantMinDist << "." << std::endl;
// Which child of node2 is closer to node1?
const size_t closerIndex = node2.GetNearestChild(node1);
if (closerIndex == 0)
std::cout << "The left child of node2 is closer to node1." << std::endl;
else if (closerIndex == 1)
std::cout << "The right child of node2 is closer to node1." << std::endl;
else // closerIndex == 2 in this case.
std::cout << "Both children of node2 are equally close to node1."
<< std::endl;
// And which child of node1 is further from node2?
const size_t furtherIndex = node1.GetFurthestChild(node2);
if (furtherIndex == 0)
std::cout << "The left child of node1 is further from node2."
<< std::endl;
else if (furtherIndex == 1)
std::cout << "The right child of node1 is further from node2."
<< std::endl;
else // furtherIndex == 2 in this case.
std::cout << "Both children of node1 are equally far from node2."
<< std::endl;
}
}
```
---
Build a `VPTree` on 32-bit floating point data and save it to disk.
```c++
// See https://datasets.mlpack.org/corel-histogram.csv.
arma::fmat dataset;
mlpack::data::Load("corel-histogram.csv", dataset);
// Build the VPTree using 32-bit floating point data as the matrix type.
// We will still use the default EmptyStatistic and EuclideanDistance
// parameters. A leaf size of 100 is used here.
mlpack::VPTree<mlpack::EuclideanDistance,
mlpack::EmptyStatistic,
arma::fmat> tree(std::move(dataset), 100);
// Save the VPTree to disk with the name 'tree'.
mlpack::data::Save("tree.bin", "tree", tree);
std::cout << "Saved tree with " << tree.Dataset().n_cols << " points to "
<< "'tree.bin'." << std::endl;
```
---
Load a 32-bit floating point `VPTree` from disk, then traverse it manually and
find the number of leaf nodes with less than 10 children.
```c++
// This assumes the tree has already been saved to 'tree.bin' (as in the example
// above).
// This convenient typedef saves us a long type name!
typedef mlpack::VPTree<mlpack::EuclideanDistance,
mlpack::EmptyStatistic,
arma::fmat> TreeType;
TreeType tree;
mlpack::data::Load("tree.bin", "tree", tree);
std::cout << "Tree loaded with " << tree.NumDescendants() << " points."
<< std::endl;
// Recurse in a depth-first manner. Count both the total number of leaves, and
// the number of leaves with less than 10 points.
size_t leafCount = 0;
size_t totalLeafCount = 0;
std::stack<TreeType*> stack;
stack.push(&tree);
while (!stack.empty())
{
TreeType* node = stack.top();
stack.pop();
if (node->NumPoints() < 10)
++leafCount;
++totalLeafCount;
if (!node->IsLeaf())
{
stack.push(node->Left());
stack.push(node->Right());
}
}
// Note that it would be possible to use TreeType::SingleTreeTraverser to
// perform the recursion above, but that is more well-suited for more complex
// tasks that require pruning and other non-trivial behavior; so using a simple
// stack is the better option here.
// Print the results.
std::cout << leafCount << " out of " << totalLeafCount << " leaves have less "
<< "than 10 points." << std::endl;
```
---
Build a `VPTree` and map between original points and new points.
```c++
// See https://datasets.mlpack.org/cloud.csv.
arma::mat dataset;
mlpack::data::Load("cloud.csv", dataset, true);
// Build the tree.
std::vector<size_t> oldFromNew, newFromOld;
mlpack::VPTree<> tree(dataset, oldFromNew, newFromOld);
// oldFromNew and newFromOld will be set to the same size as the dataset.
std::cout << "Number of points in dataset: " << dataset.n_cols << "."
<< std::endl;
std::cout << "Size of oldFromNew: " << oldFromNew.size() << "." << std::endl;
std::cout << "Size of newFromOld: " << newFromOld.size() << "." << std::endl;
std::cout << std::endl;
// See where point 42 in the tree's dataset came from.
std::cout << "Point 42 in the permuted tree's dataset:" << std::endl;
std::cout << " " << tree.Dataset().col(42).t();
std::cout << "Was originally point " << oldFromNew[42] << ":" << std::endl;
std::cout << " " << dataset.col(oldFromNew[42]).t();
std::cout << std::endl;
// See where point 7 in the original dataset was mapped.
std::cout << "Point 7 in original dataset:" << std::endl;
std::cout << " " << dataset.col(7).t();
std::cout << "Mapped to point " << newFromOld[7] << ":" << std::endl;
std::cout << " " << tree.Dataset().col(newFromOld[7]).t();
```
src/mlpack/core/tree/binary_space_tree/typedef.hpp
+3 -1
View File
@@ -198,7 +198,9 @@ template<typename BoundType,
typename MatType = arma::mat>
using VPTreeSplit = VantagePointSplit<BoundType, MatType, 100>;
template<typename DistanceType, typename StatisticType, typename MatType>
template<typename DistanceType = EuclideanDistance,
typename StatisticType = EmptyStatistic,
typename MatType = arma::mat>
using VPTree = BinarySpaceTree<DistanceType,
StatisticType,
MatType,
src/mlpack/core/tree/binary_space_tree/vantage_point_split_impl.hpp
-1
View File
@@ -89,7 +89,6 @@ SelectVantagePoint(const DistanceType& distance, const MatType& data,
mu = arma::median(distances);
}
}
assert(bestSpread > 0);
}
} // namespace mlpack
src/mlpack/core/tree/hollow_ball_bound_impl.hpp
+5 -3
View File
@@ -316,8 +316,8 @@ RangeType<ElemType> HollowBallBound<TDistanceType, ElemType>::RangeDistance(
typename std::enable_if_t<IsVector<VecType>::value>* /* junk */) const
{
if (radii.Hi() < 0)
return Range(std::numeric_limits<ElemType>::max(),
std::numeric_limits<ElemType>::max());
return RangeType<ElemType>(std::numeric_limits<ElemType>::max(),
std::numeric_limits<ElemType>::max());
else
{
RangeType<ElemType> range;
@@ -461,7 +461,7 @@ void HollowBallBound<TDistanceType, ElemType>::serialize(
ar(CEREAL_NVP(radii));
ar(CEREAL_NVP(center));
ar(CEREAL_NVP(hollowCenter));
ar(CEREAL_POINTER(distance));
if (cereal::is_loading<Archive>())
{
// If we're loading, delete the local distance since we'll have a new one.
@@ -470,6 +470,8 @@ void HollowBallBound<TDistanceType, ElemType>::serialize(
ownsDistance = true;
}
ar(CEREAL_POINTER(distance));
}
} // namespace mlpack