diff --git a/HISTORY.md b/HISTORY.md index bfbc8e6c74..7e45158018 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -13,6 +13,8 @@ _????-??-??_ * Fix compilation if only including `mlpack/methods/kde/kde_model.hpp` (#3800). + * Fix serialization and `MinDistance()` bugs with `HollowBallBound` (#3808). + ## mlpack 4.5.0 _2024-09-17_ diff --git a/doc/img/hollowballbound.png b/doc/img/hollowballbound.png new file mode 100644 index 0000000000..3660328b3a Binary files /dev/null and b/doc/img/hollowballbound.png differ diff --git a/doc/sidebar.html b/doc/sidebar.html index 28a4f637ad..26f3025a27 100644 --- a/doc/sidebar.html +++ b/doc/sidebar.html @@ -96,6 +96,11 @@ when the sidebar is built for each page. MeanSplitBallTree +
  • + + VPTree + +
  • BinarySpaceTree diff --git a/doc/user/core/distributions.md b/doc/user/core/distributions.md index 5fcedd558c..a01057b1ab 100644 --- a/doc/user/core/distributions.md +++ b/doc/user/core/distributions.md @@ -849,7 +849,7 @@ regression model's prediction on `x`. This class is meant to be used with mlpack's [HMM](/src/mlpack/methods/hmm/hmm.hpp) class for the task of -[HMM regression (pdf)](https://conservancy.umn.edu/bitstream/handle/11299/2532/1195.pdf). +[HMM regression (pdf)](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=93a56eb64e77ac83404fddfd0036e95a742fcee6). ### Constructors diff --git a/doc/user/core/trees/binary_space_tree.md b/doc/user/core/trees/binary_space_tree.md index 4c49dbc4c4..c6ce2d08ea 100644 --- a/doc/user/core/trees/binary_space_tree.md +++ b/doc/user/core/trees/binary_space_tree.md @@ -413,7 +413,7 @@ class uses a hyperrectangle bound. An example `HRectBound` is shown below; the bound is the smallest rectangle that encloses all of the points.
    -hyperrectangle bound enclosing points +hyperrectangle bound enclosing points
    mlpack supplies several drop-in `BoundType` classes, and it is also possible to @@ -423,6 +423,8 @@ write a custom `BoundType` for use with `BinarySpaceTree`: points in the smallest possible hyperrectangle * [`BallBound`](#ballbound): ball bound, encloses the descendant points in the ball with the smallest possible radius + * [`HollowBallBound`](#hollowballbound): hollow ball bound, equivalent to a + ball bound with a ball subtracted from it. * [Custom `BoundType`s](#custom-boundtypes): implement a fully custom `BoundType` @@ -577,7 +579,7 @@ operations with data points or other bounds. Once an `HRectBound` has been successfully created and set to the desired bounding hyperrectangle, there are a number of functions that can bound the -distance between a `HRectBound` and other objects. +distance between an `HRectBound` and other objects. * `b.Contains(point)` * `b.Contains(bound)` @@ -776,7 +778,7 @@ std::cout << "Distance between Manhattan distance HRectBound and " // point. arma::fmat floatData(3, 25, arma::fill::randu); mlpack::HRectBound cb; -cb |= floatData; // This will set the bound to [2.0, 3.0] in every dimension. +cb |= floatData; // Note the use of arma::fvec to represent a point, since ElemType is float. const mlpack::RangeType r3 = cb.RangeDistance(arma::fvec("1.5 1.5 4.0")); std::cout << "Distance between Chebyshev distance HRectBound and " @@ -840,8 +842,8 @@ Different constructor forms can be used to specify different template parameters any points at all). ***Note***: these constructors provide an empty bound; be sure to -[grow](#growing-and-shrinking-the-bound) the bound or -[directly modify the bound](#accessing-and-modifying-properties-of-the-bound) +[grow](#growing-the-bound) the bound or +[directly modify the bound](#accessing-and-modifying-properties-of-the-bound-1) before using it! --- @@ -1078,6 +1080,368 @@ std::cout << "Distance between Chebyshev distance BallBound and " --- +### `HollowBallBound` + +The `HollowBallBound` class represents a bounding shape that is an +arbitrary-dimensional ball bound with another smaller ball subtracted from its +inside. A `HollowBallBound` consists of a center point, an outer radius, and a +secondary center point and inner radius. An example `HollowBallBound` is shown +below in two dimensions; shaded area represents area held within the bound. + +
    +hollow ball bound +
    + +`HollowBallBound` is used directly by the [`VPTree`](vptree.md) class. + +--- + +#### Constructors + +`HollowBallBound` allows configurable behavior via its two template parameters: + +``` +HollowBallBound +``` + +Different constructor forms can be used to specify different template parameters +(and thus different bound behavior). + + * `b = HollowBallBound(dimensionality)` + - Construct a `HollowBallBound` with the given `dimensionality`. + - The bound will be empty with invalid centers and radii (e.g., `b` will not + contain any points at all). + - The bound will use the [Euclidean distance](../distances.md#lmetric) for + distance computation, and will expect data to have elements with type + `double`. + + * `b = HollowBallBound(dimensionality)` + - Construct a `HollowBallBound` with the given `dimensionality` that will use + the given `DistanceType` class to compute distances, and expect data to + have elements with type `ElemType`. + - `ElemType` should generally be `double` or `float`. + +***Note***: these constructors provide an empty bound; be sure to +[grow](#growing-the-bound-1) the bound or +[directly modify the bound](#accessing-and-modifying-properties-of-the-bound-2) +before using it! + +--- + + * `b = HollowBallBound(innerRadius, outerRadius, center)` + - Construct a `HollowBallBound` with the given `innerRadius` for the inner + ball, `outerRadius` for the outer ball, and `center`. + - Both the inner and outer ball are centered at `center`. + - `innerRadius` and `outerRadius` should have type `double`. + - `center` should have type `arma::vec`. + - The bound will use the [Euclidean distance](../distances.md#lmetric) for + distance computation, and will expect data to have elements with type + `double`. + + * `b = HollowBallBound(innerRadius, outerRadius, center)` + - Construct a `HollowBallBound` with the given `innerRadius` for the inner + ball, `outerRadius` for the outer ball, and `center`. + - Both the inner and outer ball are centered at `center`. + - `innerRadius` and `outerRadius` should have type `ElemType`. + - `center` should be a vector with element type `ElemType` (e.g. + `arma::Col`). + - The bound will use the given `DistanceType` class to compute distances, and + expect data to have elements with type `ElemType`. + +--- + +#### Accessing and modifying properties of the bound + +The individual bounds associated with each dimension of a `HollowBallBound` can +be accessed and modified. + + * `b.Dim()` will return a `size_t` indicating the dimensionality of the bound. + + * `b.Center()` returns an `arma::vec&` containing the center of the outer ball. + Its elements can be directly modified. + + * `b.HollowCenter()` returns an `arma::vec&` containing the center of the inner + ball. Its elements can be directly modified. + - It is possible that `b.HollowCenter()` is outside of the outer ball! + + * `b.OuterRadius()` will return a `double` that is the radius of the outer + ball. + - `b.OuterRadius() = r` will set the radius of the outer ball to `r`. + + * `b.InnerRadius()` will return a `double` that is the radius of the inner + ball. + - `b.InnerRadius() = r` will set the radius of the inner ball to `r`. + - It is possible that `b.InnerRadius() > b.OuterRadius()`, and this implies + that the hollow center is outside the outer ball (otherwise the bound is + empty). + + * `b[dim]` will return a [`Range`](../math.md#range) object representing the + extents of the bound in dimension `dim`. + - The range is defined as + `[b.Center()[dim] - b.OuterRadius(), b.Center()[dim] + b.OuterRadius()]`. + - ***Note:*** this returns the maximum extents of the bound and does not + consider the inner (hollow) ball. + + * `b.Diameter()` returns the diameter of the ball. This is always equal to + `2 * b.OuterRadius()`. + + * `b.MinWidth()` returns the minimum width of the bound in any dimension as a + `double`. This is always equal to `b.Diameter()`. + + * `b.Distance()` returns either a + [`EuclideanDistance`](../distances.md#lmetric) distance metric object, or a + `DistanceType` if a custom `DistanceType` has been specified in the + constructor. + + * `b.Center(center)` will store the center of the `HollowBallBound` in the + vector `center`. `center` should be of type `arma::vec`. + + * `b.MinWidth()` returns the minimum width of the bound in any dimension as a + `double`. This value is cached and no computation is performed when calling + `b.MinWidth()`. If the bound is empty, `0` is returned. + + * `b.Distance()` returns either a + [`EuclideanDistance`](../distances.md#lmetric) distance metric object, or a + `DistanceType` if a custom `DistanceType` has been specified in the + constructor. + + * `b.Center(center)` will compute the center of the `HollowBallBound` (e.g. the + vector with elements equal to the midpoint of `b` in each dimension) and + store it in the vector `center`. `center` should be of type `arma::vec`. + + * `b.Volume()` computes the volume of the hyperrectangle specified by `b`. The + volume is returned as a `double`. + + * `b.Diameter()` computes the longest diagonal of the hyperrectangle specified + by `b`. + + * A `HollowBallBound` can be serialized with + [`data::Save()` and `data::Load()`](../../load_save.md#mlpack-objects). + +***Note:*** if a custom `ElemType` was specified in the constructor, then: + + * `b[dim]` will return a `RangeType`; + * `b.OuterRadius()`, `b.InnerRadius()`, `b.MinWidth()`, and `b.Diameter()` will + return `ElemType`; + * `b.Center()` and `b.HollowCenter()` will return `arma::Col&`; and + * `b.Center(center)` expects `center` to be of type `arma::Col`. + +--- + +#### Growing the bound + +The `HollowBallBound` uses the logical `|=` to grow the bound to include points +or other bounds. + + * `b |= data` expands `b` so the outer ball includes all of the data points in + `data`, shrinking the inner ball as necessary. `data` should be a + [column-major `arma::mat`](../../matrices.md#representing-data-in-mlpack). + The expansion operation is minimal, so `b` is not expanded any more than + necessary. + - The bound is grown using [Jack Ritter's bounding sphere + algorithm](https://en.wikipedia.org/wiki/Bounding_sphere#Ritter's_bounding_sphere), + which may move the center of the bound as it iteratively adds points to the + bound. (The hollow center is not moved.) + - If the bound is empty, the centers are initialized to the first point of + `data`. + - If the bound is not empty, then `data` is expected to have dimensionality + that matches `b.Dim()`. + + * `b |= bound` expands `b` to include all of the volume included in `bound`. + The center points will not be modified. + - The outer ball's radius will be expanded to include the outer balls of both + `b` and `bound`. + - The inner (hollow) ball's radius will be shrunk to be the intersection of + the inner balls of `b` and `bound`. (This may result in `b.InnerRadius()` + being 0.) + +***Notes:*** + + - The growth operation does not grow the inner (hollow) ball. Properties + related to the inner ball should be set manually with `b.HollowCenter()` and + `b.InnerRadius()`. + + - If a custom `ElemType` was specified, then any `data` argument should be a + matrix with that `ElemType` (e.g. `arma::Mat`). + +--- + +#### Bounding distances to other objects + +Once a `HollowBallBound` has been successfully created and set to the desired +bounding balls, there are a number of functions that can bound the +distance between a `HollowBallBound` and other objects. + + * `b.Contains(point)` + * `b.Contains(bound)` + - Return a `bool` indicating whether or not `b` contains the given `point` + (an `arma::vec`) or another `bound` (an `HRectBound`). + - When passing another `bound`, `true` will be returned if `bound` even + partially overlaps with `b`. + + * `b.MinDistance(point)` + * `b.MinDistance(bound)` + - Return a `double` whose value is the minimum possible distance between `b` + and either a `point` (an `arma::vec`) or another `bound` (a + `HollowBallBound`). + - The minimum distance between `b` and another point or bound is the length + of the shortest possible line that can connect the other point or bound to + `b`. + - If `point` or `bound` are contained in `b`, then the returned distance is + 0. + + * `b.MaxDistance(point)` + * `b.MaxDistance(bound)` + - Return a `double` whose value is the maximum possible distance between `b` + and either a `point` (an `arma::vec`) or another `bound` (a + `HollowBallBound`). + - The maximum distance between `b` and a given `point` is the furthest + possible distance between `point` and any possible point falling within the + bounding hyperrectangle of `b`. + - The maximum distance between `b` and another `bound` is the furthest + possible distance between any possible point falling within the bounding + hyperrectangle of `b`, and any possible point falling within the bounding + hyperrectangle of `bound`. + - Note that this definition means that even if `b.Contains(point)` or + `b.Contains(bound)` is `true`, the maximum distance may be greater than + `0`. + + * `b.RangeDistance(point)` + * `b.RangeDistance(bound)` + - Compute the minimum and maximum distance between `b` and `point` or + `bound`, returning the result as a [`Range`](../math.md#range) object. + - This is more efficient than calling `b.MinDistance()` and + `b.MaxDistance()`. + +***Note:*** if a custom `DistanceType` and `ElemType` were specified in the +constructor, then all distances will be computed with respect to the specified +`DistanceType` and all return values will either be `ElemType` or +[`RangeType`](../math.md#range) (except for `Contains()`, which will +still return a `bool`). + +--- + +#### Example usage + +```c++ +// Create a hollow ball bound in 3 dimensions whose outer ball is the unit ball +// and whose inner ball is the ball with radius 0.5 centered at the origin. +// The bounding range for all three dimensions is [0.0, 1.0]. +mlpack::HollowBallBound b(0.5, 1.0, arma::vec(3)); + +std::cout << "Hollow unit ball bound created manually:" << std::endl; +std::cout << " - Center: " << b.Center().t(); +std::cout << " - Outer radius: " << b.OuterRadius() << "." << std::endl; +std::cout << " - Hollow center: " << b.HollowCenter().t(); +std::cout << " - Inner radius: " << b.InnerRadius() << "." << std::endl; +for (size_t i = 0; i < 3; ++i) +{ + std::cout << " - Dimension " << i << " extents: [" << b[i].Lo() << ", " + << b[i].Hi() << "]." << std::endl; +} +std::cout << std::endl; + +// Create a small dataset of 5 points. +arma::mat dataset(3, 5); +dataset.col(0) = arma::vec("2.0 2.0 2.0"); +dataset.col(1) = arma::vec("2.5 2.5 2.5"); +dataset.col(2) = arma::vec("3.0 2.0 3.0"); +dataset.col(3) = arma::vec("2.0 3.0 2.0"); +dataset.col(4) = arma::vec("3.0 3.0 3.0"); + +// If we simply build a HollowBallBound to enclose those points, the hollow part +// of the ball is unmodified and remains empty. +mlpack::HollowBallBound b2(3); +b2 |= dataset; +std::cout << "Hollow ball bound on points with only `operator|=()`:" + << std::endl; +std::cout << " - Center: " << b2.Center().t(); +std::cout << " - Outer radius: " << b2.OuterRadius() << "." << std::endl; +std::cout << " - Hollow center: " << b2.HollowCenter().t(); +std::cout << " - Inner radius: " << b2.InnerRadius() << "." << std::endl; +std::cout << std::endl; + +// On the other hand, if we initialize a HollowBallBound to a non-empty bound, +// then `operator|=()` will shrink the hollow ball as necessary. +// +// We initialize this ball bound to a "slice" with radii [3.6, 3.7]. +mlpack::HollowBallBound b3(3.6, 3.7, arma::vec(3)); +b3 |= dataset; +std::cout << "Hollow ball bound on points with pre-initialization and " + << "`operator|=()`:" << std::endl; +std::cout << " - Center: " << b3.Center().t(); +std::cout << " - Outer radius: " << b3.OuterRadius() << "." << std::endl; +std::cout << " - Hollow center: " << b3.HollowCenter().t(); +std::cout << " - Inner radius: " << b3.InnerRadius() << "." << std::endl; +std::cout << std::endl; + +// Manually create a hollow ball bound whose hollow center is different than the +// outer ball's center. +mlpack::HollowBallBound b4(3); +b4.OuterRadius() = 3.0; +b4.InnerRadius() = 1.5; +b4.Center() = arma::vec(3); +b4.HollowCenter() = arma::vec("1.0 1.0 1.0"); + +// Compute the minimum distance between a point inside the hollow unit ball's +// outer ball. +const double d1 = b.MinDistance(arma::vec("0.9 0.9 0.9")); +std::cout << "Minimum distance between hollow unit ball bound and [0.9, 0.9, " + << "0.9]: " << d1 << "." << std::endl; + +// Compute the minimum distance between a point inside the hollow unit ball's +// inner ball (so the point is not contained in the bound---it is within the +// hollow section). +const double d2 = b.MinDistance(arma::vec("0.0 0.0 0.0")); +std::cout << "Minimum distance between hollow unit ball bound and [0.0, 0.0, " + << "0.0]: " << d2 << "." << std::endl; +std::cout << std::endl; + +// Use Contains(). In this case, the 'else' will be taken. +if (b.Contains(arma::vec("1.5 1.5 1.5"))) +{ + std::cout << "Hollow unit ball bound contains [1.5, 1.5, 1.5]." << std::endl; +} +else +{ + std::cout << "Hollow unit ball bound does not contain [1.5, 1.5, 1.5]." + << std::endl; +} +std::cout << std::endl; + +// Compute the maximum distance between a point inside the unit ball and the +// unit hollow ball bound. +const double d3 = b4.MaxDistance(arma::vec("0.1 0.1 0.1")); +std::cout << "Maximum distance between hollow unit ball bound and [0.1, 0.1, " + << "0.1]: " << d3 << "." << std::endl; + +// Compute the minimum and maximum distances between the hollow unit ball bound +// and the bound built on data points. +const mlpack::Range r = b.RangeDistance(b3); +std::cout << "Distances between hollow unit ball bound and second hollow " + << "dataset bound: [" << r.Lo() << ", " << r.Hi() << "]." << std::endl; + +// Create a bound using the Manhattan (L1) distance and compute the minimum and +// maximum distance to a point. +mlpack::HollowBallBound mb(2.0, 5.0, arma::vec(3)); +const mlpack::Range r2 = mb.RangeDistance(arma::vec("1.5 1.5 4.0")); +std::cout << "Distance between Manhattan distance HollowBallBound and " + << "[1.5, 1.5, 4.0]: [" << r2.Lo() << ", " << r2.Hi() << "]." << std::endl; + +// Create a bound using the Chebyshev (L-inf) distance, using random 32-bit +// floating point elements, and compute the minimum and maximum distance to a +// point. +arma::fmat floatData(3, 25, arma::fill::randu); +mlpack::HollowBallBound cb; +cb |= floatData; +// Note the use of arma::fvec to represent a point, since ElemType is float. +const mlpack::RangeType r3 = cb.RangeDistance(arma::fvec("1.5 1.5 4.0")); +std::cout << "Distance between Chebyshev distance HollowBallBound and " + << "[1.5, 1.5, 4.0]: [" << r3.Lo() << ", " << r3.Hi() << "]." << std::endl; +``` + +--- + ### Custom `BoundType`s The `BinarySpaceTree` class allows an arbitrary `BoundType` template parameter @@ -1261,6 +1625,8 @@ to write a fully custom split: with maximum width * [`MeanSplit`](#meansplit): splits on the mean value of the points in the dimension with maximum width + * [`VantagePointSplit`](#vantagepointsplit): split by selecting a 'vantage + point' and then split points into 'near' and 'far' sets * [Custom `SplitType`s](#custom-splittypes): implement a fully custom `SplitType` class @@ -1309,6 +1675,51 @@ task*. For implementation details, see [the source code](/src/mlpack/core/tree/binary_space_tree/mean_split_impl.hpp). +### `VantagePointSplit` + +The `VantagePointSplit` class is a splitting strategy that can be used by +[`BinarySpaceTree`](#binaryspacetree). It is the default strategy for splitting +[`VPTree`s](vptree.md), and is detailed in +[the paper](https://www.mlpack.org/papers/uhlmann91.pdf). +Due to the nature of the split, ***`VantagePointSplit` should always be used +with the [`HollowBallBound`](#hollowballbound)***. + +The splitting strategy for the `VantagePointSplit` class is, given a set of +points: + + * Select a vantage point from a sample of 100 random candidate points (or use + the full set if there are fewer than 100 points): + - Compute the distances between each candidate point and 100 additional + random samples (or the full set if there are fewer than 100 points). + - Select the vantage point as the candidate with maximum average distance to + the additional random samples. + * Compute a boundary distance `mu` that is the median distance between the + vantage point and its random samples. + * Points with distance less than `mu` from the vantage point will go to the + left child. + * Points with distance greater than `mu` from the vantage point will go to the + right child. + +The `VantagePointSplit` class has three template parameters: + +``` +VantagePointSplit +``` + +If a custom number of samples `S` is desired, the easiest way to specify is via +a template typedef: + +``` +template +using MyVantagePointSplit = VantagePointSplit; +``` + +Then, `MyVantagePointSplit` can be used directly with `BinarySpaceTree` as a +`SplitType`. + +For implementation details, see +[the source code](/src/mlpack/core/tree/binary_space_tree/vantage_point_split_impl.hpp). + ### Custom `SplitType`s Custom split strategies for a binary space tree can be implemented via the diff --git a/doc/user/core/trees/vptree.md b/doc/user/core/trees/vptree.md new file mode 100644 index 0000000000..b0b916651d --- /dev/null +++ b/doc/user/core/trees/vptree.md @@ -0,0 +1,634 @@ +# `VPTree` + + + +The `VPTree` class represents a `k`-dimensional vantage point tree, +and is a well-known data structure for efficient distance operations (such as +nearest neighbor search) in low dimensions---typically less than 100. The +vantage point tree is also known as the 'metric tree'. + +A vantage point tree is a binary tree where each node selects a 'vantage +point', and child nodes partition points into those that are nearer to the +vantage point and those that are further from it. `VPTree` supports arbitrary +[distance metrics](../distances.md) that are not +[`LMetric`](../distances.md#lmetric), making it more flexible than `KDTree`. + +mlpack's `VPTree` implementation supports three template parameters for +configurable behavior, and implements all the functionality required by the +[TreeType API](../../../developer/trees.md#the-treetype-api), plus some +additional functionality specific to vantage point trees. + + * [Template parameters](#template-parameters) + * [Constructors](#constructors) + * [Basic tree properties](#basic-tree-properties) + * [Bounding distances with the tree](#bounding-distances-with-the-tree) + * [Tree traversals](#tree-traversals) + * [Example usage](#example-usage) + +## See also + + + + * [Vantage point tree on Wikipedia](https://en.wikipedia.org/wiki/Vantage-point_tree) + * [`BinarySpaceTree`](binary_space_tree.md) + * [Binary space partitioning on Wikipedia](https://dl.acm.org/doi/pdf/10.1145/361002.361007) + * [Data structures and algorithms for nearest neighbor search in general metric spaces (pdf)](https://dl.acm.org/doi/pdf/10.5555/313559.313789) + * [Satisfying General Proximity/Similarity Queries with Metric Trees (pdf)](https://www.mlpack.org/papers/uhlmann91.pdf) + * [Tree-Independent Dual-Tree Algorithms (pdf)](https://www.ratml.org/pub/pdf/2013tree.pdf) + +## Template parameters + +In accordance with the [TreeType +API](../../../developer/trees.md#template-parameters-required-by-the-treetype-policy) +(see also [this more detailed section](../../../developer/trees.md#template-parameters)), +the `VPTree` class takes three template parameters: + +``` +VPTree +``` + + * `DistanceType`: the [distance metric](../distances.md) to use for distance + computations. By default, this is + [`EuclideanDistance`](../distances.md#lmetric). + * [`StatisticType`](binary_space_tree.md#statistictype): this holds auxiliary + information in each tree node. By default, + [`EmptyStatistic`](binary_space_tree.md#emptystatistic) is used, which holds + no information. + * `MatType`: the type of matrix used to represent points. Must be a type + matching the [Armadillo API](../../matrices.md). By default, `arma::mat` is + used, but other types such as `arma::fmat` or similar will work just fine. + +The `VPTree` class itself is a convenience typedef of the generic +[`BinarySpaceTree`](binary_space_tree.md) class, using the +[`HollowBallBound`](binary_space_tree.md#hollowballbound) class as the bounding +structure, and using the +[`VantagePointSplit`](binary_space_tree.md#vantagepointsplit) splitting strategy +for construction, which splits points into those that are nearer and further +from a 'vantage point'. + +## Constructors + +`VPTree`s are efficiently constructed by permuting points in a dataset in a +quicksort-like algorithm. However, this means that the ordering of points in +the tree's dataset (accessed with `node.Dataset()`) after construction may be +different. + +--- + + * `node = VPTree(data, maxLeafSize=20)` + * `node = VPTree(data, oldFromNew, maxLeafSize=20)` + * `node = VPTree(data, oldFromNew, newFromOld, maxLeafSize=20)` + - Construct a `VPTree` on the given `data`, using `maxLeafSize` as the + maximum number of points held in a leaf. + - By default, `data` is copied. Avoid a copy by using `std::move()` (e.g. + `std::move(data)`); when doing this, `data` will be set to an empty matrix. + - Optionally, construct mappings from old points to new points. `oldFromNew` + and `newFromOld` will have length `data.n_cols`, and: + * `oldFromNew[i]` indicates that point `i` in the tree's dataset was + originally point `oldFromNew[i]` in `data`; that is, + `node.Dataset().col(i)` is the point `data.col(oldFromNew[i])`. + * `newFromOld[i]` indicates that point `i` in `data` is now point + `newFromOld[i]` in the tree's dataset; that is, + `node.Dataset().col(newFromOld[i])` is the point `data.col(i)`. + +--- + + * `node = VPTree(data, maxLeafSize=20)` + * `node = VPTree(data, oldFromNew, maxLeafSize=20)` + * `node = VPTree(data, oldFromNew, newFromOld, maxLeafSize=20)` + - Construct a `VPTree` on the given `data`, using custom template parameters + to control the behavior of the tree, using `maxLeafSize` as the maximum + number of points held in a leaf. + - By default, `data` is copied. Avoid a copy by using `std::move()` (e.g. + `std::move(data)`); when doing this, `data` will be set to an empty matrix. + - Optionally, construct mappings from old points to new points. `oldFromNew` + and `newFromOld` will have length `data.n_cols`, and: + * `oldFromNew[i]` indicates that point `i` in the tree's dataset was + originally point `oldFromNew[i]` in `data`; that is, + `node.Dataset().col(i)` is the point `data.col(oldFromNew[i])`. + * `newFromOld[i]` indicates that point `i` in `data` is now point + `newFromOld[i]` in the tree's dataset; that is, + `node.Dataset().col(newFromOld[i])` is the point `data.col(i)`. + +--- + + * `node = VPTree()` + - Construct an empty vantage point tree with no children and no points. + +--- + +***Notes:*** + + - The name `node` is used here for `VPTree` objects instead of `tree`, because + each `VPTree` object is a single node in the tree. The constructor returns + the node that is the root of the tree. + + - Inserting individual points or removing individual points from a `VPTree` is + not supported, because this generally results in a vantage point tree with + very loose bounding balls. It is better to simply build a new `VPTree` on + the modified dataset. For trees that support individual insertion and + deletions, see the `RectangleTree` class and all its variants (e.g. `RTree`, + `RStarTree`, etc.). + + - See also the + [developer documentation on tree constructors](../../../developer/trees.md#constructors-and-destructors). + + + +--- + +### Constructor parameters: + +| **name** | **type** | **description** | **default** | +|----------|----------|-----------------|-------------| +| `data` | [`arma::mat`](../../matrices.md) | [Column-major](../../matrices.md#representing-data-in-mlpack) matrix to build the tree on. Pass with `std::move(data)` to avoid copying the matrix. | _(N/A)_ | +| `maxLeafSize` | `size_t` | Maximum number of points to store in each leaf. | `20` | +| `oldFromNew` | `std::vector` | Mappings from points in `node.Dataset()` to points in `data`. | _(N/A)_ | +| `newFromOld` | `std::vector` | Mappings from points in `data` to points in `node.Dataset()`. | _(N/A)_ | + +## Basic tree properties + +Once a `VPTree` object is constructed, various properties of the tree can be +accessed or inspected. Many of these functions are required by the [TreeType +API](../../../developer/trees.md#the-treetype-api). + +### Navigating the tree + + * `node.NumChildren()` returns the number of children in `node`. This is + either `2` if `node` has children, or `0` if `node` is a leaf. + + * `node.IsLeaf()` returns a `bool` indicating whether or not `node` is a leaf. + + * `node.Child(i)` returns a `VPTree&` that is the `i`th child. + - `i` must be `0` or `1`. + - This function should only be called if `node.NumChildren()` is not `0` + (e.g. if `node` is not a leaf). Note that this returns a valid `VPTree&` + that can itself be used just like the root node of the tree! + - `node.Left()` and `node.Right()` are convenience functions specific to + `VPTree` that will return `VPTree*` (pointers) to the left and right + children, respectively, or `NULL` if `node` has no children. + + * `node.Parent()` will return a `VPTree*` that points to the parent of `node`, + or `NULL` if `node` is the root of the `VPTree`. + +--- + +### Accessing members of a tree + + * `node.Bound()` will return an + [`HollowBallBound&`](binary_space_tree.md#hollowballbound) object that + represents the hollow bounding ball of `node`. This structure encloses all + the descendant points of `node`. + + * `node.Stat()` will return an `EmptyStatistic&` (or a `StatisticType&` if a + [custom `StatisticType`](#template-parameters) was specified as a template + parameter) holding the statistics of the node that were computed during tree + construction. + + * `node.Distance()` will return a + [`EuclideanDistance&`](../distances.md#lmetric) (or a `DistanceType&` if a + [custom `DistanceType`](#template-parameters) was specified as a template + parameter). + +See also the +[developer documentation](../../../developer/trees.md#basic-tree-functionality) +for basic tree functionality in mlpack. + +--- + +### Accessing data held in a tree + + * `node.Dataset()` will return a `const arma::mat&` that is the dataset the + tree was built on. Note that this is a permuted version of the `data` matrix + passed to the constructor. + - If a [custom `MatType`](#template-parameters) is being used, the return + type will be `const MatType&` instead of `const arma::mat&`. + + * `node.NumPoints()` returns a `size_t` indicating the number of points held + directly in `node`. + - If `node` is not a leaf, this will return `0`, as `VPTree` only holds + points directly in its leaves. + - If `node` is a leaf, then the number of points will be less than or equal + to the `maxLeafSize` that was specified when the tree was constructed. + + * `node.Point(i)` returns a `size_t` indicating the index of the `i`'th point + in `node.Dataset()`. + - `i` must be in the range `[0, node.NumPoints() - 1]` (inclusive). + - `node` must be a leaf (as non-leaves do not hold any points). + - The `i`'th point in `node` can then be accessed as + `node.Dataset().col(node.Point(i))`. + - In a `VPTree`, because of the permutation of points done [during + construction](#constructors), point indices are contiguous: + `node.Point(i + j)` is the same as `node.Point(i) + j` for valid `i` and + `j`. + - Accessing the actual `i`'th point itself can be done with, e.g., + `node.Dataset().col(node.Point(i))`. + + * `node.NumDescendants()` returns a `size_t` indicating the number of points + held in all descendant leaves of `node`. + - If `node` is the root of the tree, then `node.NumDescendants()` will be + equal to `node.Dataset().n_cols`. + + * `node.Descendant(i)` returns a `size_t` indicating the index of the `i`'th + descendant point in `node.Dataset()`. + - `i` must be in the range `[0, node.NumDescendants() - 1]` (inclusive). + - `node` does not need to be a leaf. + - The `i`'th descendant point in `node` can then be accessed as + `node.Dataset().col(node.Descendant(i))`. + - In a `VPTree`, because of the permutation of points done [during + construction](#constructors), point indices are contiguous: + `node.Descendant(i + j)` is the same as `node.Descendant(i) + j` for valid + `i` and `j`. + - Accessing the actual `i`'th descendant itself can be done with, e.g., + `node.Dataset().col(node.Descendant(i))`. + + * `node.Begin()` returns a `size_t` indicating the index of the first + descendant point of `node`. + - This is equivalent to `node.Descendant(0)`. + + * `node.Count()` returns a `size_t` indicating the number of descendant points of `node`. + - This is equivalent to `node.NumDescendants()`. + +--- + +### Accessing computed bound quantities of a tree + +The following quantities are cached for each node in a `VPTree`, and so +accessing them does not require any computation. + + * `node.FurthestPointDistance()` returns a `double` representing the distance + between the center of the hollow bounding ball of `node` and the furthest + point held by `node`. + - If `node` is not a leaf, this returns 0 (because `node` does not hold any + points). + + * `node.FurthestDescendantDistance()` returns a `double` representing the + distance between the center of the hollow bounding ball of `node` and the + furthest descendant point held by `node`. + + * `node.MinimumBoundDistance()` returns a `double` representing minimum + possible distance from the center of the node to any edge of the + hollow ball bound. + - This quantity is equivalent to `node.Bound().OuterRadius()`. + + * `node.ParentDistance()` returns a `double` representing the distance between + the center of the hollow bounding ball of `node` and the center of the + hollow bounding ball of its parent. + - If `node` is the root of the tree, `0` is returned. + +***Notes:*** + + - If a [custom `MatType`](#template-parameters) was specified when constructing + the `VPTree`, then the return type of each method is the element type of the + given `MatType` instead of `double`. (e.g., if `MatType` is `arma::fmat`, + then the return type is `float`.) + + - For more details on each bound quantity, see the + [developer documentation](../../../developer/trees.md#complex-tree-functionality-and-bounds) + on bound quantities for trees. + +--- + +### Other functionality + + * `node.Center(center)` computes the center of the hollow bounding ball of + `node` and stores it in `center`. + - `center` should be of type `arma::vec&`. (If a [custom + `MatType`](#template-parameters) was specified when constructing the + `VPTree`, the type is instead the column vector type for the given + `MatType`; e.g., `arma::fvec&` when `MatType` is `arma::fmat`.) + - `center` will be set to have size equivalent to the dimensionality of the + dataset held by `node`. + - This is equivalent to calling `node.Bound().Center(center)`. + + * A `VPTree` can be serialized with + [`data::Save()` and `data::Load()`](../../load_save.md#mlpack-objects). + +## Bounding distances with the tree + +The primary use of trees in mlpack is bounding distances to points or other tree +nodes. The following functions can be used for these tasks. + + * `node.GetNearestChild(point)` + * `node.GetFurthestChild(point)` + - Return a `size_t` indicating the index of the child (`0` for left, `1` for + right) that is closest to (or furthest from) `point`, with respect + to the `MinDistance()` (or `MaxDistance()`) function. + - If there is a tie, `0` (the left child) is returned. + - If `node` is a leaf, `0` is returned. + - `point` should be of type `arma::vec`. (If a [custom + `MatType`](#template-parameters) was specified when constructing the + `VPTree`, the type is instead the column vector type for the given + `MatType`; e.g., `arma::fvec` when `MatType` is `arma::fmat`.) + + * `node.GetNearestChild(other)` + * `node.GetFurthestChild(other)` + - Return a `size_t` indicating the index of the child (`0` for left, `1` for + right) that is closest to (or furthest from) the `VPTree` node `other`, + with respect to the `MinDistance()` (or `MaxDistance()`) function. + - If there is a tie, `2` (an invalid index) is returned. ***Note that this + behavior differs from the version above that takes a point.*** + - If `node` is a leaf, `0` is returned. + +--- + + * `node.MinDistance(point)` + * `node.MinDistance(other)` + - Return a `double` indicating the minimum possible distance between `node` + and `point`, or the `VPTree` node `other`. + - This is equivalent to the minimum possible distance between any point + contained in the hollow bounding ball of `node` and `point`, or between + any point contained in the hollow bounding ball of `node` and any point + contained in the hollow bounding ball of `other`. + - `point` should be of type `arma::vec`. (If a [custom + `MatType`](#template-parameters) was specified when constructing the + `VPTree`, the type is instead the column vector type for the given + `MatType`, and the return type is the element type of `MatType`; e.g., + `point` should be `arma::fvec` when `MatType` is `arma::fmat`, and the + returned distance is `float`). + + * `node.MaxDistance(point)` + * `node.MaxDistance(other)` + - Return a `double` indicating the maximum possible distance between `node` + and `point`, or the `VPTree` node `other`. + - This is equivalent to the maximum possible distance between any point + contained in the hollow bounding ball of `node` and `point`, or between + any point contained in the hollow bounding ball of `node` and any point + contained in the hollow bounding ball of `other`. + - `point` should be of type `arma::vec`. (If a [custom + `MatType`](#template-parameters) was specified when constructing the + `VPTree`, the type is instead the column vector type for the given + `MatType`, and the return type is the element type of `MatType`; e.g., + `point` should be `arma::fvec` when `MatType` is `arma::fmat`, and the + returned distance is `float`). + + * `node.RangeDistance(point)` + * `node.RangeDistance(other)` + - Return a [`Range`](../math.md#range) whose lower bound is + `node.MinDistance(point)` or `node.MinDistance(other)`, and whose upper + bound is `node.MaxDistance(point)` or `node.MaxDistance(other)`. + - `point` should be of type `arma::vec`. (If a + [custom `MatType`](#template-parameters) was specified when constructing + the `VPTree`, the type is instead the column vector type for the given + `MatType`, and the return type is a `RangeType` with element type the same + as `MatType`; e.g., `point` should be `arma::fvec` when `MatType` is + `arma::fmat`, and the returned type is + [`RangeType`](../math.md#range)). + +### Tree traversals + +Like every mlpack tree, the `VPTree` class provides a [single-tree and dual-tree +traversal](../../../developer/trees.md#traversals) that can be paired with a +[`RuleType` class](../../../developer/trees.md#rules) to implement a single-tree +or dual-tree algorithm. + + * `VPTree::SingleTreeTraverser` + - Implements a depth-first single-tree traverser. + + * `VPTree::DualTreeTraverser` + - Implements a dual-depth-first dual-tree traverser. + +In addition to those two classes, which are required by the +[`TreeType` policy](../../../developer/trees.md), an additional traverser is +available: + + * `VPTree::BreadthFirstDualTreeTraverser` + - Implements a dual-breadth-first dual-tree traverser. + - ***Note:*** this traverser is not useful for all tasks; because the + `VPTree` only holds points in the leaves, this means that no base cases + (e.g. comparisons between points) will be called until *all* pairs of + intermediate nodes have been scored! + +## Example usage + +Build a `VPTree` on the `cloud` dataset and print basic statistics about the +tree. + +```c++ +// See https://datasets.mlpack.org/cloud.csv. +arma::mat dataset; +mlpack::data::Load("cloud.csv", dataset, true); + +// Build the vp-tree with a leaf size of 10. (This means that nodes are split +// until they contain 10 or fewer points.) +// +// The std::move() means that `dataset` will be empty after this call, and no +// data will be copied during tree building. +// +// Note that the '<>' isn't necessary if C++20 is being used (e.g. +// `mlpack::VPTree tree(...)` will work fine in C++20 or newer). +mlpack::VPTree<> tree(std::move(dataset)); + +// Print the bounding ball of the root node. (There will be no hollow ball.) +std::cout << "Bounding ball of root node:" << std::endl; +std::cout << " - Center: " << tree.Bound().Center().t(); +std::cout << " - Outer radius: " << tree.Bound().OuterRadius() << "." + << std::endl; +std::cout << " - Hollow center: " << tree.Bound().HollowCenter().t(); +std::cout << " - Inner radius: " << tree.Bound().InnerRadius() << "." + << std::endl; +std::cout << std::endl; + +// Print the bounding ball of the right child. (This will have a hollow ball.) +std::cout << "Bounding ball of right child: " << std::endl; +std::cout << " - Center: " << tree.Right()->Bound().Center().t(); +std::cout << " - Outer radius: " << tree.Right()->Bound().OuterRadius() << "." + << std::endl; +std::cout << " - Hollow center: " << tree.Right()->Bound().HollowCenter().t(); +std::cout << " - Inner radius: " << tree.Right()->Bound().InnerRadius() << "." + << std::endl; +std::cout << " - Distance between centers: " << +mlpack::EuclideanDistance::Evaluate(tree.Right()->Bound().Center(), +tree.Right()->Bound().HollowCenter()) << "." << std::endl; +std::cout << std::endl; + +// Print the number of descendant points of the root, and of each of its +// children. +std::cout << "Descendant points of root: " + << tree.NumDescendants() << "." << std::endl; +std::cout << "Descendant points of left child: " + << tree.Left()->NumDescendants() << "." << std::endl; +std::cout << "Descendant points of right child: " + << tree.Right()->NumDescendants() << "." << std::endl; +std::cout << std::endl; + +// Print the center of the vp-tree. +arma::vec center; +tree.Center(center); +std::cout << "Center of vp-tree: " << center.t(); +``` + +--- + +Build two `VPTree`s on subsets of the corel dataset and compute various +bounding quantities. + +```c++ +// See https://datasets.mlpack.org/corel-histogram.csv. +arma::mat dataset; +mlpack::data::Load("corel-histogram.csv", dataset, true); + +// Build vp-trees on the first half and the second half of points. +mlpack::VPTree<> tree1(dataset.cols(0, dataset.n_cols / 2)); +mlpack::VPTree<> tree2(dataset.cols(dataset.n_cols / 2 + 1, + dataset.n_cols - 1)); + +// Compute the maximum distance between the trees. +std::cout << "Maximum distance between tree root nodes: " + << tree1.MaxDistance(tree2) << "." << std::endl; + +// Get the leftmost grandchild of the first tree's root---if it exists. +if (!tree1.IsLeaf() && !tree1.Child(0).IsLeaf()) +{ + mlpack::VPTree<>& node1 = tree1.Child(0).Child(0); + + // Get the rightmost grandchild of the second tree's root---if it exists. + if (!tree2.IsLeaf() && !tree2.Child(1).IsLeaf()) + { + mlpack::VPTree<>& node2 = tree2.Child(1).Child(1); + + // Print the minimum and maximum distance between the nodes. + mlpack::Range dists = node1.RangeDistance(node2); + std::cout << "Possible distances between two grandchild nodes: [" + << dists.Lo() << ", " << dists.Hi() << "]." << std::endl; + + // Print the minimum distance between the first node and the first + // descendant point of the second node. + const size_t descendantIndex = node2.Descendant(0); + const double descendantMinDist = + node1.MinDistance(node2.Dataset().col(descendantIndex)); + std::cout << "Minimum distance between grandchild node and descendant " + << "point: " << descendantMinDist << "." << std::endl; + + // Which child of node2 is closer to node1? + const size_t closerIndex = node2.GetNearestChild(node1); + if (closerIndex == 0) + std::cout << "The left child of node2 is closer to node1." << std::endl; + else if (closerIndex == 1) + std::cout << "The right child of node2 is closer to node1." << std::endl; + else // closerIndex == 2 in this case. + std::cout << "Both children of node2 are equally close to node1." + << std::endl; + + // And which child of node1 is further from node2? + const size_t furtherIndex = node1.GetFurthestChild(node2); + if (furtherIndex == 0) + std::cout << "The left child of node1 is further from node2." + << std::endl; + else if (furtherIndex == 1) + std::cout << "The right child of node1 is further from node2." + << std::endl; + else // furtherIndex == 2 in this case. + std::cout << "Both children of node1 are equally far from node2." + << std::endl; + } +} +``` + +--- + +Build a `VPTree` on 32-bit floating point data and save it to disk. + +```c++ +// See https://datasets.mlpack.org/corel-histogram.csv. +arma::fmat dataset; +mlpack::data::Load("corel-histogram.csv", dataset); + +// Build the VPTree using 32-bit floating point data as the matrix type. +// We will still use the default EmptyStatistic and EuclideanDistance +// parameters. A leaf size of 100 is used here. +mlpack::VPTree tree(std::move(dataset), 100); + +// Save the VPTree to disk with the name 'tree'. +mlpack::data::Save("tree.bin", "tree", tree); + +std::cout << "Saved tree with " << tree.Dataset().n_cols << " points to " + << "'tree.bin'." << std::endl; +``` + +--- + +Load a 32-bit floating point `VPTree` from disk, then traverse it manually and +find the number of leaf nodes with less than 10 children. + +```c++ +// This assumes the tree has already been saved to 'tree.bin' (as in the example +// above). + +// This convenient typedef saves us a long type name! +typedef mlpack::VPTree TreeType; + +TreeType tree; +mlpack::data::Load("tree.bin", "tree", tree); +std::cout << "Tree loaded with " << tree.NumDescendants() << " points." + << std::endl; + +// Recurse in a depth-first manner. Count both the total number of leaves, and +// the number of leaves with less than 10 points. +size_t leafCount = 0; +size_t totalLeafCount = 0; +std::stack stack; +stack.push(&tree); +while (!stack.empty()) +{ + TreeType* node = stack.top(); + stack.pop(); + + if (node->NumPoints() < 10) + ++leafCount; + ++totalLeafCount; + + if (!node->IsLeaf()) + { + stack.push(node->Left()); + stack.push(node->Right()); + } +} + +// Note that it would be possible to use TreeType::SingleTreeTraverser to +// perform the recursion above, but that is more well-suited for more complex +// tasks that require pruning and other non-trivial behavior; so using a simple +// stack is the better option here. + +// Print the results. +std::cout << leafCount << " out of " << totalLeafCount << " leaves have less " + << "than 10 points." << std::endl; +``` + +--- + +Build a `VPTree` and map between original points and new points. + +```c++ +// See https://datasets.mlpack.org/cloud.csv. +arma::mat dataset; +mlpack::data::Load("cloud.csv", dataset, true); + +// Build the tree. +std::vector oldFromNew, newFromOld; +mlpack::VPTree<> tree(dataset, oldFromNew, newFromOld); + +// oldFromNew and newFromOld will be set to the same size as the dataset. +std::cout << "Number of points in dataset: " << dataset.n_cols << "." + << std::endl; +std::cout << "Size of oldFromNew: " << oldFromNew.size() << "." << std::endl; +std::cout << "Size of newFromOld: " << newFromOld.size() << "." << std::endl; +std::cout << std::endl; + +// See where point 42 in the tree's dataset came from. +std::cout << "Point 42 in the permuted tree's dataset:" << std::endl; +std::cout << " " << tree.Dataset().col(42).t(); +std::cout << "Was originally point " << oldFromNew[42] << ":" << std::endl; +std::cout << " " << dataset.col(oldFromNew[42]).t(); +std::cout << std::endl; + +// See where point 7 in the original dataset was mapped. +std::cout << "Point 7 in original dataset:" << std::endl; +std::cout << " " << dataset.col(7).t(); +std::cout << "Mapped to point " << newFromOld[7] << ":" << std::endl; +std::cout << " " << tree.Dataset().col(newFromOld[7]).t(); +``` diff --git a/src/mlpack/core/tree/binary_space_tree/typedef.hpp b/src/mlpack/core/tree/binary_space_tree/typedef.hpp index 81b071b400..550ac40740 100644 --- a/src/mlpack/core/tree/binary_space_tree/typedef.hpp +++ b/src/mlpack/core/tree/binary_space_tree/typedef.hpp @@ -198,7 +198,9 @@ template using VPTreeSplit = VantagePointSplit; -template +template using VPTree = BinarySpaceTree 0); } } // namespace mlpack diff --git a/src/mlpack/core/tree/hollow_ball_bound_impl.hpp b/src/mlpack/core/tree/hollow_ball_bound_impl.hpp index 4d65a0aeec..41d3122234 100644 --- a/src/mlpack/core/tree/hollow_ball_bound_impl.hpp +++ b/src/mlpack/core/tree/hollow_ball_bound_impl.hpp @@ -316,8 +316,8 @@ RangeType HollowBallBound::RangeDistance( typename std::enable_if_t::value>* /* junk */) const { if (radii.Hi() < 0) - return Range(std::numeric_limits::max(), - std::numeric_limits::max()); + return RangeType(std::numeric_limits::max(), + std::numeric_limits::max()); else { RangeType range; @@ -461,7 +461,7 @@ void HollowBallBound::serialize( ar(CEREAL_NVP(radii)); ar(CEREAL_NVP(center)); ar(CEREAL_NVP(hollowCenter)); - ar(CEREAL_POINTER(distance)); + if (cereal::is_loading()) { // If we're loading, delete the local distance since we'll have a new one. @@ -470,6 +470,8 @@ void HollowBallBound::serialize( ownsDistance = true; } + + ar(CEREAL_POINTER(distance)); } } // namespace mlpack