mlpack/doc/user/methods/decision_tree.md

DecisionTree

The DecisionTree class implements a decision tree classifier that supports numerical and categorical features, by default using Gini gain to choose which feature to split on. The class offers several template parameters and several constructor parameters that can be used to control the behavior of the tree.

Constructors

Forms:

signature	description
DecisionTree(numClasses)	Initialize tree without training.
DecisionTree(data, labels, numClasses)	Train on numerical-only data.
DecisionTree(data, labels, numClasses, minimumLeafSize, minimumGainSplit, maximumDepth)	Train on numerical-only data with hyperparameters.
DecisionTree(data, datasetInfo, labels, numClasses)	Train on mixed categorical data.
DecisionTree(data, datasetInfo, labels, numClasses, minimumLeafSize, minimumGainSplit, maximumDepth)	Train on mixed categorical data with hyperparameters.
DecisionTree(data, datasetInfo, labels, numClasses, weights)	Train on weighted mixed categorical data.
DecisionTree(data, datasetInfo, labels, numClasses, weights, minimumLeafSize, minimumGainSplit, maximumDepth)	Train on weighted mixed categorical data with hyperparameters.

Parameters:

type	name	description	default
arma::mat	data	Column-major training matrix.	(N/A)
data::DatasetInfo	datasetInfo	Dataset information, specifying type information for each dimension.	(N/A)
arma::Row<size_t>	labels	Training labels, between 0 and numClasses - 1 (inclusive).	(N/A)
arma::rowvec	weights	Weights for each training point.	(N/A)
size_t	numClasses	Number of classes in the dataset.	(N/A)
size_t	minimumLeafSize	Minimum number of points in each leaf node.	10
double	minimumGainSplit	Minimum gain for a node to split.	1e-7
size_t	maximumDepth	Maximum depth for the tree. (0 means no limit.)	0

Training

If training is not done as part of the constructor call, it can be done with the Train() member function, which has several overloads.

• tree.Train(data, labels, numClasses) (train on numerical-only data)

• tree.Train(data, labels, numClasses, minimumLeafSize, minimumGainSplit, maximumDepth) (train on numerical-only data with hyperparameters)

• tree.Train(data, labels, numClasses, weights) (train on weighted numerical-only data)

• tree.Train(data, labels, numClasses, weights, minimumLeafSize, minimumGainSplit, maximumDepth) (train on weighted numerical-only data with hyperparameters)

• tree.Train(data, datasetInfo, labels, numClasses) (train on mixed categorical data)

• tree.Train(data, datasetInfo, labels, numClasses, minimumLeafSize, minimumGainSplit, maximumDepth) (train on mixed categorical data with hyperparameters)

• tree.Train(data, datasetInfo, labels, numClasses, weights) (train on weighted mixed categorical data)

• tree.Train(data, datasetInfo, labels, numClasses, weights, minimumLeafSize, minimumGainSplit, maximumDepth) (train on weighted mixed categorical data with hyperparameters)

Types of each argument are the same as in the table for constructors above.

Classification

Once a DecisionTree is trained, the Classify() member function can be used to make class predictions for new data.

• size_t class = tree.Classify(point) (classify a single point)
• tree.Classify(point, prediction, probabilities) (classify a single point and compute class probabilities)

type	name	description
arma::vec	point	Single point for classification.
size_t&	prediction	size_t to store class prediction into.
arma::vec&	probabilities	arma::vec& to store class probabilities into.

• tree.Classify(data, predictions) (classify a set of points)
• tree.Classify(data, predictions, probabilities) (classify a set of points and compute class probabilities for each point)

type	name	description
arma::mat	data	Set of column-major points for classification.
arma::Row<size_t>&	predictions	Vector of size_ts to store class prediction into.
arma::mat&	probabilities	Matrix to store class probabilities into (number of rows will be equal to number of classes).

Simple examples

Train a decision tree on random numeric data:

// 1000 random points in 10 dimensions.
arma::mat dataset(1000, 10);
// Random labels for each point, totaling 5 classes.
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));

DecisionTree<> tree(data, labels, 5);

Train a decision tree on random mixed categorical data:

categorical example -- TODO

Other functionality

• A DecisionTree can be serialized with data::Save() an data::Load().

• tree.NumChildren() will return a size_t indicating the number of children in the node tree.

• tree.Child(i) will return a DecisionTree object representing the ith child of the node tree.

• tree.SplitDimension() returns a size_t indicating which dimension the node tree splits on.

• tree.NumClasses() returns a size_t indicating the number of classes the tree was trained on.

For complete functionality, the source code can be consulted. Each method is fully documented.

Advanced Functionality: Template Parameters

The DecisionTree<> class also supports several template parameters.

See Also