1061 lines
42 KiB
Markdown
1061 lines
42 KiB
Markdown
# Distributions
|
|
|
|
<!-- TODO: link to the completed HMM documentation -->
|
|
|
|
mlpack has support for a number of different distributions, each supporting the
|
|
same API. These can be used with, for instance, the
|
|
[`HMM`](/src/mlpack/methods/hmm/hmm.hpp) class.
|
|
|
|
* [`DiscreteDistribution`](#discretedistribution): multidimensional categorical
|
|
distribution (generalized Bernoulli distribution)
|
|
* [`GaussianDistribution`](#gaussiandistribution): multidimensional Gaussian
|
|
distribution
|
|
* [`DiagonalGaussianDistribution`](#diagonalgaussiandistribution):
|
|
multidimensional Gaussian distribution with diagonal covariance
|
|
* [`GammaDistribution`](#gammadistribution): multidimensional Gamma
|
|
distribution, includes exponential, Chi-squared, and Erlang distributions
|
|
* [`LaplaceDistribution`](#laplacedistribution): multidimensional Laplace
|
|
(double exponential) distribution
|
|
* [`RegressionDistribution`](#regressiondistribution): multidimensional
|
|
Gaussian distribution on the errors of a linear regression model
|
|
|
|
## `DiscreteDistribution`
|
|
|
|
`DiscreteDistribution` represents a multidimensional categorical distribution
|
|
(or generalized Bernoulli distribution) where integer-valued vectors (e.g.
|
|
`[0, 3, 4]`) are associated with specific probabilities in each dimension.
|
|
|
|
*Example:* a 3-dimensional `DiscreteDistribution` will have a specific
|
|
probability value associated with each integer value in each dimension. So, for
|
|
the vector `[0, 3, 4]`, `P(0)` in dimension 0 could be, e.g., `0.3`, `P(3)` in
|
|
dimension 1 could be, e.g., `0.4`, and `P(4)` in dimension 2 could be, e.g.,
|
|
`0.6`. Then, `P([0, 3, 4])` would be `0.3 * 0.4 * 0.6 = 0.072`.
|
|
|
|
### Constructors
|
|
|
|
* `d = DiscreteDistribution(numObservations)`
|
|
- Create a one-dimensional discrete distribution with `numObservations`
|
|
different observations in the one and only dimension. `numObservations` is
|
|
of type `size_t`.
|
|
|
|
* `d = DiscreteDistribution(numObservationsVec)`
|
|
- Create a multidimensional discrete distribution with
|
|
`numObservationsVec.n_elem` dimensions and `numObservationsVec[i]`
|
|
different observations in dimension `i`.
|
|
- `numObservationsVec` is of type `arma::Col<size_t>`.
|
|
|
|
* `d = DiscreteDistribution(probabilities)`
|
|
- Create a multidimensional discrete distribution with the given
|
|
probabilities.
|
|
- `probabilities` should have type `std::vector<arma::vec>`, and
|
|
`probabilities.size()` should be equal to the dimensionality of the
|
|
distribution.
|
|
- `probabilities[i]` is a vector such that `probabilities[i][j]` contains the
|
|
probability of `j` in dimension `i`.
|
|
|
|
### Access and modify properties of distribution
|
|
|
|
* `d.Dimensionality()` returns a `size_t` indicating the number of dimensions
|
|
in the multidimensional discrete distribution.
|
|
|
|
* `d.Probabilities(i)` returns an `arma::vec&` containing the probabilities of
|
|
each observation in dimension `i`.
|
|
- `d.Probabilities(i)[j]` is the probability of `j` in dimension `i`.
|
|
- This can be used to modify probabilities: `d.Probabilities(0)[1] = 0.7`
|
|
sets the probability of observing the value `1` in dimension `0` to `0.7`.
|
|
- *Note:* when setting probabilities manually, be sure that the sum of
|
|
probabilities in a dimension is 1!
|
|
|
|
* A `DiscreteDistribution` can be serialized with
|
|
[`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
|
|
|
|
### Compute probabilities of points
|
|
|
|
* `d.Probability(observation)` returns the probability of the given
|
|
observation as a `double`.
|
|
- `observation` should be an `arma::vec` of size `d.Dimensionality()`.
|
|
- `observation[i]` should take integer values between `0` and
|
|
`d.Probabilities(i).n_elem - 1`.
|
|
|
|
* `d.Probability(observations, probabilities)` computes the probabilities of
|
|
many observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`d.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
- `probabilities` will be set to size `observations.n_cols`.
|
|
- `probabilities[i]` will be set to `d.Probability(observations.col(i))`.
|
|
|
|
* `d.LogProbability(observation)` returns the log-probability of the given
|
|
observation as a `double`.
|
|
|
|
* `d.LogProbability(observations, probabilities)` computes the
|
|
log-probabilities of many observations.
|
|
|
|
### Sample from the distribution
|
|
|
|
* `d.Random()` returns an `arma::vec` with a random sample from the
|
|
multidimensional discrete distribution.
|
|
|
|
### Fit the distribution to observations
|
|
|
|
* `d.Train(observations)`
|
|
- Fit the distribution to the given observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`d.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
- `observations(j, i)` should be an integer value between `0` and the number
|
|
of observations for dimension `i`.
|
|
|
|
* `d.Train(observations, observationProbabilities)`
|
|
- Fit the distribution to the given observations, as above, but also provide
|
|
probabilities that each observation is from this distribution.
|
|
- `observationProbabilities` should be an `arma::vec` of length
|
|
`observations.n_cols`.
|
|
- `observationProbabilities[i]` should be equal to the probability that
|
|
`observations.col(i)` is from `d`.
|
|
|
|
### Example usage
|
|
|
|
```c++
|
|
// Create a single-dimension Bernoulli distribution: P([0]) = 0.3, P([1]) = 0.7.
|
|
mlpack::DiscreteDistribution bernoulli(2);
|
|
bernoulli.Probabilities(0)[0] = 0.3;
|
|
bernoulli.Probabilities(0)[1] = 0.7;
|
|
|
|
const double p1 = bernoulli.Probability(arma::vec("0")); // p1 = 0.3.
|
|
const double p2 = bernoulli.Probability(arma::vec("1")); // p2 = 0.7.
|
|
|
|
// Create a 3-dimensional discrete distribution by specifying the probabilities
|
|
// manually.
|
|
arma::vec probDim0 = arma::vec("0.1 0.3 0.5 0.1"); // 4 possible values.
|
|
arma::vec probDim1 = arma::vec("0.7 0.3"); // 2 possible values.
|
|
arma::vec probDim2 = arma::vec("0.4 0.4 0.2"); // 3 possible values.
|
|
std::vector<arma::vec> probs { probDim0, probDim1, probDim2 };
|
|
mlpack::DiscreteDistribution d(probs);
|
|
|
|
arma::vec obs("2 0 1");
|
|
const double p3 = d.Probability(obs); // p3 = 0.5 * 0.7 * 0.4 = 0.14.
|
|
|
|
// Estimate a 10-dimensional discrete distribution.
|
|
// Each dimension takes values between 0 and 9.
|
|
arma::mat observations = arma::randi<arma::mat>(10, 1000,
|
|
arma::distr_param(0, 9));
|
|
|
|
// Create a distribution with 10 observations in each of the 10 dimensions.
|
|
mlpack::DiscreteDistribution d2(
|
|
arma::Col<size_t>("10 10 10 10 10 10 10 10 10 10"));
|
|
d2.Train(observations);
|
|
|
|
// Compute the probabilities of each point.
|
|
arma::vec probabilities;
|
|
d2.Probability(observations, probabilities);
|
|
std::cout << "Average probability: " << arma::mean(probabilities) << "."
|
|
<< std::endl;
|
|
```
|
|
|
|
### Using different element types
|
|
|
|
The `DiscreteDistribution` class takes two template parameters:
|
|
|
|
```
|
|
DiscreteDistribution<MatType, ObsMatType>
|
|
```
|
|
|
|
* `MatType` represents the matrix type used to represent internal parameters
|
|
(e.g. probabilities of each observation).
|
|
* `ObsMatType` represents the matrix type used to represent observations.
|
|
|
|
* By default:
|
|
- `MatType` is `arma::mat`, but any dense matrix type matching the Armadillo
|
|
API that holds floating-point numbers can be used (e.g. `arma::fmat`).
|
|
- `ObsMatType` is `MatType`, but any matrix type matching the Armadillo API
|
|
can be used (e.g. `arma::fmat`, `arma::imat`, etc.).
|
|
|
|
* When using custom `MatType` and `ObsMatType` parameters, several method
|
|
signatures will change:
|
|
- `DiscreteDistributions(probabilities)` will expect `probabilities` to be `
|
|
`std::vector<VecType>`, where `VecType` is the column vector type
|
|
associated with `MatType` (e.g. `arma::fvec` for `arma::fmat`).
|
|
|
|
- `Probability(observation)` and `LogProbability(observation)` will expect
|
|
`observation` to be an `ObsVecType`, where `ObsVecType` is the column
|
|
vector type associated with `ObsMatType`, and will return a probability
|
|
with type equivalent to the element type of `MatType`.
|
|
|
|
- `Probability(observations, probabilities)` and
|
|
`LogProbability(observations, probabilities)` will expect `observations` to
|
|
be of type `ObsMatType` and `probabilities` to be of type `VecType`.
|
|
|
|
- `Random()` will return an `ObsVecType`.
|
|
|
|
- `Train(observations)` and `Train(observations, probabilities)` will expect
|
|
`observations` to be of type `ObsMatType` and `probabilities` to be of type
|
|
`VecType`.
|
|
|
|
- `Probabilities(dim)` will return a `VecType`.
|
|
|
|
The code below uses a `DiscreteDistribution` built on 32-bit floating point
|
|
numbers.
|
|
|
|
```c++
|
|
// Create a distribution with 10 observations in each of 3 dimensions.
|
|
mlpack::DiscreteDistribution<arma::fmat> d(arma::Col<size_t>("10 10 10"));
|
|
|
|
// Train the distribution on random data.
|
|
arma::fmat observations =
|
|
arma::randi<arma::fmat>(3, 100, arma::distr_param(0, 9));
|
|
d.Train(observations);
|
|
|
|
// Compute and print the probability of [8, 6, 7].
|
|
const float p = d.Probability(arma::fvec("8 6 7"));
|
|
std::cout << "Probability of [8, 6, 7]: " << p << "." << std::endl;
|
|
```
|
|
|
|
The code below uses a `DiscreteDistribution` that internally uses `float` to
|
|
hold probabilities, but accepts `unsigned int`s as observations.
|
|
|
|
```c++
|
|
// Create a distribution with 10 observations in each of 3 dimensions.
|
|
mlpack::DiscreteDistribution<arma::fmat, arma::umat> d(
|
|
arma::Col<size_t>("10 10 10"));
|
|
|
|
// Train the distribution on random data. Note that the observation type is a
|
|
// matrix of unsigned ints (arma::umat).
|
|
arma::umat observations =
|
|
arma::randi<arma::umat>(3, 100, arma::distr_param(0, 9));
|
|
d.Train(observations);
|
|
|
|
// Compute and print the probability of [8, 6, 7]. Note that the input vector
|
|
// is a vector of unsigned ints (arma::uvec), but the returned probability is a
|
|
// float because MatType is set to arma::fmat.
|
|
const float p = d.Probability(arma::uvec("8 6 7"));
|
|
std::cout << "Probability of [8, 6, 7]: " << p << "." << std::endl;
|
|
|
|
// Print the probability vector for dimension 0.
|
|
std::cout << "Probabilities for observations in dimension 0: "
|
|
<< d.Probabilities(0).t() << std::endl;
|
|
```
|
|
|
|
## `GaussianDistribution`
|
|
|
|
`GaussianDistribution` is a standard multivariate Gaussian distribution with
|
|
parameterized mean and covariance. (For a Gaussian distribution with a diagonal
|
|
covariance, see
|
|
[`DiagonalGaussianDistribution`](#diagonalgaussiandistribution).)
|
|
|
|
### Constructors
|
|
|
|
* `g = GaussianDistribution(dimensionality)`
|
|
- Create the distribution with the given dimensionality.
|
|
- The distribution will have a zero mean and unit diagonal covariance matrix.
|
|
|
|
* `g = GaussianDistribution(mean, covariance)`
|
|
- Create the distribution with the given mean and covariance.
|
|
- `mean` is of type `arma::vec` and should have length equal to the
|
|
dimensionality of the distribution.
|
|
- `covariance` is of type `arma::mat`, and should be symmetric and square,
|
|
with rows and columns equal to the dimensionality of the distribution.
|
|
|
|
### Access and modify properties of distribution
|
|
|
|
* `g.Dimensionality()` returns the dimensionality of the distribution as a
|
|
`size_t`.
|
|
|
|
* `g.Mean()` returns an `arma::vec&` holding the mean of the distribution.
|
|
This can be modified.
|
|
|
|
* `g.Covariance()` returns a `const arma::mat&` holding the covariance of the
|
|
distribution. To set a new covariance, use `g.Covariance(newCov)` or
|
|
`g.Covariance(std::move(newCov))`.
|
|
|
|
* `g.InvCov()` returns a `const arma::mat&` holding the precomputed inverse of
|
|
the covariance.
|
|
|
|
* `g.LogDetCov()` returns a `double` holding the log-determinant of the
|
|
covariance.
|
|
|
|
* A `GaussianDistribution` can be serialized with
|
|
[`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
|
|
|
|
### Compute probabilities of points
|
|
|
|
* `g.Probability(observation)` returns the probability of the given
|
|
observation as a `double`.
|
|
- `observation` should be an `arma::vec` of size `g.Dimensionality()`.
|
|
|
|
* `g.Probability(observations, probabilities)` computes the probabilities of
|
|
many observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`d.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
- `probabilities` will be set to size `observations.n_cols`.
|
|
- `probabilities[i]` will be set to `g.Probability(observations.col(i))`.
|
|
|
|
* `g.LogProbability(observation)` returns the log-probability of the given
|
|
observation as a `double`.
|
|
|
|
* `g.LogProbability(observations, probabilities)` computes the
|
|
log-probabilities of many observations.
|
|
|
|
### Sample from the distribution
|
|
|
|
* `g.Random()` returns an `arma::vec` with a random sample from the
|
|
multidimensional Gaussian distribution.
|
|
|
|
### Fit the distribution to observations
|
|
|
|
* `g.Train(observations)`
|
|
- Fit the distribution to the given observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`g.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
|
|
* `g.Train(observations, observationProbabilities)`
|
|
- Fit the distribution to the given observations, as above, but also provide
|
|
probabilities that each observation is from this distribution.
|
|
- `observationProbabilities` should be an `arma::vec` of length
|
|
`observations.n_cols`.
|
|
- `observationProbabilities[i]` should be equal to the probability that
|
|
`observations.col(i)` is from `g`.
|
|
|
|
### Example usage
|
|
|
|
```c++
|
|
// Create a Gaussian distribution in 3 dimensions with zero mean and unit
|
|
// covariance.
|
|
mlpack::GaussianDistribution g(3);
|
|
|
|
// Compute the probability of the point [0, 0.5, 0.25].
|
|
const double p = g.Probability(arma::vec("0 0.5 0.25"));
|
|
|
|
// Modify the mean in dimension 0.
|
|
g.Mean()[0] = 0.5;
|
|
|
|
// Set a random covariance.
|
|
arma::mat newCov(3, 3, arma::fill::randu);
|
|
newCov *= newCov.t(); // Ensure covariance is positive semidefinite.
|
|
g.Covariance(std::move(newCov)); // Set new covariance.
|
|
|
|
// Compute the probability of the same point [0, 0.5, 0.25].
|
|
const double p2 = g.Probability(arma::vec("0 0.5 0.25"));
|
|
|
|
// Create a Gaussian distribution that is estimated from random samples in 50
|
|
// dimensions.
|
|
arma::mat samples(50, 10000, arma::fill::randn); // Normally distributed.
|
|
|
|
mlpack::GaussianDistribution g2(50);
|
|
g2.Train(samples);
|
|
|
|
// Compute the probability of all of the samples.
|
|
arma::vec probabilities;
|
|
g2.Probability(samples, probabilities);
|
|
|
|
std::cout << "Average probability is: " << arma::mean(probabilities) << "."
|
|
<< std::endl;
|
|
```
|
|
|
|
### Using different element types
|
|
|
|
The `GaussianDistribution` class takes one template parameter:
|
|
|
|
```
|
|
GaussianDistribution<MatType>
|
|
```
|
|
|
|
* `MatType` represents the matrix type used to represent observations.
|
|
* By default, `MatType` is `arma::mat`, but any matrix type matching the
|
|
Armadillo API can be used (e.g. `arma::fmat`).
|
|
* When `MatType` is set to anything other than `arma::mat`, all arguments are
|
|
adapted accordingly:
|
|
- `arma::mat` arguments will instead be `MatType`.
|
|
- `arma::vec` arguments will instead be the corresponding column vector type
|
|
associated with `MatType`.
|
|
- `double` arguments will instead be the element type of `MatType`.
|
|
|
|
The code below uses a Gaussian distribution to make predictions with 32-bit
|
|
floating point numbers.
|
|
|
|
```c++
|
|
// Create a 3-dimensional 32-bit floating point Gaussian distribution with
|
|
// random mean and unit covariance.
|
|
mlpack::GaussianDistribution<arma::fmat> g(3);
|
|
g.Mean().randu();
|
|
|
|
// Compute the probability of the point [0.2, 0.3, 0.4].
|
|
const float p = g.Probability(arma::fvec("0.2 0.3 0.4"));
|
|
|
|
std::cout << "Probability of (0.2, 0.3, 0.4): " << p << "." << std::endl;
|
|
```
|
|
|
|
## `DiagonalGaussianDistribution`
|
|
|
|
`DiagonalGaussianDistribution` is a standard multiviate Gaussian distribution
|
|
with parameterized mean and diagonal covariance. (For a full-covariance
|
|
Gaussian distribution, see [`GaussianDistribution`](#gaussiandistribution).)
|
|
|
|
### Constructors
|
|
|
|
* `d = DiagonalGaussianDistribution(dimensionality)`
|
|
- Create the distribution with the given dimensionality.
|
|
- The distribution will have a zero mean and unit diagonal covariance matrix.
|
|
|
|
* `d = DiagonalGaussianDistribution(mean, covariance)`
|
|
- Create the distribution with the given mean and covariance.
|
|
- `mean` is of type `arma::vec` and should have length equal to the
|
|
dimensionality of the distribution.
|
|
- `covariance` is of type `arma::vec`, and should have length equal to the
|
|
dimensionality of the distribution. Its elements represent the diagonal of
|
|
the covariance matrix.
|
|
|
|
### Access and modify properties of distribution
|
|
|
|
* `d.Dimensionality()` returns the dimensionality of the distribution as a
|
|
`size_t`.
|
|
|
|
* `d.Mean()` returns an `arma::vec&` holding the mean of the distribution.
|
|
This can be modified.
|
|
|
|
* `d.Covariance()` returns a `const arma::vec&` holding the covariance of the
|
|
distribution. To set a new covariance, use `d.Covariance(newCov)` or
|
|
`d.Covariance(std::move(newCov))`, where `newCov` is the new diagonal of the
|
|
covariance matrix.
|
|
|
|
* A `DiagonalGaussianDistribution` can be serialized with
|
|
[`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
|
|
|
|
### Compute probabilities of points
|
|
|
|
* `d.Probability(observation)` returns the probability of the given observation
|
|
as a `double`.
|
|
- `observation` should be an `arma::vec` of size `d.Dimensionality()`.
|
|
|
|
* `d.Probability(observations, probabilities)` computes the probabilities of
|
|
many observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`d.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
- `probabilities` will be set to size `observations.n_cols`.
|
|
- `probabilities[i]` will be set to `d.Probability(observations.col(i))`.
|
|
|
|
* `d.LogProbability(observation)` returns the log-probability of the given
|
|
observation as a `double`.
|
|
|
|
* `d.LogProbability(observations, probabilities)` computes the
|
|
log-probabilities of many observations.
|
|
|
|
### Sample from the distribution
|
|
|
|
* `d.Random()` returns an `arma::vec` with a random sample from the
|
|
multidimensional diagonal Gaussian distribution.
|
|
|
|
### Fit the distribution to observations
|
|
|
|
* `d.Train(observations)`
|
|
- Fit the distribution to the given observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`d.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
|
|
* `g.Train(observations, observationProbabilities)`
|
|
- Fit the distribution to the given observations, as above, but also provide
|
|
probabilities that each observation is from this distribution.
|
|
- `observationProbabilities` should be an `arma::vec` of length
|
|
`observations.n_cols`.
|
|
- `observationProbabilities[i]` should be equal to the probability that
|
|
`observations.col(i)` is from `d`.
|
|
|
|
### Example usage
|
|
|
|
```c++
|
|
// Create a diagonal Gaussian distribution in 3 dimensions with zero mean and
|
|
// unit covariance.
|
|
mlpack::DiagonalGaussianDistribution d(3);
|
|
|
|
// Compute the probability of the point [0, 0.5, 0.25].
|
|
const double p = d.Probability(arma::vec("0 0.5 0.25"));
|
|
|
|
// Modify the mean in dimension 0.
|
|
d.Mean()[0] = 0.5;
|
|
|
|
// Set the covariance to a random diagonal.
|
|
arma::vec newCovDiag(3, arma::fill::randu);
|
|
d.Covariance(std::move(newCovDiag)); // Set new covariance.
|
|
|
|
// Compute the probability of the same point [0, 0.5, 0.25].
|
|
const double p2 = d.Probability(arma::vec("0 0.5 0.25"));
|
|
|
|
// Create a diagonal Gaussian distribution that is estimated from random samples
|
|
// in 50 dimensions.
|
|
arma::mat samples(50, 10000, arma::fill::randn); // Normally distributed.
|
|
|
|
mlpack::DiagonalGaussianDistribution d2(50);
|
|
d2.Train(samples);
|
|
|
|
// Compute the probability of all of the samples.
|
|
arma::vec probabilities;
|
|
d2.Probability(samples, probabilities);
|
|
|
|
std::cout << "Average probability is: " << arma::mean(probabilities) << "."
|
|
<< std::endl;
|
|
```
|
|
|
|
### Using different element types
|
|
|
|
The `DiagonalGaussianDistribution` class takes one template parameter:
|
|
|
|
```
|
|
DiagonalGaussianDistribution<MatType>
|
|
```
|
|
|
|
* `MatType` represents the matrix type used to represent observations.
|
|
* By default, `MatType` is `arma::mat`, but any matrix type matching the
|
|
Armadillo API can be used (e.g. `arma::fmat`).
|
|
* When `MatType` is set to anything other than `arma::mat`, all arguments are
|
|
adapted accordingly:
|
|
- `arma::mat` arguments will instead be `MatType`.
|
|
- `arma::vec` arguments will instead be the corresponding column vector type
|
|
associated with `MatType`.
|
|
- `double` arguments will instead be the element type of `MatType`.
|
|
|
|
The code below uses a Gaussian distribution to make predictions with 32-bit
|
|
floating point numbers.
|
|
|
|
```c++
|
|
// Create a 3-dimensional 32-bit floating point Gaussian distribution with
|
|
// random mean and covariance.
|
|
mlpack::DiagonalGaussianDistribution<arma::fmat> g(
|
|
arma::randu<arma::fvec>(3), arma::randu<arma::fvec>(3));
|
|
|
|
// Compute the probability of the point [0.2, 0.3, 0.4].
|
|
const float p = g.Probability(arma::fvec("0.2 0.3 0.4"));
|
|
|
|
std::cout << "Probability of (0.2, 0.3, 0.4): " << p << "." << std::endl;
|
|
```
|
|
|
|
## `GammaDistribution`
|
|
|
|
`GammaDistribution` is a multivariate Gamma distribution with two parameters for
|
|
shape (alpha) and inverse scale (beta). Certain settings of these parameters
|
|
yield the exponential distribution, Chi-squared distribution, and Erlang
|
|
distribution. This family of distributions is commonly used in Bayesian
|
|
statistics. See more on
|
|
[Wikipedia](https://en.wikipedia.org/wiki/Gamma_distribution).
|
|
|
|
### Constructors
|
|
|
|
* `g = GammaDistribution(dimensionality)`
|
|
- Create the distribution with the given dimensionality.
|
|
- The distribution will have alpha and beta parameters in each dimension set
|
|
to 0.
|
|
|
|
* `g = GammaDistribution(alphas, betas)`
|
|
- Create the distribution with the given parameters.
|
|
- `alphas` and `betas` are of type `arma::vec` and should have length equal
|
|
to the dimensionality of the distribution.
|
|
- `alphas` should hold the desired shape parameters in each dimension.
|
|
- `betas` should hold the desired inverse scale parameters in each dimension.
|
|
|
|
* `g = GammaDistribution(data, tol=1e-8)`
|
|
- Create the distribution by fitting to the given `data`.
|
|
- `tol` specifies the convergence tolerance for the fitting procedure.
|
|
- Using this constructor is equivalent to calling `g.Train(data, tol)` after
|
|
initializing a `GammaDistribution`.
|
|
|
|
### Access and modify properties of distribution
|
|
|
|
* `g.Dimensionality()` returns the dimensionality of the distribution.
|
|
|
|
* `g.Alpha(i)` returns a `double` representing the shape parameter for
|
|
dimension `i`. `g.Alpha(i) = a` will set the `i`'th dimension's shape
|
|
parameter to `a`.
|
|
|
|
* `g.Beta(i)` returns a `double` representing the inverse scale parameter for
|
|
dimension `i`. `g.Beta(i) = b` will set the `i`'th dimension's inverse scale
|
|
parameter to `b`.
|
|
|
|
* A `GammaDistribution` can be serialized with
|
|
[`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
|
|
|
|
### Compute probabilities of points
|
|
|
|
* `g.Probability(observation)` returns the probability of the given observation
|
|
as a `double`.
|
|
- `observation` should be an `arma::vec` of size `g.Dimensionality()`.
|
|
|
|
* `g.Probability(observations, probabilities)` computes the probabilities of
|
|
many observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`g.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
- `probabilities` will be set to size `observations.n_cols`.
|
|
- `probabilities[i]` will be set to `g.Probability(observations.col(i))`.
|
|
|
|
* `g.LogProbability(observation)` returns the log-probability of the given
|
|
observation as a `double`.
|
|
|
|
* `g.LogProbability(observations, probabilities)` computes the
|
|
log-probabilities of many observations.
|
|
|
|
### Sample points from the distribution
|
|
|
|
* `g.Random()` returns an `arma::vec` with a random sample from the
|
|
Gamma distribution.
|
|
|
|
### Fit the distribution to observations
|
|
|
|
* `g.Train(observations)`
|
|
- Fit the distribution to the given observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`g.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
|
|
* `g.Train(observations, observationProbabilities)`
|
|
- Fit the distribution to the given observations, as above, but also provide
|
|
probabilities that each observation is from this distribution.
|
|
- `observationProbabilities` should be an `arma::vec` of length
|
|
`observations.n_cols`.
|
|
- `observationProbabilities[i]` should be equal to the probability that
|
|
`observations.col(i)` is from `g`.
|
|
|
|
* The algorithm used for fitting the distribution is described in the paper
|
|
[Estimating a Gamma Distribution](https://tminka.github.io/papers/minka-gamma.pdf).
|
|
|
|
### Example usage
|
|
|
|
```c++
|
|
// Create a Gamma distribution in 3 dimensions with ones for the alpha (shape)
|
|
// parameters and random beta (inverse scale) parameters.
|
|
mlpack::GammaDistribution g(arma::ones<arma::vec>(3) /* shape */,
|
|
arma::randu<arma::vec>(3) /* scale */);
|
|
|
|
// Compute the probability and log-probability of the point [0, 0.5, 0.25].
|
|
const double p = g.Probability(arma::vec("0 0.5 0.25"));
|
|
const double lp = g.LogProbability(arma::vec("0 0.5 0.25"));
|
|
|
|
std::cout << "Probability of [0 0.5 0.25]: " << p << "." << std::endl;
|
|
std::cout << "Log-probability of [0 0.5 0.25]: " << lp << "." << std::endl;
|
|
|
|
// Modify the scale and inverse shape parameters in dimension 0.
|
|
g.Alpha(0) = 0.5;
|
|
g.Beta(0) = 3.0;
|
|
|
|
// Compute the probability of the same point [0, 0.5, 0.25].
|
|
const double p2 = g.Probability(arma::vec("0 0.5 0.25"));
|
|
const double lp2 = g.LogProbability(arma::vec("0 0.5 0.25"));
|
|
|
|
std::cout << "After parameter changes:" << std::endl;
|
|
std::cout << "Probability of [0 0.5 0.25]: " << p << "." << std::endl;
|
|
std::cout << "Log-probability of [0 0.5 0.25]: " << lp << "." << std::endl;
|
|
|
|
// Create a Gamma distribution that is estimated from random samples in 5
|
|
// dimensions. Note that the samples here are uniformly distributed---so a
|
|
// Gamma distribution fit will not be a good one!
|
|
arma::mat samples(5, 1000, arma::fill::randu);
|
|
samples += 2.0; // Shift samples away from zero.
|
|
|
|
mlpack::GammaDistribution g2(samples, 1e-3 /* tolerance for fitting */);
|
|
|
|
// Compute the probability of all of the samples.
|
|
arma::vec probabilities;
|
|
g2.Probability(samples, probabilities);
|
|
|
|
std::cout << "Average probability is: " << arma::mean(probabilities) << "."
|
|
<< std::endl;
|
|
```
|
|
|
|
### Using different element types
|
|
|
|
The `GammaDistribution` class takes one template parameter:
|
|
|
|
```
|
|
GammaDistribution<MatType>
|
|
```
|
|
|
|
* `MatType` represents the matrix type used to represent observations.
|
|
* By default, `MatType` is `arma::mat`, but any matrix type matching the
|
|
Armadillo API can be used (e.g. `arma::fmat`).
|
|
* When `MatType` is set to anything other than `arma::mat`, all arguments are
|
|
adapted accordingly:
|
|
- `arma::mat` arguments will instead be `MatType`
|
|
- `arma::vec` arguments will instead be the corresponding column vector type
|
|
associated with `MatType`
|
|
- `double` arguments will instead be the element type of `MatType`
|
|
- If the element type is `float`, the default tolerance (`tol`) for `Train()`
|
|
is `1e-4`
|
|
|
|
The code below uses a Gamma distribution to make predictions with 32-bit
|
|
floating point numbers.
|
|
|
|
```c++
|
|
// Create a 3-dimensional 32-bit floating point Laplace distribution with
|
|
// ones for the shape parameter and random scale parameters.
|
|
mlpack::GammaDistribution<arma::fmat> g(arma::ones<arma::fvec>(3) /* shape */,
|
|
arma::randu<arma::fvec>(3) /* scale */);
|
|
|
|
// Compute the probability of the point [0.2, 0.3, 0.4].
|
|
const float p = g.Probability(arma::fvec("0.2 0.3 0.4"));
|
|
|
|
std::cout << "Probability of (0.2, 0.3, 0.4): " << p << "." << std::endl;
|
|
```
|
|
|
|
## `LaplaceDistribution`
|
|
|
|
`LaplaceDistribution` is a multivariate Laplace distribution parameterized by a
|
|
mean vector and a single scale value. The Laplace distribution is sometimes
|
|
also called the *double exponential distribution*. See more on
|
|
[Wikipedia](https://en.wikipedia.org/wiki/Laplace_distribution).
|
|
|
|
### Constructors
|
|
|
|
* `l = LaplaceDistribution(dimensionality, scale=1.0)`
|
|
- Create the distribution with the given dimensionality.
|
|
- The distribution will have mean zero and the given `scale`.
|
|
- `scale` must be greater than 0.
|
|
|
|
* `l = LaplaceDistribution(mean, scale)`
|
|
- Create the distribution with the given parameters.
|
|
- `mean` is of type `arma::vec` and should have length equal to the
|
|
dimensionality of the distribution.
|
|
- `scale` must be greater than 0.
|
|
|
|
### Access and modify properties of distribution
|
|
|
|
* `l.Dimensionality()` returns the dimensionality of the distribution.
|
|
|
|
* `l.Mean()` returns an `arma::vec&` holding the mean of the distribution.
|
|
This can be modified.
|
|
|
|
* `l.Scale()` returns a `double` representing the distribution's scale
|
|
parameter. `l.Scale() = s` will set the scale parameter to `s`.
|
|
|
|
* A `LaplaceDistribution` can be serialized with
|
|
[`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
|
|
|
|
### Compute probabilities of points
|
|
|
|
* `l.Probability(observation)` returns the probability of the given observation
|
|
as a `double`.
|
|
- `observation` should be an `arma::vec` of size `l.Dimensionality()`.
|
|
|
|
* `l.Probability(observations, probabilities)` computes the probabilities of
|
|
many observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`l.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
- `probabilities` will be set to size `observations.n_cols`.
|
|
- `probabilities[i]` will be set to `l.Probability(observations.col(i))`.
|
|
|
|
* `l.LogProbability(observation)` returns the log-probability of the given
|
|
observation as a `double`.
|
|
|
|
* `l.LogProbability(observations, probabilities)` computes the
|
|
log-probabilities of many observations.
|
|
|
|
### Sample points from the distribution
|
|
|
|
* `l.Random()` returns an `arma::vec` with a random sample from the
|
|
Laplace distribution.
|
|
|
|
### Fit the distribution to observations
|
|
|
|
* `l.Train(observations)`
|
|
- Fit the distribution to the given observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`l.Dimensionality()`; `observations.n_cols` is the number of observations.
|
|
|
|
* `l.Train(observations, observationProbabilities)`
|
|
- Fit the distribution to the given observations, as above, but also provide
|
|
probabilities that each observation is from this distribution.
|
|
- `observationProbabilities` should be an `arma::vec` of length
|
|
`observations.n_cols`.
|
|
- `observationProbabilities[i]` should be equal to the probability that
|
|
`observations.col(i)` is from `l`.
|
|
|
|
### Example usage
|
|
|
|
```c++
|
|
// Create a Laplace distribution in 3 dimensions with uniform random mean and
|
|
// scale parameter 1.
|
|
mlpack::LaplaceDistribution l(arma::randu<arma::vec>(3) /* mean */,
|
|
1.0 /* scale */);
|
|
|
|
// Compute the probability and log-probability of the point [0, 0.5, 0.25].
|
|
const double p = l.Probability(arma::vec("0 0.5 0.25"));
|
|
const double lp = l.LogProbability(arma::vec("0 0.5 0.25"));
|
|
|
|
std::cout << "Probability of [0 0.5 0.25]: " << p << "." << std::endl;
|
|
std::cout << "Log-probability of [0 0.5 0.25]: " << lp << "." << std::endl;
|
|
|
|
// Modify the scale, and the mean in dimension 1.
|
|
l.Scale() = 2.0;
|
|
l.Mean()[1] = 1.5;
|
|
|
|
// Compute the probability of the same point [0, 0.5, 0.25].
|
|
const double p2 = l.Probability(arma::vec("0 0.5 0.25"));
|
|
const double lp2 = l.LogProbability(arma::vec("0 0.5 0.25"));
|
|
|
|
std::cout << "After parameter changes:" << std::endl;
|
|
std::cout << "Probability of [0 0.5 0.25]: " << p << "." << std::endl;
|
|
std::cout << "Log-probability of [0 0.5 0.25]: " << lp << "." << std::endl;
|
|
|
|
// Create a Laplace distribution that is estimated from random samples in 50
|
|
// dimensions. Note that the samples here are normally distributed---so a Gamma
|
|
// distribution fit will not be a good one!
|
|
arma::mat samples(50, 10000, arma::fill::randn);
|
|
|
|
mlpack::LaplaceDistribution l2;
|
|
l2.Train(samples);
|
|
|
|
// Compute the probability of all of the samples.
|
|
arma::vec probabilities;
|
|
l2.Probability(samples, probabilities);
|
|
|
|
std::cout << "Average probability is: " << arma::mean(probabilities) << "."
|
|
<< std::endl;
|
|
```
|
|
|
|
### Using different element types
|
|
|
|
The `LaplaceDistribution` class takes one template parameter:
|
|
|
|
```
|
|
LaplaceDistribution<MatType>
|
|
```
|
|
|
|
* `MatType` represents the matrix type used to represent observations.
|
|
* By default, `MatType` is `arma::mat`, but any matrix type matching the
|
|
Armadillo API can be used (e.g. `arma::fmat`).
|
|
* When `MatType` is set to anything other than `arma::mat`, all arguments are
|
|
adapted accordingly:
|
|
- `arma::mat` arguments will instead be `MatType`.
|
|
- `arma::vec` arguments will instead be the corresponding column vector type
|
|
associated with `MatType`.
|
|
- `double` arguments will instead be the element type of `MatType`.
|
|
|
|
The code below uses a Laplace distribution to make predictions with 32-bit
|
|
floating point numbers.
|
|
|
|
```c++
|
|
// Create a 3-dimensional 32-bit floating point Laplace distribution with
|
|
// random mean and scale of 2.0.
|
|
mlpack::LaplaceDistribution<arma::fmat> g(arma::randu<arma::fvec>(3), 2.0);
|
|
|
|
// Compute the probability of the point [0.2, 0.3, 0.4].
|
|
const float p = g.Probability(arma::fvec("0.2 0.3 0.4"));
|
|
|
|
std::cout << "Probability of (0.2, 0.3, 0.4): " << p << "." << std::endl;
|
|
```
|
|
|
|
## `RegressionDistribution`
|
|
|
|
The `RegressionDistribution` is a [Gaussian distribution](#gaussiandistribution)
|
|
fitted on the errors of a linear regression model. Given a point `x` with
|
|
response `y`, the probability of `(y, x)` is computed using a univariate
|
|
Gaussian distribution on the scalar residual `y - y'`, where `y'` is the linear
|
|
regression model's prediction on `x`.
|
|
|
|
<!-- TODO: fix link! -->
|
|
|
|
This class is meant to be used with mlpack's
|
|
[HMM](/src/mlpack/methods/hmm/hmm.hpp) class for the task of
|
|
[HMM regression (pdf)](https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=93a56eb64e77ac83404fddfd0036e95a742fcee6).
|
|
|
|
### Constructors
|
|
|
|
* `r = RegressionDistribution()`
|
|
- Create an empty `RegressionDistribution`.
|
|
- The distribution will not provide useful predictions; call `Train()` before
|
|
doing anything else with the object!
|
|
|
|
* `r = RegressionDistribution(predictors, responses)`
|
|
- Create the `RegressionDistribution` by estimating the parameters with the
|
|
given labeled regression data `predictors` and `responses`.
|
|
- `predictors` should be a
|
|
[column-major](../matrices.md#representing-data-in-mlpack) `arma::mat`
|
|
representing the data the distribution should be trained on.
|
|
- `responses` should be an `arma::rowvec` representing the responses for each
|
|
data point.
|
|
- The number of elements in `responses` (e.g. `responses.n_elem`) should be
|
|
the same as the number of columns in `predictors` (e.g.
|
|
`predictors.n_cols`).
|
|
|
|
### Access and modify properties of distribution
|
|
|
|
* `r.Dimensionality()` returns the dimensionality of the distribution.
|
|
- ***Note***: this is *not* the same as the number of elements in a vector
|
|
passed to `Probability()`!
|
|
|
|
* `r.Rf()` returns the [`LinearRegression&`](../methods/linear_regression.md)
|
|
model. This can be modified.
|
|
|
|
* `r.Parameters()` returns an `const arma::vec&` with length
|
|
`r.Dimensionality() + 1` representing the parameters of the linear regression
|
|
model. The first element is the bias; subsequent elements are the weights
|
|
for each dimension.
|
|
|
|
* `r.Err()` returns a [`GaussianDistribution&`](#gaussiandistribution) object
|
|
representing the univariate distribution trained on the model's residuals.
|
|
This can be modified.
|
|
|
|
* A `RegressionDistribution` can be serialized with
|
|
[`data::Save()` and `data::Load()`](../load_save.md#mlpack-objects).
|
|
|
|
### Compute probabilities of points
|
|
|
|
* `r.Probability(observation)` returns the probability of the given *labeled*
|
|
observation as a `double`.
|
|
- `observation` should be an `arma::vec` of size `r.Dimensionality() + 1`,
|
|
containing both the data point and its scalar response.
|
|
- The first element of `observation` should be the response; subsequent
|
|
elements should be the data point.
|
|
|
|
* `r.Probability(observations, probabilities)` computes the probabilities of
|
|
many labeled observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to
|
|
`r.Dimensionality() + 1`; `observations.n_cols` is the number of
|
|
observations.
|
|
- The first row of `observations` should correspond to the responses for each
|
|
data point.
|
|
- `probabilities` will be set to size `observations.n_cols`.
|
|
- `probabilities[i]` will be set to `r.Probability(observations.col(i))`.
|
|
|
|
* `r.LogProbability(observation)` returns the log-probability of the given
|
|
labeled observation as a `double`.
|
|
|
|
* `r.LogProbability(observations, probabilities)` computes the
|
|
log-probabilities of many labeled observations.
|
|
|
|
### Fit the distribution to observations
|
|
|
|
Training a `RegressionDistribution` on a given set of labeled observations is
|
|
done by first training a [`LinearRegression`](../methods/linear_regression.md)
|
|
model on the dataset, and then subsequently training a univariate
|
|
[`GaussianDistribution`](#gaussiandistribution) on the residual error of each
|
|
data point.
|
|
|
|
In the `Train()` overloads, the `observations` matrix is expected to contain
|
|
both the responses and the data points (predictors).
|
|
|
|
* `r.Train(observations)`
|
|
- Fit the distribution to the given *labeled* observations.
|
|
- `observations` should be an `arma::mat` with number of rows equal to the
|
|
dimensionality of the data plus one; `observations.n_cols` is the number of
|
|
observations.
|
|
- The first row of `observations` should correspond to the responses of the
|
|
data; subsequent rows correspond to the data itself.
|
|
|
|
* `r.Train(observations, observationProbabilities)`
|
|
- Fit the distribution to the given *labeled* observations, as above, but
|
|
also provide probabilities that each observation is from this distribution.
|
|
- `observationProbabilities` should be an `arma::rowvec` of length
|
|
`observations.n_cols`.
|
|
- `observationProbabilities[i]` should be equal to the probability that the
|
|
`i`'th observation is from `r`.
|
|
|
|
***Note***: if the linear regression model is able to exactly fit the
|
|
observations, then the resulting Gaussian distribution will have zero-valued
|
|
standard deviation, and `Probability()` will return `1` for points that are
|
|
perfectly fit and `0` otherwise.
|
|
|
|
### Example usage
|
|
|
|
```c++
|
|
// Create an example dataset that arises from a noisy random linear model:
|
|
//
|
|
// y = bx + noise
|
|
//
|
|
// Noise is added from a Gaussian distribution with zero mean and unit variance.
|
|
// Data is 10-dimensional, and we will generate 1000 points.
|
|
arma::vec b(10, arma::fill::randu);
|
|
arma::mat x(10, 1000, arma::fill::randu);
|
|
|
|
arma::rowvec y = b.t() * x + arma::randn<arma::rowvec>(1000);
|
|
|
|
// Now fit a RegressionDistribution to the data.
|
|
mlpack::RegressionDistribution r(x, y);
|
|
|
|
// Print information about the distribution.
|
|
std::cout << "RegressionDistribution model parameters:" << std::endl;
|
|
std::cout << " - " << r.Parameters().subvec(1, r.Parameters().n_elem - 1).t();
|
|
std::cout << " - Bias: " << r.Parameters()[0] << "." << std::endl;
|
|
std::cout << "True model parameters:" << std::endl;
|
|
std::cout << " - " << b.t();
|
|
std::cout << "Error Gaussian mean is " << r.Err().Mean()[0] << ", with "
|
|
<< "variance " << r.Err().Covariance()[0] << "." << std::endl << std::endl;
|
|
|
|
// Compute the probability of a point in the training set. We must assemble the
|
|
// points into a single vector.
|
|
arma::vec p1(11); // p1 will be point 5 from (x, y).
|
|
p1[0] = y[5];
|
|
p1.subvec(1, p1.n_elem - 1) = x.col(5);
|
|
std::cout << "Probability of point 5: " << r.Probability(p1) << "."
|
|
<< std::endl;
|
|
|
|
arma::vec p2(11, arma::fill::randu);
|
|
std::cout << "Probability of random point: " << r.Probability(p2) << "."
|
|
<< std::endl;
|
|
|
|
// Print log-probabilities too.
|
|
std::cout << "Log-probability of point 5: " << r.LogProbability(p1) << "."
|
|
<< std::endl;
|
|
std::cout << "Log-probability of random point: " << r.LogProbability(p2) << "."
|
|
<< std::endl << std::endl;
|
|
|
|
// Change the error distribution.
|
|
y = b.t() * x + (1.5 * arma::randn<arma::rowvec>(1000));
|
|
|
|
// Combine x and y to build the observations matrix for Train().
|
|
arma::mat observations(x.n_rows + 1, x.n_cols);
|
|
observations.row(0) = y;
|
|
observations.rows(1, observations.n_rows - 1) = x;
|
|
|
|
// Assign a random probability for each point.
|
|
arma::rowvec observationProbabilities(observations.n_cols, arma::fill::randu);
|
|
|
|
// Refit the distribution to the new data.
|
|
r.Train(observations, observationProbabilities);
|
|
|
|
// Print new error distribution information.
|
|
std::cout << "Updated error Gaussian mean is " << r.Err().Mean()[0] << ", with "
|
|
<< "variance " << r.Err().Covariance()[0] << "." << std::endl << std::endl;
|
|
|
|
// Compute average probability of points in the dataset.
|
|
arma::vec probabilities;
|
|
r.Probability(observations, probabilities);
|
|
std::cout << "Average probability of points in `observations`: "
|
|
<< arma::mean(probabilities) << "." << std::endl;
|
|
```
|
|
|
|
### Using different element types
|
|
|
|
The `RegressionDistribution` class takes one template parameter:
|
|
|
|
```
|
|
RegressionDistribution<MatType>
|
|
```
|
|
|
|
* `MatType` represents the matrix type used to represent observations.
|
|
* By default, `MatType` is `arma::mat`, but any matrix type matching the
|
|
Armadillo API can be used (e.g. `arma::fmat`).
|
|
* When `MatType` is set to anything other than `arma::mat`, all arguments are
|
|
adapted accordingly:
|
|
- `arma::mat` arguments will instead be `MatType`.
|
|
- `arma::vec` arguments will instead be the corresponding column vector type
|
|
associated with `MatType`.
|
|
- `double` arguments will instead be the element type of `MatType`.
|
|
|
|
The code below uses a regression distribution trained on 32-bit floating point
|
|
data.
|
|
|
|
```c++
|
|
// Create an example dataset that arises from a noisy random linear model:
|
|
//
|
|
// y = bx + noise
|
|
//
|
|
// Noise is added from a Gaussian distribution with zero mean and unit variance.
|
|
// Data is 3-dimensional, and we will generate 1000 points.
|
|
arma::fvec b(3, arma::fill::randu);
|
|
arma::fmat x(3, 1000, arma::fill::randu);
|
|
|
|
arma::frowvec y = b.t() * x + arma::randn<arma::frowvec>(1000);
|
|
|
|
// Now fit a RegressionDistribution to the data.
|
|
mlpack::RegressionDistribution<arma::fmat> r(x, y);
|
|
|
|
// Compute the probability of the point [0.5, 0.2, 0.3, 0.4].
|
|
// (Here 0.5 is the response, and [0.2, 0.3, 0.4] is the point.)
|
|
const float p = r.Probability(arma::fvec("0.5 0.2 0.3 0.4"));
|
|
std::cout << "Probability of (0.5, 0.2, 0.3, 0.4): " << p << "." << std::endl;
|
|
```
|