299 lines
8.6 KiB
Plaintext
299 lines
8.6 KiB
Plaintext
/*!
|
|
|
|
@file linear_regression.txt
|
|
@author James Cline
|
|
@brief Tutorial for how to use the LinearRegression class.
|
|
|
|
@page lrtutorial Linear Regression tutorial (linear_regression)
|
|
|
|
@section intro_lrtut Introduction
|
|
|
|
Linear regression is a simple machine learning technique which aims to estimate
|
|
the parameters of a linear model. Assuming we have \f$n\f$ \b predictor points
|
|
\f$\mathbf{x_i}, 0 \le i < n\f$ of dimensionality \f$d\f$ and \f$n\f$
|
|
responses \f$y_i, 0 \le i < n\f$, we are trying to estimate the best fit for
|
|
\f$\beta_i, 0 \le i \le d\f$ in the linear model
|
|
|
|
\f[
|
|
y_i = \beta_0 + \displaystyle\sum_{j = 1}^{d} \beta_j x_{ij}
|
|
\f]
|
|
|
|
for each predictor \f$\mathbf{x_i}\f$ and response \f$y_i\f$. If we take each
|
|
predictor \f$\mathbf{x_i}\f$ as a row in the matrix \f$\mathbf{X}\f$ and each
|
|
response \f$y_i\f$ as an entry of the vector \f$\mathbf{y}\f$, we can represent
|
|
the model in vector form:
|
|
|
|
\f[
|
|
\mathbf{y} = \mathbf{X} \mathbf{\beta} + \beta_0
|
|
\f]
|
|
|
|
The result of this method is the vector \f$\mathbf{\beta}\f$, including the
|
|
offset term (or intercept term) \f$\beta_0\f$.
|
|
|
|
\b mlpack provides:
|
|
|
|
- a \ref cli_lrtut "simple command-line executable" to perform linear regression
|
|
- a \ref linreg_lrtut "simple C++ interface" to perform linear regression
|
|
|
|
@section toc_lrtut Table of Contents
|
|
|
|
A list of all the sections this tutorial contains.
|
|
|
|
- \ref intro_lrtut
|
|
- \ref toc_lrtut
|
|
- \ref cli_lrtut
|
|
- \ref cli_ex1_lrtut
|
|
- \ref cli_ex2_lrtut
|
|
- \ref cli_ex3_lrtut
|
|
- \ref linreg_lrtut
|
|
- \ref linreg_ex1_lrtut
|
|
- \ref linreg_ex2_lrtut
|
|
- \ref linreg_ex3_lrtut
|
|
- \ref linreg_ex4_lrtut
|
|
- \ref further_doc_lrtut
|
|
|
|
@section cli_lrtut Command-Line 'linear_regression'
|
|
|
|
The simplest way to perform linear regression in \b mlpack is to use the
|
|
linear_regression executable. This program will perform linear regression and
|
|
place the resultant coefficients into one file.
|
|
|
|
The output file holds a vector of coefficients in increasing order of dimension;
|
|
that is, the offset term (\f$\beta_0\f$), the coefficient for dimension 1
|
|
(\f$\beta_1\f$, then dimension 2 (\f$\beta_2\f$) and so forth, as well as the
|
|
intercept. This executable can also predict the \f$y\f$ values of a second
|
|
dataset based on the computed coefficients.
|
|
|
|
Below are several examples of simple usage (and the resultant output). The '-v'
|
|
option is used so that verbose output is given. Further documentation on each
|
|
individual option can be found by typing
|
|
|
|
@code
|
|
$ linear_regression --help
|
|
@endcode
|
|
|
|
@subsection cli_ex1_lrtut One file, generating the function coefficients
|
|
|
|
@code
|
|
$ linear_regression --input_file dataset.csv -v
|
|
[INFO ] Loading 'dataset.csv' as CSV data.
|
|
[INFO ] Saving CSV data to 'parameters.csv'.
|
|
[INFO ]
|
|
[INFO ] Execution parameters:
|
|
[INFO ] help: false
|
|
[INFO ] info: ""
|
|
[INFO ] input_file: dataset.csv
|
|
[INFO ] input_responses: ""
|
|
[INFO ] output_file: parameters.csv
|
|
[INFO ] output_predictions: predictions.csv
|
|
[INFO ] test_file: ""
|
|
[INFO ] verbose: true
|
|
[INFO ]
|
|
[INFO ] Program timers:
|
|
[INFO ] load_regressors: 0.006461s
|
|
[INFO ] regression: 0.000347s
|
|
[INFO ] total_time: 0.026589s
|
|
@endcode
|
|
|
|
Convenient program timers are given for different parts of the calculation at
|
|
the bottom of the output, as well as the parameters the simulation was run with.
|
|
Now, if we look at the output file, which, unless specified, is parameters.csv:
|
|
|
|
@code
|
|
$ cat dataset.csv
|
|
0,0
|
|
1,1
|
|
2,2
|
|
3,3
|
|
4,4
|
|
|
|
$ cat parameters.csv
|
|
-0.0000000000e+00,1.0000000000e+00
|
|
@endcode
|
|
|
|
As you can see, the function for this input is \f$f(y)=0+1x_1\f$. Keep in mind
|
|
that in this example, the regressors for the dataset are the second column.
|
|
That is, the dataset is one dimensional, and the last column has the \f$y\f$
|
|
values, or responses, for each row. You can specify these responses in a
|
|
separate file if you want, using the --input_responses, or -r, option.
|
|
|
|
@subsection cli_ex2_lrtut Compute model and predict at the same time
|
|
|
|
@code
|
|
$ linear_regression --input_file dataset.csv --test_file predict.csv -v
|
|
[INFO ] Loading 'dataset.csv' as CSV data.
|
|
[INFO ] Saving CSV data to 'parameters.csv'.
|
|
[INFO ] Loading 'predict.csv' as CSV data.
|
|
[INFO ] Saving CSV data to 'predictions.csv'.
|
|
[INFO ]
|
|
[INFO ] Execution parameters:
|
|
[INFO ] help: false
|
|
[INFO ] info: ""
|
|
[INFO ] input_file: dataset.csv
|
|
[INFO ] input_responses: ""
|
|
[INFO ] model_file: ""
|
|
[INFO ] output_file: parameters.csv
|
|
[INFO ] output_predictions: predictions.csv
|
|
[INFO ] test_file: predict.csv
|
|
[INFO ] verbose: true
|
|
[INFO ]
|
|
[INFO ] Program timers:
|
|
[INFO ] load_regressors: 0.000360s
|
|
[INFO ] load_test_points: 0.000090s
|
|
[INFO ] prediction: 0.000006s
|
|
[INFO ] regression: 0.000335s
|
|
[INFO ] total_time: 0.001522s
|
|
|
|
$ cat dataset.csv
|
|
0,0
|
|
1,1
|
|
2,2
|
|
3,3
|
|
4,4
|
|
|
|
$ cat parameters.csv
|
|
-0.0000000000e+00,1.0000000000e+00
|
|
|
|
$ cat predict.csv
|
|
2
|
|
3
|
|
4
|
|
|
|
$ cat predictions.csv
|
|
2.0000000000e+00
|
|
3.0000000000e+00
|
|
4.0000000000e+00
|
|
@endcode
|
|
|
|
We used the same dataset, so we got the same parameters. The key thing to note
|
|
about the predict.csv dataset is that it has the same dimensionality as the
|
|
dataset used to create the model, one. Generally, if the model generating
|
|
dataset has \f$d\f$ dimensions, so must the dataset we want to predict for.
|
|
|
|
@subsection cli_ex3_lrtut Prediction using a precomputed model
|
|
|
|
@code
|
|
$ linear_regression --model_file parameters.csv --test_file predict.csv -v
|
|
[INFO ] Loading 'parameters.csv' as CSV data.
|
|
[INFO ] Loading 'predict.csv' as CSV data.
|
|
[INFO ] Saving CSV data to 'predictions.csv'.
|
|
[INFO ]
|
|
[INFO ] Execution parameters:
|
|
[INFO ] help: false
|
|
[INFO ] info: ""
|
|
[INFO ] input_file: ""
|
|
[INFO ] input_responses: ""
|
|
[INFO ] model_file: parameters.csv
|
|
[INFO ] output_file: parameters.csv
|
|
[INFO ] output_predictions: predictions.csv
|
|
[INFO ] test_file: predict.csv
|
|
[INFO ] verbose: true
|
|
[INFO ]
|
|
[INFO ] Program timers:
|
|
[INFO ] load_model: 0.009519s
|
|
[INFO ] load_test_points: 0.000067s
|
|
[INFO ] prediction: 0.000007s
|
|
[INFO ] total_time: 0.010081s
|
|
|
|
$ cat parameters.csv
|
|
-0.0000000000e+00,1.0000000000e+00
|
|
|
|
$ cat predict.csv
|
|
2
|
|
3
|
|
4
|
|
|
|
$ cat predictions.csv
|
|
2.0000000000e+00
|
|
3.0000000000e+00
|
|
4.0000000000e+00
|
|
@endcode
|
|
|
|
Further documentation on options should be found by using the --help option.
|
|
|
|
@section linreg_lrtut The 'LinearRegression' class
|
|
|
|
The 'LinearRegression' class is a simple implementation of linear regression.
|
|
|
|
Using the LinearRegression class is very simple. It has two available
|
|
constructors; one for generating a model from a matrix of predictors and a
|
|
vector of responses, and one for loading an already computed model from a given
|
|
file.
|
|
|
|
The class provides one method that performs computation:
|
|
@code
|
|
void Predict(const arma::mat& points, arma::vec& predictions);
|
|
@endcode
|
|
|
|
Once you have generated or loaded a model, you can call this method and pass it a
|
|
matrix of data points to predict values for using the model. The second parameter,
|
|
predictions, will be modified to contain the predicted values corresponding to
|
|
each row of the points matrix.
|
|
|
|
@subsection linreg_ex1_lrtut Generating a model
|
|
|
|
@code
|
|
#include <mlpack/methods/linear_regression/linear_regression.hpp>
|
|
|
|
using namespace mlpack::regression;
|
|
|
|
arma::mat data; // The dataset itself.
|
|
arma::vec responses; // The responses, one row for each row in data.
|
|
|
|
// Regress.
|
|
LinearRegression lr(data,responses);
|
|
|
|
// Get the parameters, or coefficients.
|
|
arma::vec parameters = lr.Parameters();
|
|
@endcode
|
|
|
|
@subsection linreg_ex2_lrtut Setting a model
|
|
|
|
Assuming you already have a model and do not need to create one, this is how
|
|
you would set the parameters for a LinearRegression instance.
|
|
|
|
@code
|
|
arma::vec parameters; // Your model.
|
|
|
|
LinearRegression lr(); // Create a new LinearRegression instance or reuse one.
|
|
lr.Parameters() = parameters; // Set the model.
|
|
@endcode
|
|
|
|
@subsection linreg_ex3_lrtut Load a model from a file
|
|
|
|
If you have a generated model in a file somewhere you would like to load and use,
|
|
you can simply pass it to the LinearRegression initializer like so.
|
|
|
|
@code
|
|
std::string filename; // The path and name of your file.
|
|
|
|
LinearRegression lr(filename); // Will load the model internally.
|
|
@endcode
|
|
|
|
@subsection linreg_ex4_lrtut Prediction
|
|
|
|
Once you have generated or loaded a model using one of the aforementioned methods,
|
|
you can predict values for a dataset.
|
|
|
|
@code
|
|
|
|
LinearRegression lr();
|
|
// Load or generate your model.
|
|
|
|
// The dataset we want to predict on; each row is a data point.
|
|
arma::mat points;
|
|
// This will store the predictions; one row for each point.
|
|
arma::vec predictions;
|
|
|
|
lr.Predict(points, predictions); // Predict.
|
|
|
|
// Now, the vector 'predictions' will contain the predicted values.
|
|
@endcode
|
|
|
|
@subsection further_doc_lrtut Further documentation
|
|
|
|
For further documentation on the LinearRegression class, consult the
|
|
\ref mlpack::regression::LinearRegression "complete API documentation".
|
|
|
|
*/
|