275 lines
11 KiB
Plaintext
275 lines
11 KiB
Plaintext
MLPACK is the first comprehensive scalable machine learning library.
|
|
Developed by the Fundamental Algorithmic and Statistical Tools
|
|
laboratory (FASTlab), MLPACK and its core functions library FASTlib
|
|
are the much needed filling of an existing void. Previously,
|
|
researchers had to either (a) settle for poorly-scaling collections of
|
|
methods implemented for academic purposes, (b) hunt down the often
|
|
difficult to find and difficult to apply yet fast code writen by
|
|
algorithms' developers, or (c) reimplement solutions to their specific
|
|
analysis problems from scratch. With MLPACK, we offer a fourth option,
|
|
in which researchers may find all the methods they need designed
|
|
favoring both speed and usability.
|
|
|
|
1. CONTENTS
|
|
-----------
|
|
|
|
MLPACK currently includes the following algorithms:
|
|
|
|
- allknn - A dual tree based $k$-nearest neighbor classifier using
|
|
kd-trees.
|
|
|
|
- allnn - A dual-tree based $k$-nearest neighbor classifier using
|
|
kd-trees, optimized for $k = 1$.
|
|
|
|
- fastica - Implements the FastICA Algorithm for Independent Component
|
|
Analysis using fixed-point optimization with various
|
|
independence-minded contrast functions.
|
|
|
|
- hmm - Implements 3 types of Hidden Markov Models: discrete, gaussian
|
|
and mixture of gaussians.
|
|
|
|
- infomax_ica - Implements the Information Maximisation algorithm for
|
|
Independent Component Analysis.
|
|
|
|
- kalman - Implements of the Kalman filter ensuring the positive
|
|
definiteness of the error covariance matrices by using QR and Cholesky
|
|
factorizations in both the measurement and time update steps.
|
|
|
|
- kde - Implements the following versions of kernel density
|
|
estimation: Using depth first dual tree, multidimensional fast fourier
|
|
transform, multidimensional fast gaussian transform and
|
|
multidimensional improved fast gaussian transform.
|
|
|
|
- mog - Implements parametric estimation of a mixture of Gaussians
|
|
using two different loss functions - maximum likelihood and the L2
|
|
error.
|
|
|
|
- naive_bayes - Implements the Naive Bayes Classifier.
|
|
|
|
- optimization - Implements two optimizers - The Nelder-Mead algorithm
|
|
and the Quasi-Newton algorithm.
|
|
|
|
- series_expansion - Implements the series expansion needed for the
|
|
fast $N$-body algorithm. Gaussian series expansion in $O(D^p)$ and
|
|
$O(p^D)$ are implemented.
|
|
|
|
- svm - Implements the Support Vector Machine classifier and
|
|
regression. Includes the Sequential Minimial Optimization algorithm.
|
|
|
|
All algorithms are available in both executable and linkable form.
|
|
[More...]
|
|
|
|
2. HOWTO
|
|
--------
|
|
|
|
2.1. Quick start
|
|
----------------
|
|
|
|
The CMake build system has been tested to work on Linux or Windows XP/Vista
|
|
under Cygwin using gcc4 and gcc3.4. It will probably work on Mac OSX using gcc
|
|
also. In the future, we will test it under the other build systems that CMake
|
|
supports, like Visual Studio and Eclipse, but for now a gcc/make development
|
|
environment is your best bet.
|
|
|
|
1. Use the Cygwin installer or your Linux package manager to install:
|
|
|
|
* gcc
|
|
* g++
|
|
* g77 or gfortran (cmake wants a Fortran compiler--make it happy)
|
|
* make
|
|
* cmake 2.6 or higher
|
|
* BLAS & LAPACK (usually separate in Linux, both included as "lapack" under
|
|
"Math" in the Cygwin installer)
|
|
* The library opt++, needed for optimization routines. This is technically
|
|
optional, but very many machine learning algorithms need to do optimization.
|
|
|
|
Optional, but strongly recommended:
|
|
* Trilinos 10.0 or above, if you want sparse matrices.
|
|
* The Boost C++ libraries (not directly used, but useful, may become mandatory
|
|
later)
|
|
|
|
2. Install the opt++ optimization library. Again, this is optional, but will
|
|
be useful if you are developing new ML algorithms. The opt++ library seems to
|
|
only be supported on systems with GNU autotools. MS VC++ users are on your own.
|
|
|
|
* The source code can be obtained from https://software.sandia.gov/opt++/
|
|
* Extract the tar file to some OPTPP_DIR.
|
|
* In the file OPTPP_DIR/newmat11/include.h, uncomment the line
|
|
"#define SETUP_C_SUBSCRIPTS".
|
|
* Now just follow the directions in the INSTALL file for opt++. By default, it
|
|
installs in /usr/local. It uses some fairly generic filenames, so you might
|
|
have to install in another location if you get conflicts.
|
|
|
|
3. If you want sparse matrices, install the Trilinos package.
|
|
|
|
* The source code can be obtained from http://trilinos.sandia.gov/
|
|
* Extract the tar file to some directory and cd there.
|
|
* Create a build directory there.
|
|
* Trilinos has very many configuration options. For the minimal configuration
|
|
that will work:
|
|
- cp FASTLIB/script/fastlib-trilinos-minimal-serial-debug-cmake \
|
|
TRILINOS/build
|
|
- Optionally edit the script to make CMAKE_INSTALL_PREFIX point where you'd
|
|
like. Fastlib will automatically check standard install dirs and
|
|
/opt/trilinos during its build.
|
|
- Make any other changes you'd like (see TRILINOS/sampleScripts and
|
|
TRILINOS/cmake/TrilinosCMakeQuickstart.txt for ideas) and run the script.
|
|
- You may get an error saying that you don't have a recent enough version of
|
|
CMake. You can install a newer version, but the configuration script
|
|
above seems to work with CMake 2.6.2. Just change the first line of
|
|
TRILINOS/CMakeLists.txt to read "CMAKE_MINIMUM_REQUIRED(VERSION 2.6)"
|
|
|
|
4. Now all dependencies should be installed. For the rest of the instructions,
|
|
assume you've extracted the MLPACK tar or zip file to /path/to/fastlib (it's
|
|
the directory this file is in).
|
|
|
|
5. After this, you will need to make sure your environment variables are set up
|
|
properly. Set FASTLIBPATH to this directory, e.g.
|
|
|
|
export FASTLIBPATH=/path/to/fastlib (bash)
|
|
setenv FASTLIBPATH=/path/to/fastlib (csh/tcsh)
|
|
|
|
6. Change to the fastlib subdirectory and create a build directory:
|
|
|
|
cd $FASTLIBPATH/fastlib
|
|
mkdir build (this can actually go anywhere you want)
|
|
cd build
|
|
|
|
7. CMake's default is to install things in /usr/local/, which may not be what
|
|
you want. So, let's configure our cmake project:
|
|
|
|
% ccmake .. (if in build subdir)
|
|
% ccmake $FASTLIBPATH/fastlib (if you put the build dir some weird place)
|
|
|
|
This will bring up a curses interface (GUIs exist for cmake; feel free to
|
|
install one). Hit "c" to configure. You should see a screen like this:
|
|
|
|
CMAKE_BUILD_TYPE *
|
|
CMAKE_INSTALL_PREFIX */usr/local
|
|
FASTLIB_WITH_OPTIMIZERS *ON
|
|
FASTLIB_WITH_SPARSE *ON
|
|
OPTPP_BASE_DIR */usr/local
|
|
TRILINOS_BASE_DIR */usr/local
|
|
|
|
|
|
From the CMake docs: "CMAKE_BUILD_TYPE... Possible values are empty, Debug,
|
|
Release, RelWithDebInfo and MinSizeRel. This variable is only supported for
|
|
make based generators."
|
|
|
|
One choice for the install prefix that will make things similar to the old
|
|
build system is to use FASTLIBPATH as the CMAKE_INSTALL_PREFIX. This will put
|
|
the libraries and header files in the lib/ and include/ subdirectories. So, go
|
|
to the CMAKE_INSTALL_PREFIX line, hit Enter, change it to whatever you want,
|
|
and hit Enter again.
|
|
|
|
Change FASTLIB_WITH_OPTIMIZERS to OFF if you don't want to include opt++
|
|
support (e.g. if the opt++ build failed for some reason)
|
|
|
|
Change FASTLIB_WITH_SPARSE to OFF if you don't want to include sparse
|
|
matrices (e.g. if the Trilinos build failed for some reason)
|
|
|
|
Hit "c" to update the configuration. If Trilinos or opt++ is required but not
|
|
found, there should be an error. Otherwise, hit "g" to generate the build
|
|
files and exit. There should now be a Makefile in the build directory. Now
|
|
run
|
|
|
|
% make (or "VERBOSE=1 make &> make.log" if you want to know all the details)
|
|
% make install
|
|
|
|
Let's assume you installed things to $FASTLIBPATH. If you look in
|
|
$FASTLIBPATH/lib and $FASTLIBPATH/include, you should see a static library and
|
|
a bunch of header files. At this point, you should be able to develop your
|
|
code using whatever tools you want.
|
|
|
|
8. Quick Test
|
|
|
|
For instance, let's test out the install by compiling one of the mlpack
|
|
programs by hand:
|
|
|
|
% cd $FASTLIBPATH/mlpack/allnn
|
|
% g++ allnn_test.cc -o allnn_test -DDISABLE_DISK_MATRIX -L$FASTLIBPATH/lib \
|
|
-I$FASTLIBPATH/include -lfastlib -llapack
|
|
% ./allnn_test (tests should pass)
|
|
|
|
That "-DDISABLE_DISK_MATRIX" is an annoying define that you'll have to add
|
|
until we add support for the memory manager to the CMake build.
|
|
|
|
An example of building a program that requires opt++:
|
|
% cd $FASTLIBPATH/examples/optimization
|
|
% g++ optimizer_tests.cc -o optimizer_tests -DDISABLE_DISK_MATRIX \
|
|
-L$FASTLIBPATH/lib -I$FASTLIBPATH/include -lfastlib -lopt -lnewmat -llapack
|
|
|
|
You might have to add the opt++ library and include paths if you installed it
|
|
somewhere nonstandard.
|
|
|
|
9. Compiling MLPACK code with CMake
|
|
|
|
% cd $FASTLIBPATH/mlpack
|
|
% mkdir build && cd build
|
|
% ccmake ..
|
|
|
|
Hit 'c' to configure.
|
|
You have to tell CMake where to find the fastlib you just installed. By
|
|
default, it looks in /usr/local, so it won't find the library. Change
|
|
FASTLIB_BASE_DIR to where you installed fastlib (e.g. the path in $FASTLIBPATH)
|
|
and hit 'c' again. FASTLIB_LIB should now be correct. Hit 'g' to generate the
|
|
Makefiles.
|
|
|
|
% make
|
|
|
|
This builds libraries and executables for each algorithm in its corresponding
|
|
subdirectory. You can run the programs from there. There is no sensible
|
|
'make install' target yet.
|
|
|
|
One easy(ish) way to develop your code would be to make a subdirectory for it
|
|
in mlpack. Then you could copy and modify the CMakeLists.txt file from another
|
|
subdirectory to create your libraries and executables. You also have to modify
|
|
the CMakeLists.txt file in the mlpack directory to recurse into your
|
|
subdirectory. Then, when you configure mlpack, it will also create make
|
|
targets for your code.
|
|
|
|
FUTURE PLANS
|
|
------------
|
|
|
|
MLPACK is growing quickly, and will soon also include:
|
|
|
|
- Affinity Propagation.
|
|
|
|
- C # versions of nearest neighbor.
|
|
|
|
- Convex optimization routines.
|
|
|
|
- Dual-tree nearest neighbor algorithm using Cover trees.
|
|
|
|
- Disk-based algorithms using memory-mapped file implementation.
|
|
|
|
- Euclidean Minimum Spanning Tree.
|
|
|
|
- Graphical model inference.
|
|
|
|
- Kernel Discriminant Analysis
|
|
|
|
- Local linear regression.
|
|
|
|
- Manifold learning algorithms (diffusion maps, Laplacian Eigenmaps,
|
|
LLE, Isomap).
|
|
|
|
- Nonnegative matrix factorization and many of its variants.
|
|
|
|
- Nonnegative SVM.
|
|
|
|
- Orthogonal range search.
|
|
|
|
- Ranking SVM.
|
|
|
|
- Sparse KDE using QP.
|
|
|
|
Happy coding!
|
|
|
|
CONTACTS
|
|
--------
|
|
Please contact the following authors of the code for any problems:
|
|
Ryan Riegel (rriegel@cc.gatech.edu)
|
|
Nikolaos Vasiloglou (nvasil@ieee.org)
|
|
Dongryeol Lee (dongryel@cc.gatech.edu)
|