* Refactor calculatedObjectives in NSGA2 optimizer
Signed-off-by: yexiaochuan <tap91624@gmail.com>
* Use Mat instead of Cube.
---------
Signed-off-by: yexiaochuan <tap91624@gmail.com>
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Deprecate the Lambda() and Sigma() functions for AugLagrangian.
* Add utility struct to differentiate between matrices and callbacks.
* Update HISTORY.
* Fix overload issues with LRSDP and new AugLagrangian types.
* Fix some documentation to kick off another build. (I think there was an issue with dealgood.)
* Remove spaces to kick the build off again.
* Normalize step size to the batch size used.
* Update history.
* Don't use if constexpr.
* Fix all step sizes in tests to account for new behavior.
* Retune DemonSGD tests to reduce failure probability.
* Merge Bandicoot tests into existing TEMPLATE_TEST_CASEs.
* Fix merge artifact.
* Fix compilation errors.
* Fix MOEAD test to be more robust.
* Oops, fix missing colon.
* Fix compilation issues.
* Remove arma:: for randu call.
* Try to fix AppVeyor build.
* Bump version number to fix mlpack integration build.
* Adapt FBS/FISTA/FASTA to bandicoot tests.
* Keep bandicoot implementation separate.
* Print which tests take a long time so I can prune down the runtime.
* Remove some tests that take a long long time.
* Filter out some more long-running GPU tests.
* Implement forward-backward splitting (FBS) and some proximal operators.
* Minor cleanups for FrankWolfe.
* Allow FunctionTest() to take instantiated functions.
* Some minor fixes to some test functions.
* Add tests for FBS and proximal operators.
* Clean up documentation.
* Add documentation for FBS to optimizers.md.
* Add some documentation about proximal operators and functions.
* Fix header guard name.
* Remove unused member.
* Some minor FBS cleanups.
* Add QuadraticFunction for testing.
* Try to clean up L1 constraint application a little bit.
* Add FISTA and tests.
* Document FISTA.
* Remove inaccurate comment.
* Clean up to match Beck and Teboulle paper instead of FASTA paper.
* Some minor efficiency cleanups.
* Add debugged and working FASTA implementation.
* Add documentation for FASTA.
* Make L1Constraint tests robust to the element type.
* Minor fixes for compilation and tests.
* Try to fix AppVeyor build.
* Bump version number to fix mlpack integration build.
* Add changelog entry for FBS/FISTA/FASTA.
* Fixes for FP16 compilation and failing tests.
* Fix compilation issue.
* Add instantiated versions of tolerances for older compilers / C++14.
* Adjust tolerances.
* Try to make MSVC happy with where values are defined vs. declared.
* Make sure that bandicoot is included if needed.
* Remove deprecated functionality for the ensmallen 3.x release.
* Bump version so that anyone doing version checks gets correct details.
* Update HISTORY.md.
* Use clearer documentation for how to convert old code.
* Try to fix AppVeyor build.
* Some changes to make the IPOP-CMAES RastriginFunction test faster.
* Normalize step size to the batch size used.
* Update history.
* Don't use if constexpr.
* Fix all step sizes in tests to account for new behavior.
* Retune DemonSGD tests to reduce failure probability.
* Fix merge artifact.
* Fix MOEAD test to be more robust.
* Add CMake modules to find Bandicoot libraries and includes.
* Add initial GitHub Actions to build against Bandicoot and run the tests.
* 1. Add type proxies to select the right type based on the used backend and datatype.
2. Update IQN optimizer to build against Armadillo and Bandicoot.
3. Update IQN test cases to test against Armadillo and Bandicoot backend.
4. Update logisitic regression function, to support Armadillo and Bandicoot.
* Refactor CNE optimizer and tests cases to work with Armadillo and Bandicoot datatypes.
* Refactor Katyusha optimizer and tests cases to work with Armadillo and Bandicoot datatypes.
* Refactor optimizers to work with Armadillo and Bandicoot datatypes:
1. AdaBelief
2. AdaBound
3. AMSBound
4. AdaGrad
5. Adam
6. Adamax
7. AMSGrad
8. Nadam
9. Nadamax
10. OptimisticAdam
11. Eve
12. FTML
13. LBFGS
14. Lookahead
15. Padam
16. QHAdam
17. SMORMS3
18. SpaleraSGD
19. SPSA
20. SWATS
21. WNGrad
22. Yogi
23. MomentumSGD
24. NesterovMomentum
25. QuasiHyperbolic
26. SGD
27. SGDR
28. SnapshotSGDR
29. CNE
30. CMAES
31. ActiveCMAES
32. PSO
33. LBestPSO
34. MOEAD
35. AugmentedLagrangian
* Refactor optimizers to work with Armadillo and Bandicoot datatypes:
1. AdaDelta
2. AdaSqrt
3. BigBatchSGD
4. CoordinateDescent (CD)
5. GradientDescent
6. GridSearch
7. NSGA2
8. ParallelSGD
* Refactor optimizers to work with Armadillo and Bandicoot datatypes:
1. DifferentialEvolution (DE)
2. RMSProp
3. SimulatedAnnealing (SA)
4. SARAH
5. SARAH+
6. SVRG
7. BarzilaiBorweinDecay
8. DemonAdam
9. DemonSGD
* Use inverse transform sampling to replicating arma::randg sampling since randg is not supported in Bandicoot.
* Apply suggestions from code review by @rcurtin.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Group test cases and fix indentations.
* Update optimizer reference styling.
* Add option to build against Bandicoot (by default we build against Bandicoot if found).
* Mention AugLagrangianType constructor in the optimizer documentation, that allows to use a different datatype i.e. arma::fvec or coot::fvec.
* Use data type specific element type for the bounds parameter.
* Make sure we follow the max character line limitation from the style guide.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Remove backports for outdates armadillo versions.
* Indentation fixes.
* Indentation fixes.
* Use the correct data type during testing.
* Do not use the namespace twice.
* Test against CPU/OpenCL/CUDA backend.
* Use the correct test case type.
* We don't need Bandicoot when we test against Armadillo (CPU) backend.
* Use the correct minimal Bandicoot version.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Comment on the added proxies.
* Merge arma and coot traits into a single file.
* Fix deprecation warning from newer Armadillo.
* Fix spacing.
* Add logging output for PSO.
* Adapt problems to be templatized.
* Templatize tests to use ENS_TEST_TYPES.
* Fix name of AdaGrad in the documentation.
* Apply suggestions from code review.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Update about half the optimizers to use FP16.
* Refactor ForwardType into two different structures to avoid std::conditional.
* Minor changes to get tests to pass on the Pi.
* Remove unused proxies.
* Update the other half of the optimizers to use FP16.
* Minor fixes to work on non-emulated FP16.
* Update to arma::hmat and enable sparse types.
* Make CMake standard easily configurable.
* Revert CategoryType to arma::Row<size_t> since that should always stay on the CPU.
* Add convenience IsMatrixType<> struct.
* Catch underflow for RMSprop.
* Revert to norm()-only implementation for gradient norm computation.
* Remove proxies, use using instead.
* Older Armadillo versions don't support changing sparse-to-dense and also element types.
* Remove sort_index() proxy and use using instead.
* Oops, fix compatibility conv_to implementation.
* Fix divide by zero condition.
* Individually accessed vector, so keep vector on the CPU, for faster access.
* Apply suggestions from code review
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Kepp the indicies on the CPU.
* Apply suggestions from code review.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Remove outdated test case.
* Use initialization list for the velocity vector.
* Apply suggestions from code review
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Readd removed test cases for MOEAD.
* Use sort_index for sorting that is supported for both armadillo and bandicoot.
* Revert changes to Parallel SGD optimizer, no benefit to upload to the GPU.
* Only reset if we haven't reached the end of the iterations.
* Use accu to the the sum of the vector.
* Use clamp instead of min, for a more efficient computation.
* Add missing data type.
* Use ENS_HAVE_COOT instead of USE_COOT and automatically detect when Bandicoot has already been included.
* Fix incorrect initialization list.
* Remove unnecessary GetFillType<MatType>::zeros, which does not work for sparse matrices.
* Clean up the last few proxy functions.
* Refactor problems to avoid adding extra template parameters.
* NSGA2: introduce a new Optimize function that takes the front and the paretoSet, so we can avoid introducing new template parameters.
* MOEAD: introduce a new Optimize function that takes the front and the paretoSet, so we can avoid introducing new template parameters.
* Remove unnecessary lr matrix.
* Minor cleanups for SA.
* Clean up member initialization for PSO.
* Update documentation for multi-objective optimizers.
* Update API for Snapshots().
* Keep the MOEAD front indices on the CPU, to be more efficient.
* Remove class template parameter for the bound parameter, to make the optimizer class easier.
* Use the correct location for the bandicoot wrapper.
* No need to set the backend for testing, we will test whatever was available during compilation.
* Use the correct def to enable the bandicoot backend.
* use the correct name for the def.
* Fixes so that Bandicoot actually gets detected and compiled in correctly.
* Oops, fix merge.
* Add tolerances for Bandicoot matrices.
* Fix missing typedef.
* Fix some failing tests.
* Fix minor compilation issues.
* Fix compilation errors and warnings.
* Minor issues discovered in self-review.
* Oops, fix missing type.
* Reduce batch size to try and prevent NaNs.
---------
Co-authored-by: Marcus Edel <marcus@urgs.org>
Co-authored-by: Marcus Edel <marcus.edel@fu-berlin.de>
* Add CMake modules to find Bandicoot libraries and includes.
* Add initial GitHub Actions to build against Bandicoot and run the tests.
* 1. Add type proxies to select the right type based on the used backend and datatype.
2. Update IQN optimizer to build against Armadillo and Bandicoot.
3. Update IQN test cases to test against Armadillo and Bandicoot backend.
4. Update logisitic regression function, to support Armadillo and Bandicoot.
* Refactor CNE optimizer and tests cases to work with Armadillo and Bandicoot datatypes.
* Refactor Katyusha optimizer and tests cases to work with Armadillo and Bandicoot datatypes.
* Refactor optimizers to work with Armadillo and Bandicoot datatypes:
1. AdaBelief
2. AdaBound
3. AMSBound
4. AdaGrad
5. Adam
6. Adamax
7. AMSGrad
8. Nadam
9. Nadamax
10. OptimisticAdam
11. Eve
12. FTML
13. LBFGS
14. Lookahead
15. Padam
16. QHAdam
17. SMORMS3
18. SpaleraSGD
19. SPSA
20. SWATS
21. WNGrad
22. Yogi
23. MomentumSGD
24. NesterovMomentum
25. QuasiHyperbolic
26. SGD
27. SGDR
28. SnapshotSGDR
29. CNE
30. CMAES
31. ActiveCMAES
32. PSO
33. LBestPSO
34. MOEAD
35. AugmentedLagrangian
* Refactor optimizers to work with Armadillo and Bandicoot datatypes:
1. AdaDelta
2. AdaSqrt
3. BigBatchSGD
4. CoordinateDescent (CD)
5. GradientDescent
6. GridSearch
7. NSGA2
8. ParallelSGD
* Refactor optimizers to work with Armadillo and Bandicoot datatypes:
1. DifferentialEvolution (DE)
2. RMSProp
3. SimulatedAnnealing (SA)
4. SARAH
5. SARAH+
6. SVRG
7. BarzilaiBorweinDecay
8. DemonAdam
9. DemonSGD
* Use inverse transform sampling to replicating arma::randg sampling since randg is not supported in Bandicoot.
* Apply suggestions from code review by @rcurtin.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Group test cases and fix indentations.
* Update optimizer reference styling.
* Add option to build against Bandicoot (by default we build against Bandicoot if found).
* Mention AugLagrangianType constructor in the optimizer documentation, that allows to use a different datatype i.e. arma::fvec or coot::fvec.
* Use data type specific element type for the bounds parameter.
* Make sure we follow the max character line limitation from the style guide.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Remove backports for outdates armadillo versions.
* Indentation fixes.
* Indentation fixes.
* Use the correct data type during testing.
* Do not use the namespace twice.
* Test against CPU/OpenCL/CUDA backend.
* Use the correct test case type.
* We don't need Bandicoot when we test against Armadillo (CPU) backend.
* Use the correct minimal Bandicoot version.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Comment on the added proxies.
* Merge arma and coot traits into a single file.
* Apply suggestions from code review.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Refactor ForwardType into two different structures to avoid std::conditional.
* Remove unused proxies.
* Revert CategoryType to arma::Row<size_t> since that should always stay on the CPU.
* Add convenience IsMatrixType<> struct.
* Individually accessed vector, so keep vector on the CPU, for faster access.
* Apply suggestions from code review
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Kepp the indicies on the CPU.
* Apply suggestions from code review.
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Remove outdated test case.
* Use initialization list for the velocity vector.
* Apply suggestions from code review
Co-authored-by: Ryan Curtin <ryan@ratml.org>
* Readd removed test cases for MOEAD.
* Use sort_index for sorting that is supported for both armadillo and bandicoot.
* Revert changes to Parallel SGD optimizer, no benefit to upload to the GPU.
* Only reset if we haven't reached the end of the iterations.
* Use accu to the the sum of the vector.
* Use clamp instead of min, for a more efficient computation.
* Add missing data type.
* Use ENS_HAVE_COOT instead of USE_COOT and automatically detect when Bandicoot has already been included.
* Fix incorrect initialization list.
* Remove unnecessary GetFillType<MatType>::zeros, which does not work for sparse matrices.
* Clean up the last few proxy functions.
* Refactor problems to avoid adding extra template parameters.
* NSGA2: introduce a new Optimize function that takes the front and the paretoSet, so we can avoid introducing new template parameters.
* MOEAD: introduce a new Optimize function that takes the front and the paretoSet, so we can avoid introducing new template parameters.
* Remove unnecessary lr matrix.
* Minor cleanups for SA.
* Clean up member initialization for PSO.
* Update documentation for multi-objective optimizers.
* Update API for Snapshots().
* Keep the MOEAD front indices on the CPU, to be more efficient.
* Remove class template parameter for the bound parameter, to make the optimizer class easier.
* Use the correct location for the bandicoot wrapper.
* No need to set the backend for testing, we will test whatever was available during compilation.
* Use the correct def to enable the bandicoot backend.
* use the correct name for the def.
* Fixes so that Bandicoot actually gets detected and compiled in correctly.
* Oops, fix merge.
* Oops, don't forget to resize the responses.
* Add new Optimize(...) method to AGEMOEA that takes the front and pareto set as input.
---------
Co-authored-by: Ryan Curtin <ryan@ratml.org>
This is an Armadillo 14.2.0 change that creates a fair amount of compilation
noise for mlpack right now.
There are likely more changes to be made but this will check the mechanics.
conradsnicta