apaam/mlpack
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
28,955 Commits 1 Branch 0 Tags
dfcf40a71f3cdf4ee80fc29ecab48d2a72c8fcd8
Commit Graph
28955 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Adam KroppandRyan Curtin dfcf40a71f Update src/mlpack/methods/ann/layer/multihead_attention.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 15:30:25 -05:00
Adam KroppandRyan Curtin d00446231a Update src/mlpack/methods/ann/layer/multihead_attention.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 15:29:23 -05:00
Adam KroppandRyan Curtin 4244a337a9 Update src/mlpack/methods/ann/activation_functions/spline_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 15:27:46 -05:00
Adam KroppandRyan Curtin cd2c13e4b4 Update src/mlpack/methods/ann/activation_functions/tanh_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 15:27:00 -05:00
Adam KroppandRyan Curtin 26b031110e Update src/mlpack/methods/ann/activation_functions/tanh_exponential_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 15:26:11 -05:00
Adam KroppandRyan Curtin 1cf92e5f30 Update src/mlpack/methods/ann/activation_functions/swish_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 15:24:57 -05:00
Adam KroppandRyan Curtin c8c8dbe020 Update src/mlpack/methods/ann/activation_functions/softsign_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 15:05:10 -05:00
Adam KroppandRyan Curtin e8f64d5c40 Update src/mlpack/methods/ann/activation_functions/softplus_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:52:50 -05:00
Adam KroppandRyan Curtin 8914a1db6a Update src/mlpack/methods/ann/activation_functions/silu_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:52:17 -05:00
Adam KroppandRyan Curtin ce538a8fe4 Update src/mlpack/methods/ann/activation_functions/rectifier_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:51:53 -05:00
Adam KroppandRyan Curtin 7ab7d41674 Update src/mlpack/methods/ann/activation_functions/quadratic_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:51:30 -05:00
Adam KroppandRyan Curtin 4f66883d5e Update src/mlpack/methods/ann/activation_functions/poisson1_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:46:28 -05:00
Adam KroppandRyan Curtin 6d6768f06b Update src/mlpack/methods/ann/activation_functions/multi_quadratic_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:39:01 -05:00
Adam KroppandRyan Curtin d99c88ecd8 Update src/mlpack/methods/ann/activation_functions/rectifier_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:30:40 -05:00
Adam KroppandRyan Curtin 18bac82267 Update src/mlpack/methods/ann/activation_functions/hard_sigmoid_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:29:12 -05:00
Adam KroppandRyan Curtin 1456ab9873 Update src/mlpack/methods/ann/activation_functions/hard_swish_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:28:48 -05:00
Adam KroppandRyan Curtin 8de759ebff Update src/mlpack/methods/ann/activation_functions/gelu_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:28:35 -05:00
Adam KroppandRyan Curtin 8f89db7fb6 Update src/mlpack/methods/ann/activation_functions/lisht_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:28:12 -05:00
Adam KroppandRyan Curtin bc94bb9b84 Update src/mlpack/methods/ann/activation_functions/mish_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:27:54 -05:00
Adam KroppandRyan Curtin 7846bc0623 Update src/mlpack/methods/ann/activation_functions/gaussian_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:27:34 -05:00
Adam KroppandRyan Curtin ae578a7347 Update src/mlpack/methods/ann/activation_functions/gaussian_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:27:07 -05:00
Adam Kropp e3a293b157 simplify dy expression 2023-11-13 14:26:10 -05:00
Adam Kropp a4ead8bcef author 2023-11-13 14:06:22 -05:00
Adam Kropp 214a731dd7 Merge remote-tracking branch 'origin/adapt_multihead' into adapt_multihead 2023-11-13 14:06:09 -05:00
Adam Kropp d9f9e4f123 Style changes, author 2023-11-13 14:05:47 -05:00
Adam KroppandRyan Curtin 68f15ed77a Update src/mlpack/methods/ann/activation_functions/elish_function.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:03:27 -05:00
Adam KroppandRyan Curtin 9451a8fce9 Update src/mlpack/tests/ann/activation_functions_test.cpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 14:02:48 -05:00
Adam KroppandRyan Curtin 008aabb8a0 Update src/mlpack/tests/ann/layer/multihead_attention.cpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 13:50:42 -05:00
Adam KroppandRyan Curtin fb94b6d174 Update src/mlpack/tests/ann/activation_functions_test.cpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 13:50:30 -05:00
Adam KroppandRyan Curtin 89f93d1272 Update src/mlpack/tests/ann/layer/layer_norm.cpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 13:50:14 -05:00
Adam KroppandRyan Curtin 5f385ff946 Update src/mlpack/methods/ann/layer/multihead_attention.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 13:49:58 -05:00
Adam KroppandRyan Curtin eb3e7f2a48 Update src/mlpack/methods/ann/layer/multihead_attention.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-13 13:47:17 -05:00
Adam Kropp f5be365000 Correct calculation of srcSeqLen with more than 2 input dimensions 2023-11-13 07:55:46 -05:00
Adam Kropp c990b93917 Comment unused parameter 2023-11-10 16:12:34 -05:00
Adam Kropp 39f7a10fdc Comment out unused params 2023-11-10 15:50:28 -05:00
Adam Kropp 2bdae7d438 Add MultiheadAttention test, fix dimension calculation 
Also added some language to the comments about the sequence lengths.
 2023-11-10 14:27:00 -05:00
Adam Kropp 3363707881 Add in LayerNorm Test 2023-11-10 08:29:13 -05:00
Adam Kropp 7b5e4d267b Merge remote-tracking branch 'upstream/master' into adapt_multihead 2023-11-09 16:01:28 -05:00
Adam Kropp 65b4878ad0 Clairfy input dimensions documentation, check valid input dims 2023-11-08 09:18:21 -05:00
Adam KroppandRyan Curtin d91c5f0f31 Update src/mlpack/methods/ann/layer/c_relu_impl.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-08 08:14:11 -05:00
Adam KroppandRyan Curtin f6d78abf42 Update src/mlpack/methods/ann/layer/multihead_attention.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-08 08:13:05 -05:00
Adam KroppandRyan Curtin 1b8313eafb Update src/mlpack/methods/ann/layer/flexible_relu_impl.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-08 08:05:27 -05:00
Adam KroppandRyan Curtin c40787c3f8 Update src/mlpack/methods/ann/layer/multihead_attention_impl.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-08 08:05:09 -05:00
Adam KroppandRyan Curtin ef3183ee68 Update src/mlpack/methods/ann/layer/multihead_attention_impl.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-08 08:04:56 -05:00
Adam KroppandRyan Curtin 517bb0b27f Update src/mlpack/methods/ann/layer/multihead_attention.hpp 
Co-authored-by: Ryan Curtin <ryan@ratml.org>
 2023-11-08 07:55:13 -05:00
Adam Kropp cc99b4ac90 Update activation functions to take input and output values 
Updated test cases as well.
 2023-11-08 00:13:57 -05:00
Ryan Curtin 7af47e22bd Fix numClasses member in SoftmaxRegression::Train() (#3553) 2023-11-05 14:32:01 +01:00
Adam Kropp 45968109ac Add LayerNorm and MultiAttentionHead tests 2023-11-03 12:14:19 -04:00
Adam Kropp 360c17c7e5 Fix c_relu test and backward method 
Defined the derivative at 0 to be exactly 0 (rather than -1 or 1).  Fixed the expected derivatives in the test case.
 2023-11-03 11:07:04 -04:00
Adam Kropp b7ab3ba063 Fix ftswish to use input instead of output 
I corrected the test case too (calculating the values separately).  It appears that they had been calculated based on the ouputs, which from my reading of the paper is incorrect.
 2023-11-03 08:34:17 -04:00