Parameter Updaters¶
Abstract Base Class¶
Simple SGD Update¶
Momentum Update¶
Nesterov Momentum Update¶
-
class
hebel.parameter_updaters.
NesterovMomentumUpdate
(model)¶ -
post_gradient_update
(gradients, batch_size, learning_parameters, stream=None)¶ Second step of Nesterov momentum method: take step in direction of new gradient and update velocity
-
pre_gradient_update
()¶ First step of Nesterov momentum method: take step in direction of accumulated gradient
-