Parameter Updaters¶
Abstract Base Class¶
Simple SGD Update¶
Momentum Update¶
Nesterov Momentum Update¶
- class hebel.parameter_updaters.NesterovMomentumUpdate(model)¶
- post_gradient_update(gradients, batch_size, learning_parameters, stream=None)¶
Second step of Nesterov momentum method: take step in direction of new gradient and update velocity
- pre_gradient_update()¶
First step of Nesterov momentum method: take step in direction of accumulated gradient