Momentum Optimizer
Backpropagation, Training & Optimization DS practice problem on Onlearn.
Difficulty: easy.
Topics: Understanding Momentum Optimizer, Momentum Coefficient, Velocity Accumulation, Broadcasting Rules, Parameter Update Rule, Gradient Descent, Numerical Optimization, Calculus, Linear Algebra, Computational Statistics, Software Engineering, Gradient-Based Optimization, Vectorized Operations, Hyperparameter Tuning, Stateful Iterative Algorithms, Numerical Stability.
Implement the momentum optimizer update step function. Your function should take the current parameter value, gradient, and velocity as inputs, and return the updated parameter value and new velocity. The function should also handle scalar and array inputs.