Nesterov Accelerated Gradient Optimizer
Calculus & Optimization DS practice problem on Onlearn.
Difficulty: easy.
Topics: Understanding Nesterov Accelerated Gradient Optimizer, Look-ahead Gradient, Velocity Update, Damping Factor, Parameter Manifold, First-order Derivative, Mathematical Foundations, Numerical Analysis, Deep Learning Optimization, Computational Linear Algebra, Stochastic Processes, Gradient Descent Variants, Momentum-based Methods, Iterative Optimization Algorithms, Vectorized Array Operations, Hyperparameter Tuning.
Implement the Nesterov Accelerated Gradient (NAG) optimizer update step function. Your function should take the current parameter value, gradient function, and velocity as inputs, and return the updated parameter value and new velocity. The function should use the "look ahead" approach where momentum is applied before computing the gradient, and should handle both scalar and array inputs.