Adamax Optimizer
Backpropagation, Training & Optimization DS practice problem on Onlearn.
Difficulty: easy.
Topics: Understanding Adamax Optimizer, Infinity Norm, Bias Correction, Exponential Moving Average, First-Order Moment, Hyperparameter Tuning, Optimization Theory, Numerical Analysis, Deep Learning Foundations, Linear Algebra, Probability and Statistics, Adaptive Learning Rate Methods, Stochastic Gradient Descent Variants, Momentum-based Optimization, Parameter Update Rules, Moving Average Estimation.
Implement the Adamax optimizer update step function. Your function should take the current parameter value, gradient, and moving averages as inputs, and return the updated parameter value and new moving averages. The function should also handle scalar and array inputs and include bias correction for the moving averages.