Adadelta Optimizer
Backpropagation, Training & Optimization DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Adadelta Optimizer, Root Mean Square Propagation, Exponentially Weighted Moving Average, Gradient Accumulation, Epsilon Smoothing, Parameter Update Rule, Optimization Theory, Numerical Analysis, Deep Learning Foundations, Computational Linear Algebra, Software Engineering for ML, Adaptive Learning Rate Methods, Stochastic Gradient Descent Variants, Moving Average Estimation, Parameter Update Dynamics, Tensor Operation Vectorization.
Implement the Adadelta optimizer update step function. Your function should take the current parameter value, gradient, and moving averages as inputs, and return the updated parameter value and new moving averages. The function should handle both scalar and array inputs, and include proper input validation.