Adagrad Optimizer
Backpropagation, Training & Optimization DS practice problem on Onlearn.
Difficulty: easy.
Topics: Understanding Adagrad Optimizer, Accumulated Squared Gradients, Epsilon Smoothing, Element-wise Division, Learning Rate Decay, Broadcasting Semantics, Numerical Optimization, Deep Learning Foundations, Computational Linear Algebra, Software Engineering, Statistical Learning Theory, Adaptive Learning Rate Methods, Gradient-Based Optimization, Vectorized Array Operations, Parameter Update Dynamics, Input Validation and Error Handling.
Implement the Adagrad optimizer update step function. Your function should take the current parameter value, gradient, and accumulated squared gradients as inputs, and return the updated parameter value and new accumulated squared gradients. The function should also handle scalar and array inputs, and include proper input validation.