Gradient Clipping by Value
Backpropagation, Training & Optimization DS practice problem on Onlearn.
Difficulty: easy.
Topics: Gradient Clipping by Value, Gradient Clipping by Value, Exploding Gradients, Thresholding, L2 Norm Clipping, Vanishing Gradient Problem, Optimization Theory, Numerical Analysis, Deep Learning Foundations, Computational Statistics, Software Engineering for ML, Gradient-Based Optimization, Stability and Convergence, Backpropagation Algorithms, Hyperparameter Tuning, Error Handling and Robustness.
Implement a function that performs gradient clipping by value. This technique is used during neural network training to prevent exploding gradients by limiting gradient values to a specified range. Given a numpy array of gradients and a clip value, clip each gradient element to be within the range [ clip value, clip value]. Any value greater than clip value should be set to clip value, and any value less than clip value should be set to clip value. Values already within the range should remain unchanged. The function should work with gradient arrays of any shape (1D, 2D, etc.) and return the clipped gradients with the same shape as the input.