Local Response Normalization (LRN)
Initialization, Normalization & Regularization DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Local Response Normalization (LRN) in Convolutional Neural Networks, Channel-wise sliding window, Sum of squared activations, Normalization denominator calculation, Broadcasting in multidimensional arrays, Hyperparameter k (bias unit) for avoiding division by zero, Deep Learning, Neural Network Architectures, Tensor Operations, Gradient Flow, Numerical Stability, Feature Map Normalization, Lateral Inhibition, Hyperparameter Tuning, Convolutional Layer Design, Activation Function Pre-processing.
Implement the Local Response Normalization (LRN) layer from scratch using NumPy. Given a 3D input tensor of shape (channels, height, width), the LRN formula is b {x,y}^i = a {x,y}^i / (k + alpha sum {j=max(0, i n/2)}^{min(C 1, i+n/2)} (a {x,y}^j)^2)^beta. Assume the input is a 3D numpy array, n is the size of the neighborhood, and k, alpha, and beta are constant hyperparameters.