He Weight Initialization
Initialization, Normalization & Regularization DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding He (Kaiming) Weight Initialization in Deep Neural Networks, He (Kaiming) Normal Initialization, Fan-in vs Fan-out connections, ReLU activation variance scaling, Normal distribution sampling, Weight matrix dimensionality, Deep Learning Foundations, Neural Network Architecture, Statistical Distributions, Numerical Optimization, Gradient Flow Dynamics, Weight Initialization Strategies, Activation Function Behavior, Variance Preservation, Backpropagation Stability, Vanishing and Exploding Gradients.
Implement a function that initializes a weight matrix for a neural network layer using the He (Kaiming) initialization method for ReLU activation functions. The function should take the number of input units (fan in) and the number of output units (fan out) as arguments and return a NumPy array of shape (fan in, fan out) populated with values sampled from a normal distribution with mean 0 and variance 2/fan in.