Xavier/Glorot Weight Initialization
Initialization, Normalization & Regularization DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Neural Network Weight Initialization, Fan-in and Fan-out Calculation, Uniform Distribution Sampling, Variance Scaling, Hyperparameter Heuristics, Weight Symmetry Breaking, Deep Learning Foundations, Neural Network Architecture, Numerical Optimization, Probabilistic Modeling, Matrix Calculus, Weight Initialization, Vanishing and Exploding Gradients, Activation Variance Preservation, Stochastic Processes, Layer Connectivity Analysis.
Implement a function that performs Xavier/Glorot uniform initialization for a weight matrix of shape (fan in, fan out). The weights should be sampled from a uniform distribution in the range [ limit, limit], where limit = sqrt(6 / (fan in + fan out)). Use numpy for random generation.