Spectral Normalization
Initialization, Normalization & Regularization DS practice problem on Onlearn.
Difficulty: hard.
Topics: Understanding Spectral Normalization in Generative Adversarial Networks, Power Iteration, Spectral Norm, Operator Norm, Weight Matrix Normalization, Spectral Radius, Linear Algebra, Deep Learning Optimization, Generative Modeling, Matrix Calculus, Numerical Analysis, Lipschitz Continuity, Singular Value Decomposition, Weight Constraint, Stability in GANs, Iterative Approximation.
Implement a SpectralNormalization layer class for a linear transformation. The class should maintain an internal estimate of the weight matrix's spectral norm using the power iteration method and normalize the weights during the forward pass to ensure the Lipschitz constant is 1.