PReLU Forward and Backward Pass
Neural Units & Activations DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Parametric ReLU (PReLU) Dynamics and Gradient Flow, Leaky ReLU Variants, Learnable Hyperparameters, Element-wise Masking, Chain Rule Application, Gradient of ReLU, Deep Learning Foundations, Calculus and Optimization, Linear Algebra, Neural Network Architectures, Computational Graphs, Activation Functions, Gradient Descent, Backpropagation, Parameter Learning, Non-linear Transformations.
Implement a class or function pair for the PReLU activation function. PReLU is defined as f(x) = max(0, x) + alpha min(0, x). You must implement two functions: 'prelu forward(x, alpha)' which returns the output, and 'prelu backward(grad output, x, alpha)' which returns the gradients with respect to the input 'x' and the parameter 'alpha'. Assume 'alpha' is a scalar for simplicity.