A Simple Residual Block with Shortcut Connection
Sequence Models & Generative Models DS practice problem on Onlearn.
Difficulty: easy.
Topics: A Simple Residual Block with Shortcut Connection, Identity Shortcut Connection, Element-wise Addition, Vanishing Gradient Mitigation, ReLU Activation Function, Weight Layer Normalization, Deep Learning Architectures, Optimization Theory, Numerical Linear Algebra, Signal Processing, Computational Graph Theory, Convolutional Neural Networks, Gradient-Based Optimization, Matrix Operations, Feature Map Transformation, Backpropagation Algorithms.
Implement a function that creates a simple residual block using NumPy. The block should take a 1D input array, process it through two weight layers (using matrix multiplication), apply ReLU activations, and add the original input via a shortcut connection before a final ReLU activation.