Implementing a Simple RNN
Sequence Models & Generative Models DS practice problem on Onlearn.
Difficulty: medium.
Topics: Implementing a Simple RNN, Hidden State Update, Vanishing Gradient Problem, Tanh Non-linearity, Weight Matrix Sharing, Temporal Dependency Modeling, Linear Algebra, Calculus, Optimization Theory, Sequence Modeling, Probability Theory, Matrix Transformations, Backpropagation Through Time, Weight Initialization Strategies, Activation Function Dynamics, Recurrent Neural Architectures.
Write a Python function that implements a simple Recurrent Neural Network (RNN) cell. The function should process a sequence of input vectors and produce the final hidden state. Use the tanh activation function for the hidden state updates. The function should take as inputs the sequence of input vectors, the initial hidden state, the weight matrices for input to hidden and hidden to hidden connections, and the bias vector. The function should return the final hidden state after processing the entire sequence, rounded to four decimal places.