Tanh Activation Function
Neural Units & Activations DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding the Tanh Activation Function and its Gradient, Zero-centered property, Vanishing gradient problem, Hyperbolic tangent identity, Saturation regions, Range normalization (-1 to 1), Deep Learning Foundations, Calculus, Numerical Analysis, Neural Network Architecture, Optimization Theory, Activation Functions, Non-linear Mapping, Gradient Flow, Weight Initialization, Backpropagation.
Implement a function that computes the Tanh activation for a given input value. Additionally, explain why the Tanh function is preferred over Sigmoid in hidden layers due to its zero centered nature.