Derivatives of Activation Functions
Calculus & Optimization DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Derivatives of Common Activation Functions, Sigmoid derivative property, ReLU non-differentiability at zero, Leaky ReLU gradient flow, Tanh gradient saturation, Swish function derivative using product rule, Calculus, Machine Learning, Numerical Analysis, Optimization, Neural Networks, Chain Rule, Gradient Descent, Backpropagation, Activation Functions, Non-linear Mapping.
Implement a class or set of functions that compute the derivatives of the following activation functions: Sigmoid, Tanh, ReLU, Leaky ReLU (alpha=0.01), and Swish (beta=1.0). Your implementation should accept a list or NumPy array of inputs and return the derivative values calculated at those specific input points.