Pegasos Kernel SVM Implementation
Instance-Based, Kernel & Probabilistic Methods DS practice problem on Onlearn.
Difficulty: hard.
Topics: Understanding Pegasos Kernel SVM Implementation, Subgradient Descent, Hinge Loss Function, Radial Basis Function Kernel, Dual Representation, L2 Regularization, Optimization Theory, Statistical Learning Theory, Linear Algebra, Kernel Methods, Numerical Analysis, Convex Optimization, Support Vector Machines, Gradient-Based Learning, Reproducing Kernel Hilbert Spaces, Vectorized Computation.
Write a Python function that implements a deterministic version of the Pegasos algorithm to train a kernel SVM classifier from scratch. The function should take a dataset (as a 2D NumPy array where each row represents a data sample and each column represents a feature), a label vector (1D NumPy array where each entry corresponds to the label of the sample), and training parameters such as the choice of kernel (linear or RBF), regularization parameter (lambda), and the number of iterations. Note that while the original Pegasos algorithm is stochastic (it selects a single random sample at each step), this problem requires using all samples in every iteration (i.e., no random sampling ). The function should perform binary classification and return the model's alpha coefficients and bias.