Notes and links on SVMs (WIP)
What is an SVM?
Seems like a nice math trick to find the location of the median point, line, surface, or plane that separates data.
It can work with data that is not linearly separable in 2 dimensions but is in 3 (or more) dimensions.
All you need is another point of view
The algebra
SVM kernals
SVM Kernels : Data Science Concepts
How to take a “short-cut” when calculating the products of the data transformed into a higher dimension. Keeps the complexity at n² rather than n*P (where P is the number of dimensions) + n².
- linear
- polynomial
- radial basis function (RBF)
Visualisation with Scikit-learn
Knime
Foundational Support Vector Machine Publications
-
A Training Algorithm for Optimal Margin Classifiers
Bernhard E. Boser, Isabelle M. Guyon & Vladimir N. Vapnik (1992)
https://dl.acm.org/doi/10.1145/130385.130401 -
Support-Vector Networks
Corinna Cortes & Vladimir Vapnik (1995)
https://doi.org/10.1007/BF00994018 -
The Nature of Statistical Learning Theory
Vladimir N. Vapnik (1995)
https://doi.org/10.1007/978-1-4757-3264-1 -
A Tutorial on Support Vector Machines for Pattern Recognition
Christopher J. C. Burges (1998)
https://doi.org/10.1023/A:1009715923555 -
Support Vector Regression Machines
Harris Drucker, Christopher J. C. Burges, Linda Kaufman, Alex J. Smola & Vladimir Vapnik (1996)
https://papers.nips.cc/paper/1238-support-vector-regression-machines -
Making Large-Scale SVM Learning Practical
Thorsten Joachims (1999)
https://www.cs.cornell.edu/~tj/publications/joachims_99a.pdf -
New Support Vector Algorithms
Bernhard Schölkopf, Alex J. Smola, Robert C. Williamson & Peter L. Bartlett (2000)
https://doi.org/10.1162/089976600300015565
Additional Foundational and Influential SVM Papers
-
Support Vector Machines and Kernel Methods: The New Generation of Learning Machines
Nello Cristianini & Bernhard Schölkopf (2002)
A comprehensive overview of kernel methods and SVMs, highlighting their theoretical foundations and practical applications across various domains.
Link -
The Entire Regularization Path for the Support Vector Machine
Trevor Hastie, Saharon Rosset, Robert Tibshirani & Ji Zhu (2004)
Introduces an algorithm to efficiently compute the entire solution path of SVMs as the regularization parameter varies, providing insights into model selection and regularization effects.
Link -
Learning from Distributions via Support Measure Machines
Krikamol Muandet, Kenji Fukumizu, Francesco Dinuzzo & Bernhard Schölkopf (2012)
Extends SVMs to operate on probability distributions by embedding them into a reproducing kernel Hilbert space, enabling learning from complex data types.
Link -
Learning Optimally Sparse Support Vector Machines
Andrew Cotter, Shai Shalev-Shwartz & Nathan Srebro (2013)
Proposes methods to train SVMs that achieve optimal sparsity, reducing the number of support vectors without compromising performance.
Link -
Deep Learning using Linear Support Vector Machines
Yichuan Tang (2013)
Demonstrates that integrating linear SVMs into deep learning architectures can enhance classification performance compared to traditional softmax layers.
Link -
Learning by Transduction
Alex Gammerman, Volodya Vovk & Vladimir Vapnik (2013)
Introduces a transductive learning approach based on SVMs, focusing on predicting specific test instances rather than generalizing across the entire input space.
Link -
Support Vector Machines with Applications
Javier M. Moguerza & Alberto Muñoz (2006)
Provides a thorough introduction to SVMs, discussing their theoretical underpinnings and showcasing applications in various fields.
Link -
Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing
Authors: Vladimir Vapnik, Steven E. Golowich, Alex J. Smola (1996)
Overview: This paper extends SVMs to regression tasks, introducing the concept of Support Vector Regression (SVR), which has become a fundamental technique in machine learning.
Link: Proceedings of NIPS 1996 -
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods
Authors: Nello Cristianini, John Shawe-Taylor (2000)
Overview: A comprehensive textbook that provides an in-depth introduction to SVMs and kernel methods, suitable for both students and practitioners.
Link: Cambridge University Press -
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
Authors: Bernhard Schölkopf, Alexander J. Smola (2002)
Overview: This book delves into the theoretical foundations of SVMs and kernel methods, offering insights into their practical applications.
Link: MIT Press -
Support Vector Machines: Theory and Applications
Authors: Theodoros Evgeniou, Massimiliano Pontil (2001)
Overview: A collection discussing the theory behind SVMs and their diverse applications across various domains.
Link: Springer -
A Comprehensive Survey on Support Vector Machine Classification
Authors: K. M. M. Prabhu, R. K. Selvakumar (2020)
Overview: This survey provides an overview of SVM classification techniques, applications, and future research directions.
Link: ScienceDirect -
Methods for Class-Imbalanced Learning with Support Vector Machines: A Review and an Empirical Evaluation
Authors: Salim Rezvani, Farhad Pourpanah, Chee Peng Lim, Q. M. Jonathan Wu (2024)
Overview: Discusses approaches to handle class imbalance in SVMs, including resampling and algorithmic methods.
Link: Springer -
Comprehensive Review on Twin Support Vector Machines
Authors: M. Tanveer, T. Rajani, R. Rastogi, Y. H. Shao, M. A. Ganaie (2021)
Overview: Explores Twin SVMs, which solve two smaller optimization problems, offering computational advantages.
Link: arXiv -
High-Performance Support Vector Machines and Its Applications
Authors: Taiping He, Tao Wang, Ralph Abbey, Joshua Griffin (2019)
Overview: Proposes a distributed SVM algorithm suitable for large-scale data and demonstrates its applications.
Link: arXiv