The shortest distance from the separating hyperplane to the nearest test observation.

SVM Margin

The maximum distance from the decision boundary area to the nearest test observation.

SVM Classifier Margin

The hyperplane used to classify data depending on which side observations fall under the maximum margin hyperplane.

SVM Maximal Margin Classifier

Maximal margin hyperplanes (AKA optimal separating hyperplanes) are used in order to find which hyperplane would give the greatest separation between output classes. These hyperplanes are found by computing the classifier margin.

University of Michigan - Ann Arbor

Support Vector Machines are intended for binary classification (cases where there are two classes). The main goal is to find a hyperplane that best divides a dataset into two classes. This is quantified by achieving the maximum distance between data points of both classes separated by the hyperplane. The further away the data points are from the hyperplane, the more confidently we are able to classify future data points into their respective class.

Support Vector Machines

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Support vectors are the data points that are closer to the hyperplane. These data points are important because moving them will change which location of the hyperplane maximizes the distance between points along the decision boundary, meaning that they influence the position and orientation of the hyperplane. These support vectors are used to find the maximum distance between data points of both classes (maximum margin).

It should be noted that other points in the classifier are rendered irrelevant, as only support vectors have the power to move the hyperplane.

The green linear line is the hyperplane. The distance between the black lines is the maximum margin. The data points that are circled in black are the support vectors.

What are Support Vectors?

- Hard Margin SVM (Solving the maximum margin separator problem)
- Soft Margin SVM (Allow minimally misclassified points)

Types of SVM

This video gives a general overiew of what support vector machines are and how they work.

[StatQuest with Josh Starmer]. (2019, September 30). Support Vector Machines, Clearly Explained!!! [Video File]. Retrieved from [www.youtube.com/watch?v=efR1C6CvhmE](https://www.youtube.com/watch?v=efR1C6CvhmE) 

Support Vector Machines - StatQuest Video

Support Vector Machines are intended for binary classification, however there are extensions of it intended for use with more than two classes. There are two popular approaches for SVMs when we have an arbitrary number of K classes:
- One-versus-one
- One-versus-all

 SVMs with More than Two Classes

For dataset like p1.1, it's hard to use support vector classifier(soft margin classifier) since no matter where we put the classifier it will have a lot of misclassifications. 
In order to solve this problem, we can use support vector machine (SVM). The main idea behind SVM is we create a new y-axis and then square the value on x-axis. Now the value is two dimensional. We can then draw a support vector classifier that separates these two groups.

SVM is an extension of support vector classifier, which uses kernels to enlarge features in a certain way like what is depicted in the picture.

Learn Before

Related

Learn After