When there are K > 2 classes, we are able to use one-vs-one classification. The process involves selecting two pairs of classes at a time, along with a test observation that is assigned for each of the paired classes. Then, during the classification phases, we keep track of each "success" (# of times the test observation is assigned to each of the K classes). After we have the tallies of successes, the final classification is performed on the class to which the test observation had the highest # of successes out of all the paired classes.

University of Michigan - Ann Arbor

Support Vector Machines are intended for binary classification, however there are extensions of it intended for use with more than two classes. There are two popular approaches for SVMs when we have an arbitrary number of K classes:
- One-versus-one
- One-versus-all

 SVMs with More than Two Classes

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

One-versus-one in SVM

When there are K > 2 classes, we are able to use a one-vs-all classification. The process involves comparing individual K classes to the remaining K - 1 classes. It trains one classifier per class in total K classifiers. We fit K SVMs for each class to differentiate that class from all of the other classes. The final step is to assign the observation to the class which resulted in the largest parameters because we are more confident that the test observation belongs to that class compared to all of the other classes.

Learn Before

Related