University of Michigan - Ann Arbor

K-nearest neighbors can be used to classify data. Given a training set X_train with labels y_train, and given a new instance x_test to be classified:

- Find the k most similar instances (X_NN) to x_test that are in X_train.
- Get the labels y_NN for the instances in X_NN.
- Predict the label for x_test by combining the labels y_NN. The combination can be done through different methods, e.g., a simple majority vote can find the label that is the most common among the k most similar examples.

For example, if K = 1, then the first closest neighbor will be used to classify the unknown data point. If K = 14, then the 14 nearest neighbors will be used. If the data point falls between multiple clusters, then you choose the category that holds the most.

Classification Algorithm of K-Nearest Neighbors

Subramanian, D. (2020, January 3). A Simple Introduction to K-Nearest Neighbors Algorithm. Retrieved from https://towardsdatascience.com/a-simple-introduction-to-k-nearest-neighbors-algorithm-b3519ed98e

Reference for KNN

1. A distance metric
2. K = the of the nearest neighbors to use in making a prediction
3. Optional weighting function on the neighbor points
4. Method for aggregating the classes of neighbor points

Parameters to specify when training a k-Nearest Neighbors algorithm

Which of the following is true about the k-nearest neighbors classification algorithm, assuming uniform weighting on the k neighbors? Select all that apply.

Used the data for classes 0, 1, and 2 plotted in the image.

Learn Before

Related