https://medium.com/@kunal_gohrani/different-types-of-distance-metrics-used-in-machine-learning-e9928c5e26c7#:~:text=Manhattan%20distance%20is%20usually%20preferred,similarity%20between%20two%20data%20points.

Different Types of Distance Metrics used in Machine Learning

Manhattan distance, aka city block distance, is the number of units one must traverse to get from point X to point Y along a grid-like path. In other words, each step involves a change in value across exactly one dimension. In machine learning, Manhattan distance is preferred (over higher values of p) for high-dimensionality data because of simpler computation and because of something called the "curse of high dimensionality."

More formally, let $x_i$ be the value of point X across the $i$th dimension, and let $y_i$ be the value of point Y across the $i$th dimension (NOT to be confused with the conventional (x,y) notation for a point in 2D). Then, in $n$ dimensions, the Manhattan distance between points X and Y is
$d(X,Y) = \sum_{i=1}^n|x_i - y_i|$.

(Note that this is an instantiation of the Minkowski distance formula with p=1.)

Manhattan Distance (p=1)

https://bib.dbvis.de/uploadedFiles/155.pdf

On the Surprising Behavior of Distance Metrics
in High Dimensional Space

The curse of high dimensionality is the empirical phenomenon that in high dimensional space, concepts of proximity, distance, and/or nearest neighbor lose qualitative meaning. This problem is more severe for Euclidean distance than for Manhattan distance, hence why Manhattan distance is preferred for high dimensional data.

Curse of High Dimensionality

In order to find the Euclidean distance, one has to take the square root of the sum of the squared difference between two points. The formula for calculating the Euclidean distance is given by:

	 $\sqrt{ \sum_{i=1}^k {(x_i - y_i)}^2 }$


Euclidean Distance

The power parameter p controls which distance metric is used in determining neighbors of a point. The default is p=2, which specifies Euclidean distance. Setting p=1 specifies Manhattan distance (aka city block distance), which calculates distance along a grid-like path. Manhattan distance is preferred for high-dimensional data.

University of Michigan - Ann Arbor

Useful Parameters for KNN Classifier

https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html

Documentation for sklearn.neighbors.KNeighborsClassifier

"p" Parameter for KNN Classifier

The weights parameter can be set to 'uniform' or to 'distance'. Setting weights='uniform' means that the k nearest neighbors of a point have equal influence on how that point gets classified. On the other hand, weights='distance' causes closer neighbors to have greater influence.

"weights" Parameter for KNN Classifier

The n_neighbors parameter specifies the number of neighbors to consider when classifying points. This number is sometimes referred to as "K". A high n_neighbors risks overfitting the test data and being poor at predicting new data (high bias). On the other hand, a low n_neighbors risks underfitting the data, hence oversimplifying the model and resulting in a "jumpy" plot of predictions (high variance). The best value for K will depend on the dataset, so one value for K may work well for some datasets, but not for others.

Learn Before

Related

Learn After