- Green Line: Manhattan Distance
- Red Line: Euclidean Distance

Visual Representation of the Difference Between Manhattan and Euclidean Distances

Euclidean distance is the straight-line distance between two points in a Euclidean space. For two points $$x$$ and $$y$$ in a $$k$$-dimensional space, the distance is calculated as the square root of the sum of the squared differences between their coordinates:

$$d(x, y) = \\sqrt{ \\sum_{i=1}^k (x_i - y_i)^2 }$$

University of Michigan - Ann Arbor

Google

Often times, statistical methods rely on measuring the distance between quantitative data points. However, there are various ways to measure this distance. These include:

1. Euclidean distance 
2. Manhattan distance
3. Hamming Distance

Measuring Distance Between Quantitative Data Points

The power parameter p controls which distance metric is used in determining neighbors of a point. The default is p=2, which specifies Euclidean distance. Setting p=1 specifies Manhattan distance (aka city block distance), which calculates distance along a grid-like path. Manhattan distance is preferred for high-dimensional data.

"p" Parameter for KNN Classifier

The reference below has a good reference to KNN, as well as different ways in which distance is measured.

Brownlee, J. (2019, August 12). K-Nearest Neighbors for Machine Learning. Retrieved March 23, 2020, from [machinelearningmastery.com/k-nearest-neighbors-for-machine-learning/](https://machinelearningmastery.com/k-nearest-neighbors-for-machine-learning/) 

K-Nearest Neighbors for Machine Learning (Reference)

Euclidean Distance

The Manhattan distance between two points is the sum of the absolute differences of their corresponding Cartesian coordinates. For two points $$x$$ and $$y$$ in a $$k$$-dimensional space, the distance is calculated as: $$d(x, y) = \sum_{i=1}^k |x_i - y_i|$$

Manhattan Distance

The Hamming distance is a metric useful for comparing categorical variables. It measures distance by counting the number of positions at which corresponding elements differ. For each position, if the two elements are the same, the difference is 0; if they are different, the difference is 1. Its formula is given by: $$D_H = \sum_{i=1}^k |x_i - y_i|$$ where $$|x_i - y_i| = 0$$ if $$x_i = y_i$$, and $$|x_i - y_i| = 1$$ if $$x_i \neq y_i$$.

Hamming Distance

We need to specify a distance metric in our future space, so we know how to properly select the nearby neighbors. In the fruit classification example, we use the simple straight line, or Euclidean distance to measure the distance between points.

Specifying a distance metric for k-Nearest Neighbors algorithm

https://medium.com/@kunal_gohrani/different-types-of-distance-metrics-used-in-machine-learning-e9928c5e26c7#:~:text=Manhattan%20distance%20is%20usually%20preferred,similarity%20between%20two%20data%20points.

Different Types of Distance Metrics used in Machine Learning

Manhattan distance, aka city block distance, is the number of units one must traverse to get from point X to point Y along a grid-like path. In other words, each step involves a change in value across exactly one dimension. In machine learning, Manhattan distance is preferred (over higher values of p) for high-dimensionality data because of simpler computation and because of something called the "curse of high dimensionality."

More formally, let $x_i$ be the value of point X across the $i$th dimension, and let $y_i$ be the value of point Y across the $i$th dimension (NOT to be confused with the conventional (x,y) notation for a point in 2D). Then, in $n$ dimensions, the Manhattan distance between points X and Y is
$d(X,Y) = \sum_{i=1}^n|x_i - y_i|$.

(Note that this is an instantiation of the Minkowski distance formula with p=1.)

Manhattan Distance (p=1)

https://bib.dbvis.de/uploadedFiles/155.pdf

On the Surprising Behavior of Distance Metrics
in High Dimensional Space

The curse of high dimensionality is the empirical phenomenon that in high dimensional space, concepts of proximity, distance, and/or nearest neighbor lose qualitative meaning. This problem is more severe for Euclidean distance than for Manhattan distance, hence why Manhattan distance is preferred for high dimensional data.

Learn Before

Related

Learn After