How might K-means be used in conjunction with supervised methods to predict on an unlabeled data set?

Dhiraj, K. (2019, August 29). Difference between K-Nearest Neighbor(K-NN) and K-Means Clustering. Medium. [medium.com/@dhiraj8899/difference-between-k-nearest-neighbor-k-nn-and-k-means-clustering-d9a44859182f](https://medium.com/@dhiraj8899/difference-between-k-nearest-neighbor-k-nn-and-k-means-clustering-d9a44859182f) 

Medium: Difference between K-Means and KNN

This reference gets into mathmatical ways how KNN and K-Nearest Neighbors are different concepts

Umamaheswaran, Venkatesh. (2018, November 12). Comprehending K-means and KNN Algorithms. Medium. Retrieved From [becominghuman.ai/comprehending-k-means-and-knn-algorithms-c791be90883d](https://becominghuman.ai/comprehending-k-means-and-knn-algorithms-c791be90883d) 

Math/Python Explanation: Difference Between K-Means and KNN

1. Randomly assign a number, from 1 to K, to each of the observations. These serve as initial cluster assignments for the observations.
2. Iterate until the cluster assignments stop changing:
- (a) For each of the K clusters, compute the cluster centroid. The kth cluster centroid is the vector of the p feature means for the observations in the kth cluster.
- (b) Assign each observation to the cluster whose centroid is closest

Algorithm of K-means Clustering

This picture depicts an example of the K-Means Clustering process. Random data points are selected and serve as the initial clusters. How many there are depends on the value of k, which in this picture’s case, k = 2. It then measures the Euclidean distance between the 1st data point and the two initial clusters, and assigns it to whichever cluster is the closest. It repeats this process, calculates the mean values, and reclusters based on that. The process repeats until the cluster no longer changes (seen in final image).


Image of K-Means Clustering Process

1. Choosing K manually
2. Clustering data of varying sizes and density
3. Clustering outliers
4. Scaling with number of dimensions
5. Quantifying how well a data cluster corresponds to reality

Limitations of K-means clustering

1. Relatively simple to implement
2. Scales to large data sets 
3. Guarantees convergence
4. Can warm-start the positions of centroids
5. Easily adapts to new examples
6. Generalizes to clusters of different shapes and sizes, such as elliptical clusters
7. The principle is relatively simple, the implementation is also very easy, and the convergence speed is fast. 
8. The clustering effect is better. 
9. The interpretability of the algorithm is strong. 
10. The main parameter to be adjusted is only the number of clusters K.

Advantages of K-means clustering

There are two main waits to pick the optimal k value for k-means clustering: 

1. Elbow method: Plot the explained variation as a function of the number of clusters and pick the elbow of the curve as the number of clusters to use.

2. Silhouette score: Calculated using the silhouette coefficient - (x-y)/max(x,y) where x is mean distance to the instances of the next closest cluster and y is the mean distances to the other instances in the same cluster. You pick the k value with the biggest silhouette score.

Picking optimal k value

One popular way to determine the optimal number of k is by using the elbow method. To use this method, you will run K means n times, each time adding another cluster. After each iteration you graph a score for that number of k. The point in the graph where there is an “elbow” is the point of optimal clusters. The picture below shows an example graph that would be used for determining the number of clusters -- in this example case 3 clusters is optimal


The Elbow Method for Selecting Optimal K

Hands-On Machine Learning with R: Chapter 20 K-means Clustering
https://bradleyboehmke.github.io/HOML/kmeans.html

Hands-On Machine Learning with R: Chapter 20 K-means Clustering

K-Means Clustering is a form of unsupervised learning that aims to identify distinct categories in data by segmenting the data by Euclidean Distance into 'k' number of clusters.

University of Michigan - Ann Arbor

- K-means clustering
- Hierarchical clustering

Method of clustering in unsupervised statistical learning

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

K-Means Clustering

Method of clustering that seeks to build a hierarchy of clusters. 

Starts by treating each observation as separate cluster. Then it will repeated execute the following two steps: 

1. Identify the two clusters closest together
2. Merge the two most similar clusters 


The iterative process continues until all clusters are merged together

Hierarchical Clustering  

Some decisions must be made before performing K-means and hierarchical clustering. With these methods there is no single right answer and any solution that exposes some interesting aspects of the data should be considered.

Practical Issues in Clustering

Below is a video reference for both k-means and hierarchical clustering. It provides an easy introduction to both clustering methods and their algorithms.

[Luis Serrano]. (2019, January 27). Clustering: K-means and Hierarchical [Video File]. Retrieved from [www.youtube.com/watch?v=QXOkPvFM6NU](https://www.youtube.com/watch?v=QXOkPvFM6NU) 

Clustering: K-means and Hierarchical (Reference)

There are various applications for both k-means and hierarchical clustering using real life data. Some of these include:
- Genetics and evolutionary biology:
Genomes holds a lot of information about a species. That information can be clustered to understand how species have evolved over time
- Recommender systems:
Users can be clustered based on the similarity of their interests
- Social networks:
Demographics, behavior, and other factors can be clustered to target certain groups of users with specific types of information (echo chamber)

Applications of Clustering

Spectral Clustering is a clustering algorithm that formulates the building of clusters as a graph partitioning problem and uses the eigenvectors of a similarity matrix to break the data into clusters.

Spectral Clustering

A Gaussian mixture model (GMM) is a unsupervised learning model which states that all generated data points are derived from a mixture of a finite Gaussian distributions that has no known parameters. The parameters for Gaussian mixture models are derived from an iterative expectation-maximization algorithm. 

Gaussian Mixture Model (GMM) Clustering 

- Density based spatial clustering of applications with noise 
- Don’t need to specify # of clusters 
- Relatively efficient 
 -Identifies likely noise points 
- Parameters:
       - Neighbor count parameter: min_samples
       - Neighborhood radius parameter: eps 
- Core point: point that lie in more dense region 
- For a given data point, if there are min_sample other data points that lie within a distance of eps, that given data point is labelled as a core sample 
- All core samples with a distance of eps units apart are put in the same cluster 
- Noise: points that don’t belong in any cluster 
- Boundary point: points within a distance of eps units from core points but not core points themselves

Learn Before

Related

Learn After