Start with all observations in one cluster, then splits form as one moves down the hierarchy.

Divisive: Top-down approach

One cluster for each observation, then clusters are moved together as one moves up the hierarchy of clusters

Steps: 
1. Each data point is initially in its own cluster with one item 
2. Most similar cluster of each stage is merged 
3. Stopping measure: number of clusters 

Linkage criteria
- Ward's method
- least increase in total variance (around cluster centroids) 
2. Average linkage 
- average distance between clusters
3. Complete linkage 
- max distance between clusters




Agglomerative: Bottom-up approach

1. Divisive: Top-down approach
2. Agglomerative: Bottom down approach


University of Michigan - Ann Arbor

Method of clustering that seeks to build a hierarchy of clusters. 

Starts by treating each observation as separate cluster. Then it will repeated execute the following two steps: 

1. Identify the two clusters closest together
2. Merge the two most similar clusters 


The iterative process continues until all clusters are merged together

Hierarchical Clustering  

Types of hierarchical clustering

Given the explorative nature of Unsupervised Learning, there may be scenarios where one not know how many clusters that may appear in a set of data. Hierarchical clustering, unlike K-means clustering, does not require a pre-determined number of clusters. 

Why Hierarchical Clustering?

Using Hierarchical clustering can lead to a great deal of bias as not all data is hierarchical in nature. Assuming that data lower in a dendogram can also be classified by properties higher on the dendogram can lead to a much worse result than using K-means clustering instead.

Assumption of Hierarchical Structure

Linkage between clusters can be determined in four main ways, three of which are pictured below:
- Simple: Find the pairwise dissimilarity between each pair of observations and select the most similar
- Average Linkage: Find all pairwise dissimilarities and use the mean value
- Complete Linkage: Compute pairwise dissimilarities and use the least similar 
- Centroid Linkage: Not pictured. Dissimilarity between the centroids of two clusters.

Measuring Similarity between Clusters

Hierarchical Clustering analysis is presented in the form of a tree based representation of the observations, which is known as a dendrogram. 

A graphical representation of a hierarchical cluster transformed to a dendrogram is pictured below.

Learn Before

Related

Learn After