The core idea of Binary Relevance is to decompose the multi-label classification problem and transform it into q binary classification problems, where each binary classifier corresponds to a label to be predicted.

Advantage:
The realization method is simple and easy to understand;
When there is no mutual dependence between the y values, the model works well

Shortcoming:
If there is a mutual dependence between y, then the generalization ability of the final model is relatively weak;
It is necessary to construct q two classifiers, where q is the number of y values to be predicted. When q is large, there are correspondingly more models to be constructed.

Binary Relevance

The core idea of Classifier Chains is to decompose the multi-label classification problem and convert it into the form of a binary classifier chain. The construction of the binary classifier after the chain is based on the prediction results of the previous classifier; When constructing the model, the label order is first shuffled and sorted, and then the model corresponding to each label is constructed from beginning to end.

Advantage:
Considering the dependency between tags, the pan-China capability of the final model is better than the model built by Binary Relevance.

Shortcoming:
It is difficult to find a more appropriate dependency between tags.

Classifier Chains

The core idea of Calibrated Label Ranking is to decompose the multi-label classification problem and convert it into a label sorting problem. The final label is the largest label value after sorting.

Advantage:
Considering the combination of tags, the final model has a relatively good generalization ability.

Shortcoming:
Only the combination of two tags is considered, and all the dependencies between tags are not considered.

Calibrated Label Ranking 

•	Binary Relevance (first-order)
•	Classifier Chains (high-order)
•	Calibrated Label Ranking (second-order)

University of Michigan - Ann Arbor

Multi-label classification is a machine learning problem focused on predicting classes that are not mutually exclusive, allowing a single example to be assigned to multiple categories simultaneously. This formulation is particularly useful for auto-tagging tasks, where an item (such as a blog post or an image) might be associated with several correlated tags at once.

Multi-Label Classification

A simplified example of Multi-Label Classification is photo classification. This is where a given photo may have multiple objects in the scene and a model may predict the presence of multiple known objects in the photo, such as “bicycle,” “apple,” “person,” etc.

Example of Multi-Label Classification

Bernoulli distribution is a distribution over a single binary variable. It can only have results in 0 or 1. The sum of the probability of 0  and the probability of 1 is equal to 1. If we denote the probability of 0 as $\sigma$, Bernoulli distribution then has a mean of $\sigma$ and a variance of  $\sigma * (1- \sigma)$.

Bernoulli Distribution

Problem Transformation Methods of Multi-Label Classification

Algorithm adaptation is a way to directly apply the existing single label algorithm to multiple labels, mainly including:

Multi-Label KNN：

For each instance, firstly, obtain the K instances closest to it, and then use the label set of these instances to judge the value of the predicted label set of this instance through the maximum a posteriori probability (map).

Multi-Label Decision Tree：

The core of using the decision tree to deal with multi-label content is to give more fine-grained information entropy gain criteria to build this decision tree model.

Learn Before

Related

Learn After