http://notesbyanerd.com/2014/12/17/multi-class-performance-measures/

University of Michigan - Ann Arbor

- Macro-average: each class has equal weight. 
1. Compute metric with each class 
2. Average resulting metrics across classes 
- Micro-average: Each instance has equal weight, largest class has most influence
1. Aggregate outcomes across classes
2. Compute metric with aggregate outcomes 

If want to weight metric towards largest classes → use micro-averaging
If you want to weight metric towards smallest classes → use macro-averaging 


Multi-Class Evaluation Metrics

https://datascience.stackexchange.com/questions/15989/micro-average-vs-macro-average-performance-in-a-multiclass-classification-settin

Micro Average vs Macro average Performance in a Multiclass classification setting

Multi-class Performance Measures

Macro-average is the average of some metric (e.g. precision or recall) among classes, where all classes are weighted equally.  Since this effectively gives a "boost" to smaller classes, macro-averages are more indicative of the precision/recall within smaller classes than within larger classes. According to the referenced article, however, macro-averaging can be bad practice when classes vary widely in their number of instances.

Macro-average

Micro-average is the average of some metric (e.g. precision or recall) among classes, where classes are weighted proportionally by their number of instances. Another way of saying this is that all instances are weighted equally. Since larger classes have more instances and all instances have equal weight, micro-averages are more indicative of the precision/recall within larger classes. In other words, larger classes have more influence.

Micro-average

Having three evaluation metrics makes it harder for you to quickly choose between two different algorithms, and will slow down the speed with which your team can iterate. True/False?

Combine the two or two categories of data in K categories, and then use the combined data to train a model to generate 𝐾(𝐾−1)/2 classifier models, merge the results of these classifiers, and classify The prediction result of the device uses majority voting to output the final prediction result value

OVO (One-Versus-One)

In one-to-many model training, one category is used as a positive example, and the remaining examples are used as negative examples to train K models; when making predictions, if one of the K models is output as a positive example, then the final The prediction result belongs to this category of the classifier; if multiple positive examples are generated, the confidence of the classifier can be selected as the indicator, and the classifier with the highest confidence is selected as the final prediction result. Common confidences are accuracy score and recall rate.

OVA/OVR (One-Versus-All/One-Versus-the-Rest)

Divide the model construction application into two stages: encoding and decoding stages;
Coding stage: perform M division among K categories. Each division divides part of the data into positive categories and part of the data into negative categories. Each division builds a model. The result of the model is for each Each category defines a point;
Decoding stage: Use the trained model to predict the test sample, define the sample to be predicted as a point in the space, calculate the distance between the point to be tested and the category, and select the category with the closest distance as the final forecast category.

Learn Before

Related