- Each instance has equal weight.
- Largest classes have most influence.

1- Aggregate outcomes across all classes.

2- Compute metric with aggregate outcomes.

University of Michigan - Ann Arbor

Precision, also known as Positive-Predictive Value, is an important evaluation metric and is defined as True Positives / (True Positives + False Positives), i.e., among recognized outcomes (labels), what % are correct (actual) outcomes.

This is important especially when tasks need to be certain that a positive prediction is actually a True Positive. 

Precision is important in many tasks that directly impact consumers (as a False Positive would not be good). Some examples include search engine classification, sentiment analysis, and movie/item recommendation services.



Precision of a Classification Model

In statistical analysis of a binary classification, the F1 score is a measure of a test's accuracy, defined as the average of precision and recall. It is the harmonic mean of the precision and recall, with the highest possible value being 1.0, indicating perfect precision and recall, and the lowest possible value is 0, if either the precision or the recall is zero.

Some applications for using F1 score include measuring search, document classification, and query classification performance. The F-score has been widely used in the natural language processing literature, such as in the evaluation of named entity recognition and word segmentation.

Some common criticisms of F1 score include that it gives equal importance to recall and precision. It also does not take into account true negatives, thus making it susceptible to unbalanced class bias.

$$F_1 = 2\frac{1}{(\frac{1}{precision}+\frac{1}{recall})}$$

F1 Score

Each class has equal weight.

1- Compute metric within each class.

2- Average resulting metrics across all classes.

Maro-average Precision of a Classification Model

Micro-average Precision of a Classification Model

F-Measure is a metric that incorporates aspects of both precision and recall, defined as:
$$F_\beta = \frac{(\beta^2 + 1)PR}{\beta^2P + R}$$
The $\beta$ parameter differentially weights the importance of recall $R$ and precision $P$, based perhaps on the needs of an application. Values of $\beta > 1$ favor recall, while values of $\beta < 1$ favor precision. When $\beta = 1$, the measure is simply the F-1 score, where precision and recall are equally balanced.

Learn Before

Related