University of Michigan - Ann Arbor

1. Confusion Matrix
2. Accuracy, Recall, Precision
3. F1 Score
4. Receiving Operating Characteristics(ROC),Area under the Curve(AUC)
5. Log Loss

Evaluation Metrics of Classification Models

After training a classificaiton model, we can measure the accuracy of a model by predicting the outcome of the observations in the test set.  If the model cannot predictive expected outcome, the accuracy is lower, and the opposite is true for the correct predictions.  

Clasification Accuracy

Formulas for evaluation


Predict_proba
-typical rule: choose most like class
-Adjusting threshold affeccts predictions of classifier
-Higher threshold results in a more conservative classifier (e.g. only predict class I if estimated probability of class I is above 70%)
-Not all models provide realistic probability estimates

Predicted Probability of class membership(predict_proba)

Having three evaluation metrics makes it harder for you to quickly choose between two different algorithms, and will slow down the speed with which your team can iterate. True/False?

Learn Before

Related