https://towardsdatascience.com/metrics-to-evaluate-your-machine-learning-algorithm-f10ba6e38234

Written by: Aditya Mishra
Publised on: Feb 24, 2018

towardsdatascience.com: Metrics to Evaluate your Machine Learning Algorithm (section: Classification Accuracy)

After training a classificaiton model, we can measure the accuracy of a model by predicting the outcome of the observations in the test set.  If the model cannot predictive expected outcome, the accuracy is lower, and the opposite is true for the correct predictions.  

University of Michigan - Ann Arbor

1. Confusion Matrix
2. Accuracy, Recall, Precision
3. F1 Score
4. Receiving Operating Characteristics(ROC),Area under the Curve(AUC)
5. Log Loss

Evaluation Metrics of Classification Models

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Clasification Accuracy

Formulas for evaluation


Predict_proba
-typical rule: choose most like class
-Adjusting threshold affeccts predictions of classifier
-Higher threshold results in a more conservative classifier (e.g. only predict class I if estimated probability of class I is above 70%)
-Not all models provide realistic probability estimates

Predicted Probability of class membership(predict_proba)

Having three evaluation metrics makes it harder for you to quickly choose between two different algorithms, and will slow down the speed with which your team can iterate. True/False?

Learn Before

Related

Learn After