
Predict_proba
-typical rule: choose most like class
-Adjusting threshold affeccts predictions of classifier
-Higher threshold results in a more conservative classifier (e.g. only predict class I if estimated probability of class I is above 70%)
-Not all models provide realistic probability estimates

University of Michigan - Ann Arbor

1. Confusion Matrix
2. Accuracy, Recall, Precision
3. F1 Score
4. Receiving Operating Characteristics(ROC),Area under the Curve(AUC)
5. Log Loss

Evaluation Metrics of Classification Models

https://www.coursera.org/learn/python-machine-learning?utm_source=gg&utm_medium=sem&utm_content=01-CatalogDSA-ML1-US&campaignid=9918777773&adgroupid=100491712477&device=c&keyword=&matchtype=b&network=g&devicemodel=&adpostion=&creativeid=432357975999&hide_mobile_promo&gclid=CjwKCAjwnef6BRAgEiwAgv8mQb1CwbQkIUWRMGczmdtUo6LaKB5U4kg_Tet6Vk7sTpzIvIZtBwSxyxoC26sQAvD_BwE

Coursera: Applied Machine Learning in Python

After training a classificaiton model, we can measure the accuracy of a model by predicting the outcome of the observations in the test set.  If the model cannot predictive expected outcome, the accuracy is lower, and the opposite is true for the correct predictions.  

Clasification Accuracy

Formulas for evaluation

Predicted Probability of class membership(predict_proba)

Having three evaluation metrics makes it harder for you to quickly choose between two different algorithms, and will slow down the speed with which your team can iterate. True/False?

Learn Before

Related