- Model A test set accuracy: 0.88
- Model B test set accuracy: 0.76
- Model C test set accuracy: 0.69

University of Michigan - Ann Arbor

Google

ROC stands for "receiver operating characteristics," a name derived from communications theory, the field in which ROC curves were first used. 

The ROC curve illustrates how false positive and true positive rates for a classifier vary as you change the threshold of the classifier. In the sample graph the threshold values are not shown, but a threshold value corresponding to a point in the upper-left of the graph would likely optimize your classifier. Such a threshold value would make your classifier predict many positive observations as positive (high true positive rate), and not predict many negative observations as positive (low false positive rate). The “steepness” of the curve maximizes true positive rate while minimizing the false positive rate 

AUC simply stands for "area under curve". 

Generally, higher the ROC AUC for a classifier, the better it is. This is because having more threshold values for which the true positive rate is higher than the false positive rate and having larger ratios of true positive rate to false positive rate (both good things) increases the ROC AUC. ROC AUC can take on values between 0 and 1. A value close to 1 means the classifier is quite good. A value of 0.5 means the classifier is no better than guessing randomly. The blue line on the sample graph has an AUC of 0.95 and the dotted line has an AUC of 0.5. 

ROC Curve and ROC AUC

Can these accuracy scores tell you which ROC curve belongs to each model?

- Model X test-set AUC: 0.88
- Model Y test-set AUC: 0.50
- Model Z test-set AUC: 0.61

Curve descriptions:
- Curve 1 hugs the upper-left corner most closely.
- Curve 2 lies almost exactly on the diagonal line.
- Curve 3 sits a little above the diagonal but well below Curve 1.

Learn Before

Related