https://towardsdatascience.com/top-5-metrics-for-evaluating-classification-model-83ede24c7584

University of Michigan - Ann Arbor

Classification problems involve predicting a categorical label for a given input. The model's output for a classification task is typically a probability distribution over the set of possible labels, indicating the likelihood of the input belonging to each category.

Classification problem in statistical learning

There is a paper on predicting a user's personality from 16 personality types (based on the Myers-Briggs Type Index). The input values were the user's written text they provide on an Internet forum. This is a classification because the output is a qualitative prediction. For example, the user could be classified as an Extrovert.


Example of a classification problem in statistical learning

Classifiers are techniques used to predict qualitative variables.
- Methods that are not very computer intensive (widely-used)
- Methods that are much more computer intensive
- Decision Tree that can be used on regression and classification problems

Methods of Classification

1. Confusion Matrix
2. Accuracy, Recall, Precision
3. F1 Score
4. Receiving Operating Characteristics(ROC),Area under the Curve(AUC)
5. Log Loss

Evaluation Metrics of Classification Models

Classification Evaluation Metrics

What is the differnce between classification and regression?

In machine learning, classification refers to a predictive modeling problem where a class label is predicted for a given example of input data. There are three main types of classification tasks that you may encounter; they are:
- Binary Classification,
- Multi-Class Classification,
- Multi-Label Classification.


3 Types of Classification Tasks in Machine Learning

Imbalanced Classification refers to classification tasks where the number of examples in each class is unequally distributed. Typically, imbalanced classification tasks are binary classification tasks where the majority of examples in the training dataset belong to the normal class and a minority of examples belong to the abnormal class.

Balanced Classification is the opposite. It is primarily equally distributed and commonly occurs in multi-label or multi-class classification. 

Imbalanced Classification vs. Balanced Classification

- Confusion Matrix
- False-Positive rate (Type-I error)
- False-Negative rate (Type-II error)
- Accuracy
- Recall
- F1 Score
- Optimizing vs. Satisficing
- Specificity (True Negative rate Specificity)
- Negative Predictive Value
- False Discovery Rate
- True-Positive rate (Recall, Sensitivity)
- Positive-Predictive Value (Precision)
- ROC curve and ROC AUC

Common Performance Metrics for Classification

Classification is essentially a function mapping input data to output categories. If there are missing data in the input, the function can perform poorly in the prediction task. Therefore, having multiple functions that do the predicting job collectively is crucial.

Classification with Missing Data

A data science team is building a predictive model for an e-commerce company. Which of the following tasks represents a classification problem?

Given the following scenario, explain why this task is a classification problem. In your explanation, describe the nature of the model's predicted output.

Learn Before

Related