https://www.mygreatlearning.com/blog/5-common-mistakes-while-working-with-machine-learning-algorithms/

Common errors in machine learning source

Data leakage is when information that should not be in the training set is included. This data includes information that a model would not have access to in the future and this leads to a model that is overly accurate for the training data, but will not be nearly as effective once it is tested. Data leakage will become apparent if the training model is much better than the test results and to prevent data leakage, be aware of the data that is included in the training data.

Data leakage

Linear data is data that follows a successive pattern whereas non-linear data follows a hierarchical pattern. This is an important concept as linear data and non-linear data structures are compatible with different types of machine learning algorithms, and if the wrong algorithms are chosen, the learned model will not effectively predict the data. 

Linear versus non-linear data

Data cleaning involves removing errors from data sets and this is extremely important as models follow the “garbage in garbage out” principle in which erroneous data leads to erroneous conclusions. In order to identify errors in the datasets, a variety of rules, patterns, statistical techniques, and more can be used. Common errors include incomplete rows, duplicate rows, incorrectly labeled data, and more.

What is data cleaning and why is it important?

Error analysis helps you understand which part to work on to decrease those errors and what to prioritize


Error Analysis 

•	Data leakage

•	Modeling non-linear data with linear models 

•	Forgetting to regularize data if needed

•	Substandard data cleaning

•	Failing to understand the underlying logic behind certain packages

University of Michigan - Ann Arbor

A Basic Supervised Statistical Learning Workflow

Which one shows a basic machine learning workflow?

In this step, we choose:
- A feature representation of the dataset, e.g., image pixels
- Type of classifier to use, e.g., K-nearest neighbor classifier

Classification Problem Representation

In this step, we decide on the criterion to distinguish good vs. bad classifiers. For example, the percentage of correct predictions on test set.
The learned models will be evaluated on evaluation functions (also called objective functions or scoring functions) in order to distinguish between successful and unsuccessful models. The desired, external evaluation function might be different from the one the algorithm uses.

Classification Problem Evaluation

In this step, we search for the most optimal values for (hyper)parameters that give the best classification model for the chosen evaluation criterion. For example, we can try a range of values for the "k" parameter in k-nearest neighbor classifier.

Classification Problem Optimization

For a supervised learning algorithm to do well, you need to check the following four assumptions:
- **Low Bias**: Fits training set well on cost function (e.g., comparable to human level performance)
- **Low Variance**: Fits dev set on cost function
- Fits test set well on cost function
- Performs well in real world (e.g., happy users)

Chain of Assumptions in Supervised Statistical Learning

Domingos, P. (2012). A few useful things to know about machine learning. Communications of the ACM, 55(10), 78-87. doi:10.1145/2347736.2347755

Learn Before

Related

Learn After