Link: [kaggle.com](https://www.kaggle.com/dansbecker/using-categorical-data-with-one-hot-encoding) 

University of Michigan - Ann Arbor

Cite helpful visuals for helping explain data science concepts

Helpful data science images

In order to avoid the problem of false assumed relationships present with LabelEncoder, you can use OneHotEncoder to transform a column into a series of columns where each column represents the binary choice of 'did the original column contain value x'. This prevents the creation of false ordinal relationship. However, OneHotEncoder only takes columns of integer values greater than 1, but you can easily transform categorical variables using LabelEncoder or pd.get_dummies

sklearn.preprocessing.OneHotEncoder

https://www.textbook.ds100.org/ch/15/bias_cv.html



Image Reference: Train Test Error

Image Reference: Linear vs Logistic Regression

Graphical representation of using the decrease in Gini Index to determine the most important variables. 

Image Reference: Variable Importance

One hot encoding picture

Link: [runestone.academy](https://runestone.academy/runestone/books/published/httlads/_images/DSPipeline1.svg) 

Image Reference: Data Science Pipeline

https://manisha-sirsat.blogspot.com/2019/04/confusion-matrix.html

Image Reference: Confusion Matrix

https://towardsdatascience.com/understanding-auc-roc-curve-68b2303cc9c5

ROC & AUC

https://www.intermediait.com/blog/2019/11/18/evaluating-model-performance-generalization-bias-variance-tradeoff-and-overfitting-vs-underfitting/

Workflow to reduce BIAS and VARIANCE

Recent Developments on Statistical and Neural Network Tools Focusing on Biodiesel Quality - Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/TOPOLOGY-OF-A-MULTILAYER-PERCEPTRON-NEURAL-NETWORK_fig1_274240858 [accessed 12 Feb, 2021]

Topology of a multilayer perceptron neural network

https://towardsdatascience.com/understanding-learning-rates-and-how-it-improves-performance-in-deep-learning-d0d4059c1c10

Learning Rate in Deep Learning

PART OF SPEECH TAGGER FOR ARABIC TEXT BASED SUPPORT VECTOR MACHINES: A REVIEW - Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/Methods-for-implementing-POS-tagger_fig1_331684946

Methods for implementing POS tagger


https://www.sentisum.com/library/nlp-and-text-mining

NLP and other relationships

https://medium.com/@tomyuz/a-sentiment-analysis-approach-to-predicting-stock-returns-d5ca8b75a42

Sentiment Analysis Process

Image Reference: Interpreting bias and variance

Scikit-learn enables allows for the transformation of categorical data into quantitative data which can be used in machine learning model training. However, an important thing to note about LabelEncoder is that it will transform your categorical data into ordinal numbers which may cause your model interpret do believe there are relationships between unrelated categories.

sklearn.preprocessing.LabelEncoder

The `pandas.get_dummies()` function is used to convert categorical variables into binary indicator columns. To apply categorical imputation, the `dummy_na=True` argument can be passed, which instructs the function to treat `NaN` as a valid category and generate a dedicated dummy column to explicitly flag missing entries.

Learn Before

Related