D'Souza, Jocelyn. A Quick Guide to Boosting in ML. (2018, March). Retrieved from [medium.com/greyatom/a-quick-guide-to-boosting-in-ml-acf7c1585cb5](https://medium.com/greyatom/a-quick-guide-to-boosting-in-ml-acf7c1585cb5) 

University of Michigan - Ann Arbor

Boosting algorithms are methods to help transform weak learners to stronger learners. There are 3 common boosting algorithms used in data science:

1. AdaBoosting (Adaptive Boosting)
2. GradientBoosting
3. XGBoost (Extreme Gradient Boosting)

Common Boosting algorithm

Starts with any simple model, then creates a new model based on the errors of the previous model, repeats till convergence. Iteratively makes a better model, by accounting for the errors of the predecessor model. At each stage you have created several different model. One thing to note here is that you simply dont take the model from the last stage but weighted sum of all models created before.

Boosting creates several trees (models) iteratively

A Quick Guide to Boosting in ML.

When nothing works, Boosting does. Nowadays many people use either XGBoost or LightGBM or CatBoost to win competitions at Kaggle or Hackathons. AdaBoost is the first stepping stone in the world of Boosting.
AdaBoost is one of the first boosting algorithms to be adapted in solving practices. Adaboost helps you combine multiple “weak classifiers” into a single “strong classifier”.

Learn Before

Related