Given the multiple linear regression model:
$y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \beta_3 x_{i3} + \cdots +  \beta_p x_{ip} + \epsilon_i$
A natural way to extend the multiple linear regression model is to replace each linear component $\beta_j x_{ij}$ with a (smooth) nonlinear function $f_j(x_{ij})$. We would then write the model as:
$$y_i = \beta_0 + \sum_{j=1}^p f_j(x_{ij}) + \epsilon_i$$ 

GAM for Regression Problems

GAMs can also be used in situations where Y is qualitative. 
For simplicity, here we will assume Y takes on values zero or one, and let $p(X) = Pr(Y = 1 \mid X)$ be the conditional probability (given the predictors) that the response equals one. Recall the logistic regression model:
$log(\frac{p(X)}{1−p(X)}) = \beta_0 + \beta_1X_1 + \beta_2X_2 +···+ \beta_pX_p$
Note that this logit is the log of the odds of $P(Y =1 \mid X)$ versus $P(Y =0 \mid X)$. With GAMs, we have a new model:
$log(\frac{p(X)}{1−p(X)}) = \beta_0 + \beta_1 f(X_1) + \beta_2 f(X_2) +···+ \beta_p f(X_p)$.

GAM for Classification Problem

Generalized additive models (GAMs) provide a general framework for
generalized additive model extending a standard linear model by allowing non-linear functions of each of the variables, while maintaining additivity. 

University of Michigan - Ann Arbor

- GAM
- Boosting
- Support vector machines

Modern SL Methods (examples)

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Generalized Additive Models

Boosting algorithms are methods to help transform weak learners to stronger learners. There are 3 common boosting algorithms used in data science:

1. AdaBoosting (Adaptive Boosting)
2. GradientBoosting
3. XGBoost (Extreme Gradient Boosting)

Common Boosting algorithm

- Generalized Additive Models
- Trees
- Random Forests
- Boosting
- Support Vector Machines
- Naive Bayes Classifier

Learn Before

Related

Learn After