Similar to GAMs in regression problems, GAMs can also be used when responses are qualitative. Assuming that Y takes the value of either 0 or 1, we can start off with the logistic regression model $log( \frac{p(X)}{1-p(X)}) = \beta _0 +  \beta _1X_{1} +  \beta _2X_{2}+...+ \beta _pX_{p}$ and replace the linear component ($\beta_jX_j$) with separate non-linear functions ($f_j(X_j)$) for every predictor variable. Our corresponding model is thus $log( \frac{p(X)}{1-p(X)}) = \beta _0 +  f_1(X_{1}) +  f_2(X_{2})+...+ f_p(X_{p})$.

University of Michigan - Ann Arbor

Generalized Additive Models (GAMs), similar to multiple linear regression models, attempt to predict a response value (Y) with many predictor variables ($X_1,...,X_p$). GAMs maintain their additivity by applying separate non-linear functions to predictor variables ($f_j(X_j)$) and adding them together. They can be further broken down into quantitative and qualitative responses.

Generalized Additive Models

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Pros:
- With GAMS, one can fit non-linear functions to each predictor to automatically model non-linear relationships that standard linear regression will miss
- The non-linear fits *can* make more accurate predictions for the response Y
- The additive nature of GAMs means we can still examine the effect of each predictor variable on Y while holding other variables fixed.
- Degrees of freedom can summarize the smoothness of the GAM

Cons:
- The main limitation is that GAMs are restricted to be additive, meaning important interactions *can be missed*. However, as with regular linear regression, we can manually add interactions by including additional predictors.













Pros and Cons of GAMS

Example with wage data: 

Since this is simply a linear regression model with several basis functions, we simply use the lm() function.

```R 
gam1 <- lm(wage ~ ns(age, 5) + education, 
           data=Wage)
```

To fit more complex splines, or other components that cannot be specified as basis functions, we need to use the gam library in R. the s() function is used to indicate a smoothing spline.

```R
library(gam)

## Loaded gam 1.09.1
gam2 <- gam(wage~s(year, 4) + s(age, 5) + education, 
            data=Wage)
```

R implementation of GAMs

For GAMs in regression problems, in order to depict a non-linear relationship between the predictors and their response, we start off with the multiple linear regression model $y_i =  \beta _0 +  \beta _1x_{i1} +  \beta _2x_{i2}+...+ \beta _px_{ip}+ \epsilon _i$.We then replace the linear component ($ \beta _jx_{ij}$) with a non-linear function $f_j(x_{ij})$ so that the original multiple linear regression model becomes 
$y_i =  \beta _0 + f_1(x_{i1}) + f_2(x_{i2})+...+f_p(x_{ip})+ \epsilon _i$. A separate non-linear function $f_j$ is applied to every predictor variable in the regression model, then added together. This is why it is considered an additive model.

GAMs in Regression Problems

GAMs for Classification Problems

https://codeburst.io/pygam-getting-started-with-generalized-additive-models-in-python-457df5b4705f



Learn Before

Related