https://codeburst.io/pygam-getting-started-with-generalized-additive-models-in-python-457df5b4705f



University of Michigan - Ann Arbor

Generalized Additive Models (GAMs), similar to multiple linear regression models, attempt to predict a response value (Y) with many predictor variables ($X_1,...,X_p$). GAMs maintain their additivity by applying separate non-linear functions to predictor variables ($f_j(X_j)$) and adding them together. They can be further broken down into quantitative and qualitative responses.

Generalized Additive Models

Pros:
- With GAMS, one can fit non-linear functions to each predictor to automatically model non-linear relationships that standard linear regression will miss
- The non-linear fits *can* make more accurate predictions for the response Y
- The additive nature of GAMs means we can still examine the effect of each predictor variable on Y while holding other variables fixed.
- Degrees of freedom can summarize the smoothness of the GAM

Cons:
- The main limitation is that GAMs are restricted to be additive, meaning important interactions *can be missed*. However, as with regular linear regression, we can manually add interactions by including additional predictors.













Pros and Cons of GAMS

Example with wage data: 

Since this is simply a linear regression model with several basis functions, we simply use the lm() function.

```R 
gam1 <- lm(wage ~ ns(age, 5) + education, 
           data=Wage)
```

To fit more complex splines, or other components that cannot be specified as basis functions, we need to use the gam library in R. the s() function is used to indicate a smoothing spline.

```R
library(gam)

## Loaded gam 1.09.1
gam2 <- gam(wage~s(year, 4) + s(age, 5) + education, 
            data=Wage)
```

R implementation of GAMs

For GAMs in regression problems, in order to depict a non-linear relationship between the predictors and their response, we start off with the multiple linear regression model $y_i =  \beta _0 +  \beta _1x_{i1} +  \beta _2x_{i2}+...+ \beta _px_{ip}+ \epsilon _i$.We then replace the linear component ($ \beta _jx_{ij}$) with a non-linear function $f_j(x_{ij})$ so that the original multiple linear regression model becomes 
$y_i =  \beta _0 + f_1(x_{i1}) + f_2(x_{i2})+...+f_p(x_{ip})+ \epsilon _i$. A separate non-linear function $f_j$ is applied to every predictor variable in the regression model, then added together. This is why it is considered an additive model.

GAMs in Regression Problems

Similar to GAMs in regression problems, GAMs can also be used when responses are qualitative. Assuming that Y takes the value of either 0 or 1, we can start off with the logistic regression model $log( \frac{p(X)}{1-p(X)}) = \beta _0 +  \beta _1X_{1} +  \beta _2X_{2}+...+ \beta _pX_{p}$ and replace the linear component ($\beta_jX_j$) with separate non-linear functions ($f_j(X_j)$) for every predictor variable. Our corresponding model is thus $log( \frac{p(X)}{1-p(X)}) = \beta _0 +  f_1(X_{1}) +  f_2(X_{2})+...+ f_p(X_{p})$.

Learn Before

Related