https://data.princeton.edu/eco572/smoothing2

University of Michigan - Ann Arbor

Regression splines divide the range of independent variables ($X$) into $K$ distinct chunks and fit a polynomial regression to each chunk to eventually form one model. The estimation of the polynomials is constrained in the predetermined ranges, so they connect smoothly together. Below the data is fit in chunks, with the separately fit chunks meeting at 25, 50, and 75.
![Image from the cited reference](https://firebasestorage.googleapis.com/v0/b/onecademy-1.appspot.com/o/UploadedImages%2FCharlie-Logan_Fri%2C%2021%20Feb%202020%2020%3A10%3A23%20GMT.png?alt=media&token=856ee94d-fbe5-4199-bbb0-5810ade2c241)

Regression Splines

Example of Regression Splines

Regression splines combine the benefits of polynomials and step functions to provide us with a more flexible estimated function. Assuming that we divide the range of independent variables $X$ into enough regions, we can theoretically fit any function. However, this flexibility comes with the expense of overfitting and less interpretability of the coefficients. So, the analyzer should consider a trade-off when deciding on the number of regions.

Regression splines vs. polynomials and step functions

Regression splines often give superior results to polynomial regression. This is because unlike polynomials, which must use a high degree (exponent in the highest monomial term, e.g. $X^{15}$) to produce ﬂexible ﬁts, splines introduce ﬂexibility by increasing the number of knots but keeping the degree ﬁxed. 
From the figure below, we can see that polynomials can show unexpected and wild behavior, especailly near the tails. This is not preferred in the larger range prediction.



Regression splines vs. polynomial regression

While it may seem intuitive to place more knots where the data has more variance, in practice it is best to uniformly distribute the knots in the data as to not overfit in certain areas. Commonly the number and placement of knots is determined by selecting your desired degrees of freedom and letting data software distribute them automatically. 

Learn Before

Related