https://data.princeton.edu/eco572/smoothing2

University of Michigan - Ann Arbor

Regression splines divide the range of independent variables ($$X$$) into $$K$$ distinct chunks and fit a polynomial regression to each chunk to eventually form one model. The estimation of the polynomials is constrained in the predetermined ranges, so they connect smoothly together. Below the data is fit in chunks, with the separately fit chunks meeting at 25, 50, and 75. ![Image from the cited reference](https://firebasestorage.googleapis.com/v0/b/onecademy-1.appspot.com/o/UploadedImages%2FCharlie-Logan_Fri%2C%2021%20Feb%202020%2020%3A10%3A23%20GMT.png?alt=media&token=856ee94d-fbe5-4199-bbb0-5810ade2c241)

Regression Splines

Example of Regression Splines

While it may seem intuitive to place more knots where the data has more variance, in practice it is best to uniformly distribute the knots in the data as to not overfit in certain areas. Commonly the number and placement of knots is determined by selecting your desired degrees of freedom and letting data software distribute them automatically. 

Regression Splines: Deciding how many knots to use and where to place them

Regression splines combine the benefits of polynomial curve fitting and step functions to provide a more flexible estimated function. By dividing the range of the independent variable $$X$$ into enough regions, we can theoretically fit any function. However, this flexibility comes at the expense of potential overfitting and reduced interpretability of the coefficients. Therefore, a trade-off must be considered when deciding on the number of regions.

Regression Splines vs. Polynomial Curve Fitting and Step Functions

Regression splines often give superior results compared to polynomial regression. Unlike polynomials, which must use a high degree (the exponent in the highest monomial term, e.g., $$X^{15}$$) to produce flexible fits, splines introduce flexibility by increasing the number of knots but keeping the degree fixed. As shown in the accompanying figure, polynomials can exhibit unexpected and wild behavior, especially near the tails. This is not preferred for larger range predictions.

Learn Before

Related