Theory

Smoothing Spline Formulation

If we simply fit a smooth curve g(x)g(x) to a set of data by minimizing RSS=i=1n(yig(xi))2RSS = \sum^n_{i=1}(y_i - g(x_i))^2 without putting any constraints, there is a problem that we can always make RSSRSS zero simply by choosing gg such that it interpolates all of the yiy_i, leading to the overfitting of the data. What we want is a function gg that makes RSSRSS small, but that is also smooth. Instead, we can change the optimization formulation from RSSRSS to the function below: i=1n(yig(xi))2+λg(t)2dt\sum^n_{i=1}(y_i - g(x_i))^2 + \lambda \int g''(t)^2 dt, where λ\lambda is a nonnegative tuning parameter. The function gg that minimizes the formulation above is known as a smoothing spline.

0

6

Updated 2026-06-16

Tags

Data Science

Learn After