The tuning parameter ($\lambda$) is able to control how rough a smoothing spline can be. This also means that it controls its effective degrees of freedom. When $\lambda$ = 0, the entire term will have no effect and thus the line will be rough and jumpy. As $\lambda$ $\rightarrow$ $\infty$ it will be completely smooth, becoming a straight line. When $\lambda$ is an intermediate value, the line will be somewhat smooth, but still provide an approximation of the training data.

University of Michigan - Ann Arbor

If we simply ﬁt a smooth curve $g(x)$ to a set of data by minimizing RSS = $\sum^n_{i=1}(y_i - g(x_i))^2$ without putting any constraints, there is a problem that we can always make RSS zero simply by choosing $g$ such that it interpolates all of the $y_i$, leading to the overfitting of the data. What we want is a function $g$ that makes RSS small, but that is also smooth.
Instead, we can change the optimization formulation from RSS to the function below:
$$\sum^n_{i=1}(y_i - g(x_i))^2 + \lambda \int g''(t)^2 dt$$,
where λ is a nonnegative tuning parameter. The function $g$ that minimizes the formulation above is known as a smoothing spline.

Smoothing splines avoid overfitting

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

Learn Before

Related