1. Gather the fraction s = $$\frac{k}{n}$$ of training points whose $$X_i$$ are closest to $$X_0$$.
2. Assign a weight $$K_{i0}=K(x_i,x_0)$$ to each point in this neighborhood, so that the point furthest from $$X_0$$ has weight zero, and the closest has highest weight. All but these k nearest neighbors get weight zero.
3. Fit a weighted least squares regression of the $$y_i$$ on the $$x_i$$ using the aforementioned weights, by finding $$\hat{\beta _0}$$ and $$\hat{\beta _1}$$ that minimize $$\sum^n _{i=1} K_{i0}(y_i-\beta_0-\beta_1 x_i)^2$$.
4. The fitted value at $$x_0$$ is given by $$\hat{f(x_0)}=\hat{\beta_0}+\hat{\beta_1}x_0$$.

University of Michigan - Ann Arbor

Local regression works by choosing a target point ($x_0$) and its nearby training observations in order to find a fit. Points are then assigned weights (given by $K_{i0} = K(x_i, x_0)$) based on their distance from $x_0$; points closest to $x_0$ are assigned the highest weight, and as points get farther from $x_0$, they are assigned a lower weight. The point farthest from $x_0$ will be assigned a weight of 0. A weighted least squares regression is then performed by minimizing  $$\sum_{i=1}^{n} K_{i0} (y_i -  \beta_0- \beta_1x_i)^{2}$$. The new fitted value of $x_0$ is then found using the new regression model.

Local Regression

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

https://towardsdatascience.com/loess-373d43b03564 

This post is from one of my favorite data science blogs on the net, Towards Data Science. It breaks down one of last week's reading topics, local regression. 

Learn Before

Related