Simple linear regression commonly uses a least squares approach, where the linear model attempts to minimize the residual distance between the actual points and the predicted line. Consequently, its residual is defined as:
$$e_i = y_i -\hat{y}_i$$

where $y_i$ represents the actual ith response value and $\hat{y}_i$ represents the ith response value of the linear model. 

Least Squares Approach

```python
#imports 
import numpy as np 
from sklearn.linear_model import LinearRegression

#make up data 
array_x = np.array([5, 10, 15, 20, 25]).reshape((-1, 1))
#calling reshape makes sure that the array is 2D 
array_y = np.array([5, 3, 15, 23, 6])

#create a model 
model_xy = LinearRegression()

#using model 
mode.fit(array_x, array_y)

#getting results 
rsq_val = model_xy(array_x, array_y)
intercept = model_xy.intercept_
slope = model_xy.coef_

#predicting a response 
y_pred = model_xy.predict(x)

```

Python: Simple Linear Regression 

In R, lm() is a function that can take many parameters. Common parameters include formula, which describes a model, and data, which contains a dataset. 
 

In simple linear regression in R, a formula parameter is written YVAR~XVAR, where XVAR is the predictor variable and YVAR is the response variable. 

A simple linear regression equation with data and formula parameters may look like:
``` R
lm(formula = YVAR~XVAR, data = datavar)
```


Simple Linear Regression in R: lm()

Link: $\href{http://www.sthda.com/english/articles/40-regression-analysis/165-linear-regression-essentials-in-r/}{sthda}$
The image references a simple linear regression line, where the 'e' represents the residual error, 'b0' represents the y-intercept, and 'b1' represents the change in Y (response value) for one unit increase in X.

Image Reference: Simple Linear Regression

Simple Linear Regression predicts a model for the relationship between a predictor matrix (independent variable) $X$ and the response (dependent variable) $Y$.
$$Y=\beta _1 X+\beta _0+\epsilon$$
The coefficients $\beta _1$ and $\beta _0$ are called slope and intercept, respectively. The estimated values for these coefficients are shown as $\hat{\beta _1}$ and $\hat{\beta _0}$.
$$\hat{Y}=\hat{\beta _1} X+\hat{\beta _0}$$

University of Michigan - Ann Arbor

Based on the number of independent variables used, there are two types of linear regression:
- Simple Linear Regression
- Multiple Linear Regression

Types of linear regression

James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 112, pp. 3-7). New York: springer.

An Introduction to Statistical Learning with Applications in R

Simple Linear Regression

Multiple Linear Regression is a linear regression that uses **two or more independent variables** to predict the outcome of a response variable. It is more effective and practical to predict or estimate dependent variables by the optimal combination of multiple independent variables than only one independent variable. 
$$y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + ... + \beta_p x_{ip} + \epsilon$$

Multiple Linear Regression

While simple linear regression can help us understand the relationship between a single predictor and a response, using more than one predictor often proves to be beneficial in predicting responses. Multiple linear regression assigns different slope coefficients to each predictor to build a more comprehensive model than its simple counterpart.

Simple Linear Regression vs. Multiple Linear Regression

https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a#:~:text=This%20is%20a%20form%20of,linear%20regression%20looks%20like%20this

Learn Before

Related

Learn After