Before a linear regression model can be trained using algorithms like stochastic gradient descent, its learnable parameters must be assigned initial values. The standard approach is to initialize the weight parameters randomly by drawing from a normal (Gaussian) distribution with a mean of $$0$$ and a small standard deviation, such as $$0.01$$. Meanwhile, the bias parameter is typically initialized to exactly $$0$$.

Claude

In a multiple-feature linear regression model, weights are parameters that determine the influence of each individual feature on the target prediction. When the model is expressed as a weighted sum, such as $$\hat{y} = w_1 x_1 + \dots + w_d x_d + b$$, each weight quantifies the specific contribution of its corresponding feature. In compact linear algebra notation, these parameters are collected into a weight vector $$\mathbf{w} \in \mathbb{R}^d$$.

Linear Regression Weight Parameters

Dive into Deep Learning

Linear Regression Parameters Initialization

Defining a linear regression model involves specifying its forward pass, which mathematically relates input features to the predicted output. This is accomplished by computing the matrix-vector product of the input feature matrix $$\mathbf{X}$$ and the model's weight vector $$\mathbf{w}$$, resulting in a new vector. The scalar bias offset $$b$$ is then added to this product. Through the tensor broadcasting mechanism, this single scalar bias is automatically added to each component of the $$\mathbf{Xw}$$ vector. In an object-oriented model design, this computation is defined within a `forward` method.

Learn Before

Related