The two-dimensional linear model is among the simplest regression models encountered in machine learning. Its inherent simplicity makes it highly effective for testing the accuracy of regression algorithms, as it allows practitioners to avoid complicating factors such as insufficient amounts of data or dealing with an underdetermined system of equations.

Claude

To evaluate machine learning models, we often generate synthetic datasets where the underlying ground truth relationship is known. For a linear regression task, we can draw a design matrix $$\mathbf{X}$$ of features from a standard normal distribution. The corresponding labels $$\mathbf{y}$$ are computed by applying a ground truth linear function defined by true weights $$\mathbf{w}$$ and bias $$b$$, and then corrupting the output with additive noise $$\boldsymbol{\epsilon}$$ drawn from a normal distribution with mean $$\mu=0$$ and standard deviation $$\sigma = 0.01$$: $$\mathbf{y}= \mathbf{X} \mathbf{w} + b + \boldsymbol{\epsilon}$$. This procedure ensures that the generated labels simulate realistically observed data containing inherent random variation.

Synthetic Data Generation for Linear Regression

Dive into Deep Learning

The SyntheticRegressionData class encapsulates the procedural generation of synthetic datasets for linear regression models. Inheriting from a base DataModule (such as d2l.DataModule), its __init__ constructor takes the true parameters w and b, alongside hyperparameters like noise (defaulting to $$0.01$$), num_train, num_val, and batch_size. After invoking save_hyperparameters(), it constructs the feature matrix $$\mathbf{X}$$ from a normal distribution and computes the label vector $$\mathbf{y}$$ using the formula $$\mathbf{y} = \mathbf{X} \mathbf{w} + b + \boldsymbol{\epsilon}$$. This structure standardizes the dataset preparation for training and validation.

Learn Before

Related