Gaussian noise is statistical noise that has a probability density function equal to that of the normal (Gaussian) distribution.

For example, when applying Gaussian noise to an image, the mean value of the noise determines the average change in pixel values. If the noise has a mean of -2, on average 2 is subtracted from each pixel. If the noise has a mean of 3, on average 3 is added to each pixel.

University of Michigan - Ann Arbor

Google

A normal distribution, also known as a Gaussian distribution, is a continuous probability distribution defined by its mean $$\mu$$ and variance $$\sigma^2$$ (where $$\sigma$$ is the standard deviation). The probability density function is given by the formula: $$p(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp\left(-\frac{1}{2 \sigma^2} (x - \mu)^2ight)$$. Changing the mean $$\mu$$ corresponds to a shift of the distribution along the $$x$$-axis, while increasing the variance $$\sigma^2$$ spreads the distribution out and lowers its peak.

Gaussian (Normal) Distribution

The mathematical formula for the normal distribution's probability density function can be directly implemented in Python. Using standard libraries like `math` and `numpy` (or their deep learning equivalents in PyTorch, JAX, or TensorFlow), a function can be defined to calculate the probability density $$p$$ for a given value $$x$$, given a mean $$\mu$$ and standard deviation $$\sigma$$. The code implementation is as follows:

```python
def normal(x, mu, sigma):
    p = 1 / math.sqrt(2 * math.pi * sigma**2)
    return p * np.exp(-0.5 * (x - mu)**2 / sigma**2)
```

This runnable implementation aligns with the formal definition: $$p(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp\left(-\frac{1}{2 \sigma^2} (x - \mu)^2\right)$$.

Normal Distribution Probability Density Function Code

The Central Limit Theorem is a foundational mathematical principle in probability theory stating that the average of $$n$$ independent random samples drawn from any distribution (with mean $$\mu$$ and standard deviation $$\sigma$$) will approximately follow a normal distribution centered at the true mean $$\mu$$ with a standard deviation of $$\sigma/\sqrt{n}$$, as the sample size $$n$$ grows large. In machine learning, this theorem explains why the empirical error of a classifier evaluated on a test set converges to its true population error at an asymptotic rate of $$\mathcal{O}(1/\sqrt{n})$$.

Central Limit Theorem

The probability density function plot for a Gaussian distribution has a symmetric bell shape, where $$\sigma$$ defines the width of the curve and $$\mu$$ describes the center of the curve. The y-axis represents the probability density. The curve shown below illustrates a standard normal distribution, which has a mean of $$\mu = 0$$ and a standard deviation of $$\sigma = 1$$.

Plot of Normal Distribution Density Function

When we have $$m$$ i.i.d. samples $$\left\{x^{(1)},...,x^{(m)}\right\}$$ from a Normal distribution with parameters $$\mu$$ and $$\sigma$$, such that: $$p(x^{(i)};\mu, \sigma) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(-\frac{1}{2\sigma^2}\left(x^{(i)}-\mu\right)^2\right)$$, an unbiased estimator for $$\mu$$ is the sample mean: $$\hat{\mu}_m = \frac{1}{m} \sum_{i=1}^m x^{(i)}$$.

Learn Before

Related