Learn Before
Concept

Exponentially Weighted Average

Exponentially weighted average is a technique frequently used for time-series data. By taking the average sum of previous data, you could smooth your data series and get an approximate trend of it.

Consider you have a series of data points θ0,...,θn\theta_0,...,\theta_n, {vt=θtt=0vt=βvt1+(1β)θtotherwise\left\{ \begin{array}{ll}v_t = \theta_t & t=0 v_t = \beta v_{t-1} +(1-\beta)\theta_t & otherwise \end{array}\right. If we expand the second formula, vt=βvt1+(1β)θtv_t = \beta v_{t-1}+(1-\beta)\theta_t =(1β)θt+β(βvt2+(1β)θt1)= (1-\beta)\theta_t+\beta(\beta v_{t-2}+(1-\beta)\theta_{t-1}) =(1β)θt+(1β)βθt1+(1β)β2θt2+...= (1-\beta)\theta_t + (1-\beta)\beta\theta_{t-1}+ (1-\beta)\beta^2\theta_{t-2}+... To get a sense of how the weighted term changes as β\beta gets closer to 1, (1ϵ)1/ϵ1ϵβ1/(1β)1ϵ(1 - \epsilon)^{1 / \epsilon}\approx \frac{1}{\epsilon} \Rightarrow \beta^{1/(1-\beta)}\approx \frac{1}{\epsilon} If we denote wiw_i be the weight we assign to θi\theta_i, then wt1/(1β)=1ϵwtw_{t-1/(1-\beta)}=\frac{1}{\epsilon}w_t Therefore, we are approximately average over $1/(1-\beta)days when calculatingv_t$.

0

4

Updated 2026-05-15

Tags

Data Science

D2L

Dive into Deep Learning @ D2L