Learn Before
Concept icon
Concept

Exponentially Weighted Average

Exponentially weighted average is a technique frequently used for time-series data. By taking the average sum of previous data, you could smooth your data series and get an approximate trend of it.

Consider you have a series of data points θ0,...,θn\theta_0,...,\theta_n, {vt=θtt=0vt=βvt−1+(1−β)θtotherwise\left\{ \begin{array}{ll}v_t = \theta_t & t=0 v_t = \beta v_{t-1} +(1-\beta)\theta_t & otherwise \end{array}\right. If we expand the second formula, vt=βvt−1+(1−β)θtv_t = \beta v_{t-1}+(1-\beta)\theta_t =(1−β)θt+β(βvt−2+(1−β)θt−1)= (1-\beta)\theta_t+\beta(\beta v_{t-2}+(1-\beta)\theta_{t-1}) =(1−β)θt+(1−β)βθt−1+(1−β)β2θt−2+...= (1-\beta)\theta_t + (1-\beta)\beta\theta_{t-1}+ (1-\beta)\beta^2\theta_{t-2}+... To get a sense of how the weighted term changes as β\beta gets closer to 1, (1−ϵ)1/ϵ≈1ϵ⇒β1/(1−β)≈1ϵ(1 - \epsilon)^{1 / \epsilon}\approx \frac{1}{\epsilon} \Rightarrow \beta^{1/(1-\beta)}\approx \frac{1}{\epsilon} If we denote wiw_i be the weight we assign to θi\theta_i, then wt−1/(1−β)=1ϵwtw_{t-1/(1-\beta)}=\frac{1}{\epsilon}w_t Therefore, we are approximately average over 1/(1−β)1/(1-\beta) days when calculating vtv_t.

0

4

Concept icon
Updated 2026-05-15

Tags

Data Science

D2L

Dive into Deep Learning @ D2L