In the Adam optimizer, the state variables for momentum ($\mathbf{v}_t$) and the second moment ($\mathbf{s}_t$) are typically initialized to zero ($\mathbf{v}_0 = \mathbf{s}_0 = 0$). This initialization introduces a significant bias towards smaller values during the initial training steps. To correct this bias, Adam re-normalizes the terms using the sum of the weights $\sum_{i=0}^{t-1} \beta^i = \frac{1 - \beta^t}{1 - \beta}$. The resulting debiased, or normalized, state variables are computed as $\hat{\mathbf{v}}_t = \frac{\mathbf{v}_t}{1 - \beta_1^t}$ and $\hat{\mathbf{s}}_t = \frac{\mathbf{s}_t}{1 - \beta_2^t}$.