While asymptotic convergence provides intuition for how test error behaves as the dataset size approaches infinity, practical machine learning relies on finite datasets. Because the classification error is a bounded random variable (between $0$ and $1$), valid finite sample bounds can be established using Hoeffding's inequality. It guarantees that the probability of the empirical error $\epsilon_\mathcal{D}(f)$ exceeding the true error $\epsilon(f)$ by a margin of $t$ or more is exponentially bounded: $P(\epsilon_\mathcal{D}(f) - \epsilon(f) \geq t) < \exp\left( - 2n t^2 \right)$