There is a fundamental distinction between probabilities and statistics. Probabilities are theoretical quantities that describe the underlying data generating process, such as the inherent fairness of a coin. In contrast, statistics are empirical quantities computed directly from observed data, such as the observed frequency of a specific outcome in a sample.