Constructed example. Its rounded critical values are those of the one-sample t-test example with 9 degrees of freedom in the KPU chapter cited below.

Assumptions. A one-sample t test on $n = 10$ independent observations from a normal distribution whose variance is unknown and positive. The null hypothesis is that the population mean equals a specified value $\mu_0$. When the null hypothesis is true, the test statistic $T$ has a t distribution with 9 degrees of freedom, which is symmetric around 0. Let $c_1$ be the exact .95 quantile of that distribution (about 1.833) and $c_2$ the exact .975 quantile (about 2.262).

A researcher plans a two-tailed test at alpha = .05, which rejects when $|T| > c_2$. After seeing that the sample mean fell below $\mu_0$, the researcher switches to a one-tailed test in that direction, rejecting when $T < -c_1$. Had the mean fallen above $\mu_0$, the researcher would have tested the upper tail instead, rejecting when $T > c_1$.

Suppose the null hypothesis is true and the assumptions hold.

• A one-tailed test whose direction is fixed before seeing the data has a false-rejection probability of exactly .05: its nominal level.
• The procedure as actually run rejects whenever $T < -c_1$ or $T > c_1$, so its whole-procedure false-rejection probability is .05 + .05 = .10.
• The branch in which the mean fell below $\mu_0$ (that is, $T < 0$) has probability .5, so within that branch the conditional false-rejection rate is .05 / .5 = .10. The other branch gives the same value.

The level printed with each test is still .05, but this procedure has twice that error probability, because the direction was chosen using the same data. Under these assumptions, running the planned two-tailed test, or fixing the direction before seeing the data, keeps the probability at .05.

These numbers hold for the stated assumptions and for choosing a direction after looking. They do not show that every data-dependent step, such as an assumption check before choosing between tests, doubles the error rate: some such procedures stay close to the nominal level and others do not, so each has to be evaluated as a whole.

Source of the rounded critical values: KPU, Some Basic Null Hypothesis Tests (CC BY-NC-SA 4.0; values cited, text not adapted).