Activity (Process)

Paired Bootstrap Resampling for Hypothesis Testing

The paired bootstrap is a hypothesis-testing and confidence-interval procedure for comparing two procedures (or systems) AA and BB evaluated on the same nn items. Let di=m(A,i)m(B,i)d_i = m(A,i) - m(B,i) be the per-item paired difference of a performance metric mm, with observed mean dˉ\bar d. For b=1,,Bb = 1,\dots,B replicates (with BB typically in the thousands), draw a bootstrap index set I(b){1,,n}I^{*(b)} \subset \{1,\dots,n\} of size nn with replacement and recompute the paired statistic on the same resampled items for both systems, e.g. dˉ(b)=1niI(b)di.\bar d^{*(b)} = \tfrac{1}{n}\sum_{i\in I^{*(b)}} d_i. The empirical distribution {dˉ(b)}b=1B\{\bar d^{*(b)}\}_{b=1}^{B} approximates the sampling distribution of the paired difference, from which a percentile confidence interval [dˉ(α/2),dˉ(1α/2)][\bar d^{*(\alpha/2)},\bar d^{*(1-\alpha/2)}] is read off and a two-sided p-value can be derived (e.g., as the bootstrap probability that the difference reverses sign). Resampling item indices rather than the two systems independently preserves the pairing structure, which removes between-item variance and gives more power than an unpaired test.

0

1

Updated 2026-05-16

Contributors are:

Who are from:

Tags

Science