In a multi-head attention model, the linear bias scalar for each attention head is determined by a geometric progression. The value for the k-th head is calculated using the formula: β_k = 1 / (2^(8/k)). For a model with at least 4 heads, what is the calculated bias scalar for the 4th head (i.e., when k=4)?

In a multi-head attention mechanism, a scalar bias for each head is determined by a geometric progression. The value for the k-th head is calculated using the formula: β_k = 1 / (2^(8/k)). How does the value of this scalar bias change as the head index k increases?

Selecting an Appropriate Bias Scalar Formula