In training following function was minimized:

$NLL(p(X), y) = \sum_{i=1}^S y_i \log p(x_i) + (1-y_i) \log (1-p(x_i))$ here ys are outcomes and xs are inputs.

Biases, embeddings follow hyperpriors - $w_k \sim N ( \mu, 1 /\ \lambda)$ $v_{kf} \ sim N ( \mu, 1 /\ \lambda)$ $\mu$, $\lambda$ follow hyperpriors - $\mu \sim N(0, 1)$ $\lambda \sim \Gamma (0, 1)$

Due to these hyperpriors the authors don't need finetuning.