Computational Infeasibility of the Bayesian Predictive Distribution Integral
A significant practical challenge in applying the Bayesian prompt ensembling formula is the direct computation of its integral. Because the space of possible prompts, x, can be extremely large or even infinite, it is often computationally infeasible to evaluate the integral directly.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Computational Infeasibility of the Bayesian Predictive Distribution Integral
A team is using a probabilistic method to combine the outputs from a language model for a variety of different prompts (x) to solve a single problem (p). The final probability of a specific output (y) is calculated by integrating over all possible prompts. The formula for this is: Pr(y|p) = ∫ Pr(y|x) Pr(x|p) dx. In this formula, Pr(y|x) is the model's likelihood of the output given a prompt, and Pr(x|p) is a prior distribution representing the assumed suitability of a prompt for the problem. How would the calculation of Pr(y|p) be affected if the prior distribution Pr(x|p) was assumed to be uniform, meaning every possible prompt is considered equally suitable?
Calculating Predictive Probability with Prompt Priors
A probabilistic approach to combining outputs from different prompts for a single problem involves the following formula: Match each mathematical term from the formula with its correct conceptual description.