Monte Carlo Sampling for Approximating the Predictive Distribution
In the Bayesian viewpoint of prompt ensembling, directly computing the predictive distribution integral for an output is computationally infeasible due to the potentially infinite space of possible prompts . To address this, methods like Monte Carlo sampling are used to approximate the integral. This involves using a manageable, finite set of sample prompts, weighted by their likelihoods given the problem, defined by the prior distribution of prompts .
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Learn After
Estimating Model Performance Under Uncertainty
A machine learning engineer is working with a large ensemble of language models. To generate a final prediction, they need to average the outputs over an intractably large space of possible input prompts. They decide to approximate this average by using a manageable, finite set of sample prompts. What is the fundamental trade-off inherent in this approximation strategy?
Impact of Sample Size on Estimation Accuracy