Prediction via Optimization
The specific prediction, ŷ, made by a model is formally determined by an optimization rule that operates on an objective function. For instance, in probabilistic classification models, the prediction is the class 'y' that maximizes the conditional probability P(y|x), expressed as ŷ = argmax_y P(y|x). The 'arg' operator (argmax for maximization, argmin for minimization) identifies the input value that achieves the optimal value of the function.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Components of Karim's Constrained Choice Problem
Formulating Karim's Constrained Optimization Problem
A farmer plans to grow two types of crops: wheat (W, measured in bushels) and corn (C, measured in bushels). The farmer's primary goal is to achieve the highest possible profit from their land. The market price gives a profit of $5 per bushel of wheat and $8 per bushel of corn. The farmer faces limitations on available land and water. Given this situation, which of the following expressions represents the farmer's objective function?
Formulating an Objective Function for a Diet Plan
Analyzing a Business Decision
In any problem where a decision-maker faces limitations on their choices, the function representing the goal they are trying to optimize must be a quantity they wish to maximize.
Evaluating Competing Policy Objectives
A decision-maker faces a problem where they must make a choice under certain limitations. Match each decision-maker's scenario with the correct mathematical expression that represents their primary goal.
A city's transportation department is designing a new public bus system. Their main goal is to reduce the average commute time for its citizens, but they must work within a specific budget and a limited number of buses. The mathematical formula that represents the average commute time, which the department is trying to minimize, is referred to as the ______.
A city planner is trying to determine the optimal allocation of funds between building new parks and repairing existing roads. The goal is to maximize the overall well-being of the city's residents, subject to a fixed annual budget. To solve this as a formal optimization problem, the planner must follow a series of logical steps. Arrange the following steps in the correct logical order for setting up this problem.
Evaluating a Public Health Objective Function
Interpreting a Firm's Objective
Prediction via Optimization
Preference for Divergence-Based Objective Functions
Chain Rule
Autoregressive Conditional Probability
General Notation for Conditional Probability Models
Prediction via Optimization
A language model is analyzing a text corpus of 10,000 two-word phrases. The analysis reveals the following counts:
- The word 'deep' is the first word in 400 phrases.
- The word 'learning' is the second word in 250 phrases.
- The specific phrase 'deep learning' occurs 80 times.
Based on this data, what is the probability that the second word of a phrase is 'learning', given that the first word is 'deep'?
Predictive Text Model Comparison
Interpreting Conditional Probabilities in Text
Learn After
Using Optimized Predictions as Learning Targets
General Formula for Prediction via Maximum Probability
A language model is given the input 'The ocean is...' and calculates the conditional probability for four candidate words to be the next word. Based on the values below, which word would a model that predicts by maximizing probability choose?
- P("deep" | "The ocean is...") = 0.75
- P("cold" | "The ocean is...") = 0.15
- P("running" | "The ocean is...") = 0.09
- P("quiet" | "The ocean is...") = 0.01
Evaluating a Prediction Strategy
Spam Filter Classification