Learn Before
Evaluating a Policy Update for a Chatbot
A chatbot is being trained to be more helpful. In a situation where a user says 'I can't find my order,' the chatbot needs to decide on its next action. Before a training update, the reference policy gave the action 'Provide a link to the order tracking page' a probability of 0.3. After the training update, the new policy gives the same action a probability of 0.75. Analyze this policy change. First, determine if the action is now more favored by the new policy compared to the reference policy. Second, explain why this specific change likely represents a successful training step towards the goal of being more helpful.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In a reinforcement learning process, a policy is updated. For a specific state-action pair, the probability of selecting the action under the original policy was 0.2. After the update, the probability of selecting the same action in the same state under the new policy is 0.5. Based on the relationship between these two probabilities, what can be inferred about the policy update for this specific action?
Evaluating a Policy Update for a Chatbot
Consider a reinforcement learning agent being trained. For a specific state-action pair, the ratio of the action's probability under the newly updated policy to its probability under the original reference policy is calculated to be 0.75. This result signifies that the training update has made the agent more likely to select this action in the future.