Short Answer

Comparing AI Evaluation Systems

An AI development team is building a chatbot to provide factual answers to historical questions. They are considering two different automated systems to improve its responses:

System 1: An automated process that cross-references the chatbot's answer against a curated database of historical facts to assign a 'correctness' score.

System 2: A model trained on data where human historians have rated pairs of answers for the same question, indicating which one is more comprehensive, well-written, and nuanced.

Analyze these two systems. Explain which system functions like a verifier and which functions like a reward model, and describe the fundamental difference in what each system is designed to evaluate.

0

1

Updated 2025-10-05

Contributors are:

Who are from:

Tags

Ch.3 Prompting - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Analysis in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science