Case Study

Factual Accuracy for a Corporate Chatbot

A financial services company wants to deploy an internal chatbot to help its employees answer complex questions about the company's latest, highly-detailed compliance policies. The primary goal is to ensure every answer is factually correct and strictly based on the official policy documents. The development team is considering two approaches:

  1. Approach A: Build a system that first searches the company's internal database of policy documents to find the specific paragraphs relevant to an employee's question. Then, it passes these retrieved paragraphs to a large language model with the instruction to formulate an answer based only on the provided text.

  2. Approach B: Use a powerful, general-purpose large language model directly. The model would be prompted with the employee's question and a simple instruction to 'act as a compliance expert' and provide an accurate answer.

Which approach is more suitable for the company's goal? Justify your choice by evaluating the strengths and weaknesses of each approach in the context of the stated requirements.

0

1

Updated 2025-10-04

Contributors are:

Who are from:

Tags

Ch.3 Prompting - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Evaluation in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science