Case Study

Optimizing a Chatbot Response System

As the lead designer for a new customer support chatbot, you must decide on the strategy for its two-stage response system. In Stage 1, an initial model generates 10 possible responses to a user's query. In Stage 2, a more powerful and accurate model selects the single best response from that set of 10 to show the user. You are presented with two configurations for Stage 1:

  • Configuration A: Generates 10 responses that are all very similar to each other. They are consistently grammatically correct and directly relevant, but lack creativity and sound robotic. The final selected answers are always adequate but never exceptional.
  • Configuration B: Generates 10 highly varied responses. Some are creative and empathetic, some are slightly off-topic, and a few contain minor errors. The final selected answers are inconsistent: sometimes they are brilliant and solve the user's problem perfectly, but other times they are unhelpful.

Which configuration (A or B) provides a more promising foundation for long-term improvement of the chatbot? Justify your choice by explaining the trade-off involved, and propose one specific change to the overall system to mitigate the primary weakness of your chosen configuration.

0

1

Updated 2025-10-02

Contributors are:

Who are from:

Tags

Ch.5 Inference - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Evaluation in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science