Self-Instruct for Generating Fine-Tuning Data

A development team is fine-tuning a language model to function as a specialized customer support chatbot. They have collected a large dataset of high-quality, expert-written answers to common customer issues. To create the training pairs, the team manually wrote simple, direct questions corresponding to each answer. After deployment, they observe that the model performs well on straightforward queries but fails to provide correct answers when users phrase their questions in unconventional, complex, or indirect ways. Which of the following strategies represents the most effective next step to address this specific performance issue?

Evaluating Fine-Tuning Data Generation Strategies

Analyzing Data Generation Strategies for Model Robustness