Iterative LLM-Based Prompt Search

Expansion in Prompt Search

Applying Classic Optimization Techniques to Prompt Optimization

A team is developing a system to automatically find the best prompt for summarizing legal documents. Their process is as follows:

1. They create a large, diverse list of 100 potential prompts.
2. They use a small, representative dataset to calculate an accuracy score for each of the 100 prompts.
3. They select the prompt with the highest accuracy score from the initial list and the process concludes.

Which critical element of an effective search strategy is missing from their approach?

Evaluating Prompt Search Strategies

Critique of a Prompt Finding Method