Analyze the following scenario and identify the primary source of computational inefficiency. Explain how the nature of the task exacerbates this problem.

Google

Strategy X: For each email, the system sends a long prompt that includes a detailed, 200-word definition of each category, several examples, and the full text of the customer email.
Strategy Y: For each email, the system sends a short prompt con

The inefficiency caused by long and complex natural language prompts is particularly acute in applications where a task must be performed repeatedly. In such scenarios, inputting the same lengthy prompt a large number of times leads to significant and unnecessary computational costs.

Inefficiency of Long Prompts in Repetitive Tasks

LLM-Powered Email Categorization System

A company is developing an automated system to classify thousands of customer support emails per day into one of three categories: 'Urgent', 'Standard', or 'Spam'. They are considering two prompting strategies for their large language model.

- **Strategy X:** For each email, the system sends a long prompt that includes a detailed, 200-word definition of each category, several examples, and the full text of the customer email.
- **Strategy Y:** For each email, the system sends a short prompt con

A company designs a customer service chatbot. For every user question, the system prepends a 500-word block of text containing detailed instructions and company policies. This combined text is then sent to a language model. Analyze the computational efficiency of this design for a system that handles thousands of user questions daily, and identify the primary source of the inefficiency.

Learn Before

Related