A language model is given two tasks. Task A is: 'Write a short story about a journey to the mountains.' Task B is: 'Write a five-character regulated poem about a journey to the mountains.' Explain why Task B is a more complex and demanding test of the model's ability to follow instructions.

Google

A specific example of generating text based on user requirements is the task of creating a five-character regulated Chinese poem. This task requires the model to follow strict constraints on form, style, and content, which would be specified in a prompt.

Example of Constrained Text Generation: Chinese Poem

An example prompt for generating a five-character regulated Chinese poem consists of several key components. It establishes a persona ('You are a poet.'), gives a direct command ('Please write a traditional Chinese poem.'), and lists specific constraints. These constraints can define the poem's structure ('Structure: Five-character regulated poem') and its desired emotional tone, such as conveying happiness and renewal through vivid spring imagery.

Example Prompt for Generating a Chinese Poem

A developer is testing a language model's ability to follow a complex set of rules that govern not just the topic, but also the specific structure and stylistic conventions of the output. Which of the following tasks would be the most effective and direct test of this specific capability?

Analyzing Task Complexity in Text Generation

Evaluate the user's conclusion in the following scenario. Is the model's failure to produce the desired poem a limitation of its creative ability, or is there a different issue at play? Justify your answer and suggest a revised prompt that would be more likely to generate the user's intended output.

Learn Before

Related