https://arxiv.org/pdf/2107.13586.pdf
It's one amazing survey paper proposing existing prompt-tuning methods.

University of Michigan - Ann Arbor

Sentences created according to template-based generation are called prompts.

Prompts

It freezes most of the pre-trained language model and only tunes the continuous prompt part to reduce the memory use for task saving.

Prompt Tuning

The Power of Scale for Parameter-Efficient Prompt Tuning

1. Create prompt templates.
2. Create prompt vertablizer.
3. Fill text into prompts and use pre-trained model to predict.
4. Map the prediction results to the groundtruth results.

Basic Workflow of Prompt

It means to use several prompts to ask the same type of questions to improve the generalization ability of the generation module.

Prompt Ensemble

Use several similar prompts as prefix to reinforcement the model's ability to learn the prompt format.

Prompt Augmentation 

Separate one prompt into several prompts to improve the prompt's description ability.

Learn Before

Related