Critiquing a Document Similarity System
A legal tech company is developing a feature to find similar documents within a large database of contracts. Their current method uses a pre-trained, general-purpose language model. To get a single vector representation for each contract, they process the text and then calculate the average of the output vectors for all the words. This approach has proven unreliable, often failing to capture the nuanced legal arguments and instead just matching documents with overlapping keywords.
Critique this averaging-based approach. Explain why it is likely failing and propose a more effective strategy that involves adapting the pre-trained model to specialize in this task. Justify why your proposed strategy would lead to more meaningful document representations.
0
1
Tags
Ch.4 Alignment - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A development team is building a system to classify customer support emails as 'Urgent' or 'Not Urgent'. They start with a general-purpose, pre-trained language model. Their initial strategy involves feeding an email into the model and using the numerical representation of the final word as input for a classifier. This approach yields poor results, often misclassifying long emails where the concluding words are not indicative of the overall sentiment.
To improve performance, the team modifies their approach. They add a new classification component and retrain the entire system on their dataset of labeled emails. The specific goal of this retraining is to adjust the model's parameters so that it produces a single, fixed-size numerical summary that captures the meaning of the entire email. This new summary vector is then used by the classifier, leading to a significant increase in accuracy.
Which of the following statements provides the most accurate evaluation of the team's successful adaptation process?
Choosing a Fine-Tuning Strategy for Sequence Summarization
Critiquing a Document Similarity System