Compression of Pre-trained Language Models
Pre-trained language models (PLMs) are computationally expensive to deploy and serve. They often require compression, such as through knowledge distillation, to satisfy the latency and capacity constraints of real-world applications.
0
1
Tags
Data Science
Related
PLM Selection
Domain adaptation
Task-Specific Layers for Pre-Trained Language Models
Compression of Pre-trained Language Models
Ways to compress PTMs
A development team has created a large, high-performance language model for a new smartphone application that provides real-time text summarization. During user testing, they observe that while the summaries are highly accurate, the application is slow to respond and causes the phone's battery to drain rapidly. Which of the following strategies would be the most appropriate first step to address these specific performance issues on the device?
Deployment Strategy for a New AI Assistant
Deployment Challenges of Large Models
For any real-world application, applying compression techniques to a large pre-trained model is the optimal deployment strategy because it reduces model size and improves computation efficiency without compromising the model's performance.
Compression of Pre-trained Language Models