Learn Before
Computational Benefits of Freezing Pretrained Parameters
When utilizing a pretrained model solely as a feature extractor, its parameters are explicitly frozen to prevent them from being updated during training. Because these feature extraction layers remain fixed, the optimization algorithm does not need to compute or store their gradients during backpropagation. This approach substantially reduces the overall training time and minimizes the memory footprint required for storing gradients, making it highly efficient for adapting large models to new tasks.
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Related
POPULAR PRE-TRAINED MODELS
Number of Layers to Freeze When Using a Pre-trained Model for Transfer Deep Learning
Implementing Freezing Layers When Using a Pre-trained Model for Transfer Deep Learning
An engineer needs to build a model to classify 15 types of local wildflowers using a custom dataset of only 900 images. They select a very deep and complex neural network that was previously trained on a dataset of over a million general-purpose images (e.g., animals, vehicles, household objects). The engineer's strategy is to retrain all layers of this complex network from scratch, using only their small wildflower dataset. What is the most likely outcome of this strategy?
You are tasked with building an image classifier for a new, specialized task (e.g., identifying specific types of industrial equipment), but you only have a small, custom dataset. You decide to adapt a model that has already been trained on a very large, general image dataset. Arrange the following steps in the correct logical order to implement this strategy.
Adapting a Pre-trained Network for a New Task
Hot Dog Recognition Example in Fine-Tuning
Computational Benefits of Freezing Pretrained Parameters