Learn Before
Explain why the shortcut input cannot be combined directly with the residual output during this stage transition, and describe how applying a linear projection resolves this issue.
0
1
Tags
Prep Sessions
Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Ch.3 Deep Residual Network Architecture - Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Residual Formulation and Shortcut Connections - Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Identity versus Projection Shortcuts - Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Ch.4 Residual Network Experiments and Applications - Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
ImageNet Classification and Model Variations - Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Related
Match each shortcut strategy for handling dimension changes to its corresponding implementation rule.
Order the operations performed to compute the output of a residual block with a projection shortcut when changing dimensions.
Explain why Option B achieves higher accuracy than Option A, and state which type of shortcut should be favored whenever dimensions match to minimize model size and computational complexity.
Match each mathematical or structural element of dimension matching to its specific role in residual networks.
Order the shortcut strategies by the total number of parameters they add to the network's shortcut connections, from fewest (least parameters) to most (greatest parameters).
Based on the evaluated shortcut strategies, evaluate whether switching from Option B to Option C is justified in terms of accuracy and model complexity, and explain why identity shortcuts are generally favored.
Match each shortcut evaluation strategy to its corresponding performance trade-off and empirical finding in residual networks.
When downsampling feature map sizes across stages in a residual network, the operation is performed with a stride of ___ .
Explain why the shortcut input cannot be combined directly with the residual output during this stage transition, and describe how applying a linear projection resolves this issue.