Measuring Pre-Activation Residual-Branch Responses on CIFAR-10
To examine how network layers transform intermediate representations on CIFAR-10, capture the output of each convolution immediately after batch normalization and before subsequent ReLU activation or element-wise addition. For each layer, summarize the strength of these pre-activation responses using their standard deviation, enabling comparisons across layers and architectures. In a residual network, the measurement describes the learned residual branch rather than the combined block output .
0
1
Tags
Prep Sessions
Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Ch.4 Residual Network Experiments and Applications - Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Layer Response Analysis on CIFAR-10 - Foundational Deep Learning Architectures: Transformers and Residual Networks @ University of Michigan - Ann Arbor
Learn After
At which point in a network layer are responses captured to analyze intermediate representations on CIFAR-10?
In residual networks, the layer response measurement reflects the total combined output .
Which statistical metric is computed across layers to quantify the strength of intermediate layer responses?