- Short connections are inserted in the plain 34-layer network.
- Solid lines showcases connections having same dimension.
- Dotted lines indicate connections having varying dimensions.
- Either zero padding or projection shortcut to increase dimension.

San Jose State University

Plain vs. ResNets Convolutional Neural Network Architectures

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

Deep Residual Learning for Image Recognition

- This is a 34 layer network.
- 3x3 filters are deployed in most of the convolutional layers in this model.
- Dimensionality reduction of feature map is achieved using a stride of 2 instead of 
   maxpool layer as in VGG-19
- Feature map's dimensionality reduction results in doubling of filters used in order to 
   conserve time complexity
- The number of filters in this network is less than that of VGG-19.
- This model has only 18% floating point operation of VGG-19 network.

Plain Network for ImageNet Dataset

ResNet  for ImageNet Dataset

- Image resizing for sale augmentation
- Batch normalisation after convolution and before activation.
- Stochastic gradient descent with a mini batch size of 256

Implementation of Networks

1. 34-layer ResNet reduces top-1 error by 3.5%, compared to 34-layer plain convolutional neural network:

2. 18-layer ResNet convereges faster than 18-layer plain convolutional neural network in early stage.

|     | plain | ResNet |
| -----------| ----------- | ----------- |
| 18-layer | 27.94 %     | 27.88%       |
| 34-layer | 28.54%   | 25.03%      |


Learn Before

Related