In optimal RegNet configurations, the width of the network (number of channels) increases linearly with the block index. This relationship is approximated by the equation $$c_j \approx c_0 + c_a j$$, where $$j$$ represents the block index and $$c_a > 0$$ is the slope. Because the block width can only be changed at the beginning of each stage rather than per block, this continuous linear scaling is practically implemented as a piecewise constant function engineered to match the theoretical linear dependence.

Claude

The RegNet architecture emerges from the $$\textrm{AnyNetX}_E$$ design space, which simplifies the generic AnyNet design by enforcing a set of easily interpretable principles. Specifically, it requires sharing both the bottleneck ratio ($$k_i = k$$) and the group width ($$g_i = g$$) across all stages $$i$$. Furthermore, it mandates that both the network width ($$c_i$$) and the network depth ($$d_i$$) must increase monotonically across successive stages, meaning $$c_i \leq c_{i+1}$$ and $$d_i \leq d_{i+1}$$.

RegNet Design Space

Dive into Deep Learning

RegNet Channel Width Scaling

The RegNetX32 is an effective 32-layer architectural variant derived from the RegNet design space. It is configured without a bottleneck ($$k = 1$$) and uses a uniform group width of $$g = 16$$ across all stages. For its stage progression, the first stage is $$d_1=4$$ blocks deep with a width of $$c_1 = 32$$ channels, and the second stage is $$d_2=6$$ blocks deep with a width of $$c_2 = 80$$ channels. As with all RegNetX models, each stage progressively reduces the spatial resolution while increasing the number of output channels.

RegNetX32 Variant

The fundamental design principles of the RegNet space—such as shared bottleneck ratios, shared group widths, and linear channel scaling—are robust enough to apply beyond standard convolutional blocks. They remain highly effective when applied to Squeeze-and-Excitation (SE) network designs, which utilize a global channel activation mechanism. This specific adaptation of the RegNet principles with SE blocks is known as the RegNetY architecture.

Learn Before

Related