The RegNetX32 model can be implemented programmatically by subclassing the generic AnyNet architecture and supplying its specific hyperparameters. The model initialization establishes $$32$$ stem channels, a group width of $$16$$, and a bottleneck ratio of $$1$$. The body of the network is configured through an architectural tuple defining the depths and channel widths of its stages, specifically using depths of $$(4, 6)$$ and channels of $$(32, 80)$$ for the first two stages. This setup automates the construction of the network's sequential blocks.

PyTorch Implementation:
```python
class RegNetX32(AnyNet):
    def __init__(self, lr=0.1, num_classes=10):
        stem_channels, groups, bot_mul = 32, 16, 1
        depths, channels = (4, 6), (32, 80)
        super().__init__(
            ((depths[0], channels[0], groups, bot_mul),
             (depths[1], channels[1], groups, bot_mul)),
            stem_channels, lr, num_classes)
```

MXNet Implementation:
```python
class RegNetX32(AnyNet):
    def __init__(self, lr=0.1, num_classes=10):
        stem_channels, groups, bot_mul = 32, 16, 1
        depths, channels = (4, 6), (32, 80)
        super().__init__(
            ((depths[0], channels[0], groups, bot_mul),
             (depths[1], channels[1], groups, bot_mul)),
            stem_channels, lr, num_classes)
```

JAX Implementation:
```python
class RegNetX32(AnyNet):
    lr: float = 0.1
    num_classes: int = 10
    stem_channels: int = 32
    arch: tuple = ((4, 32, 16, 1), (6, 80, 16, 1))
```

TensorFlow Implementation:
```python
class RegNetX32(AnyNet):
    def __init__(self, lr=0.1, num_classes=10):
        stem_channels, groups, bot_mul = 32, 16, 1
        depths, channels = (4, 6), (32, 80)
        super().__init__(
            ((depths[0], channels[0], groups, bot_mul),
             (depths[1], channels[1], groups, bot_mul)),
            stem_channels, lr, num_classes)
```

RegNetX32 Code Implementation

The RegNetX32 is an effective 32-layer architectural variant derived from the RegNet design space. It is configured without a bottleneck ($$k = 1$$) and uses a uniform group width of $$g = 16$$ across all stages. For its stage progression, the first stage is $$d_1=4$$ blocks deep with a width of $$c_1 = 32$$ channels, and the second stage is $$d_2=6$$ blocks deep with a width of $$c_2 = 80$$ channels. As with all RegNetX models, each stage progressively reduces the spatial resolution while increasing the number of output channels.

Claude

The RegNet architecture emerges from the $$\textrm{AnyNetX}_E$$ design space, which simplifies the generic AnyNet design by enforcing a set of easily interpretable principles. Specifically, it requires sharing both the bottleneck ratio ($$k_i = k$$) and the group width ($$g_i = g$$) across all stages $$i$$. Furthermore, it mandates that both the network width ($$c_i$$) and the network depth ($$d_i$$) must increase monotonically across successive stages, meaning $$c_i \leq c_{i+1}$$ and $$d_i \leq d_{i+1}$$.

RegNet Design Space

Dive into Deep Learning

In optimal RegNet configurations, the width of the network (number of channels) increases linearly with the block index. This relationship is approximated by the equation $$c_j \approx c_0 + c_a j$$, where $$j$$ represents the block index and $$c_a > 0$$ is the slope. Because the block width can only be changed at the beginning of each stage rather than per block, this continuous linear scaling is practically implemented as a piecewise constant function engineered to match the theoretical linear dependence.

RegNet Channel Width Scaling

RegNetX32 Variant

The fundamental design principles of the RegNet space—such as shared bottleneck ratios, shared group widths, and linear channel scaling—are robust enough to apply beyond standard convolutional blocks. They remain highly effective when applied to Squeeze-and-Excitation (SE) network designs, which utilize a global channel activation mechanism. This specific adaptation of the RegNet principles with SE blocks is known as the RegNetY architecture.

Learn Before

Related

Learn After