Learn Before
ResNet Function Decomposition
ResNet decomposes a target function into a simple linear term and a more complex nonlinear one, mathematically expressed as . This approach is conceptually similar to a Taylor expansion, which decomposes a function into terms of increasingly higher order at a given point (e.g., f(x) = f(0) + x \cdot \left[f'(0) + x \cdot \left[\frac{f''(0)}{2!} + \cdots ight] ight]). By isolating the identity mapping (), ResNet allows the neural network to focus on learning the more complex nonlinear residual ().
0
1
Tags
D2L
Dive into Deep Learning @ D2L
Related
Inductive Bias of Residual Connections
ResNet Function Decomposition
Residual Connections Enable Deeper ResNet Training
As plain networks grow deeper, their training accuracy can paradoxically worsen, a phenomenon known as the ___ problem.
Order the steps carried out during a forward pass in a residual block to compute the target function f(x) from an input x.
Explain why reformulating the layers to learn the residual mapping g(x) makes approximating the identity function f(x) = x easier for the network compared to learning f(x) directly.
Match each concept from residual learning to its corresponding description.