ResNet: Inside the Residual Block
Open the stages of ResNet-18, 34 and 50. Follow tensor shapes, compare shortcuts and make the residual addition visible.
What does a shortcut actually add to a deep network?
A residual branch computes F(x), while the shortcut supplies x or a learned projection. Matching shapes permit elementwise addition. Torchvision’s ResNet blocks apply ReLU after the addition, and stages progressively change channel counts and spatial dimensions.
Make a prediction.
Then move a control.
Follow the features.
Find the shortcut.
Travel from an RGB image to a prediction, then open a residual block to see its two paths.
Stage 1 · Block 1
Select a block. First blocks may change channels or spatial size.
| Main convolutions | Main BatchNorm | Projection + BN | Total |
|---|---|---|---|
A tiny numerical example
One scalar, an identity shortcut and ReLU. This illustrates the addition; it does not simulate convolution.
Why the depth is called 18, 34 or 50
The conventional depth counts the stem convolution, the main-branch convolutions in each residual block, and the final fully connected layer. BatchNorm, activation, pooling and projection-shortcut layers are not included in that naming convention.
Residual connections make identity-like mappings easier to represent and can improve optimization of deep networks. They do not guarantee nonzero gradients or better results on every dataset.
Shapes and parameter counts follow torchvision. The scalar addition is an explicitly simplified teaching example.
The idea,
in Python.
A compact implementation of the core concept. Open the code when you are ready to connect the visual to the algorithm.
Open the core implementation Python ↗
import torch
from torchvision import models
# weights=None: inspect the architecture without downloading weights.
model = models.resnet18(weights=None).eval()
x = torch.randn(1, 3, 224, 224)
def inspect(name):
def hook(module, inputs, output):
print(name, tuple(output.shape))
return hook
handles = [getattr(model, name).register_forward_hook(inspect(name))
for name in ["maxpool", "layer1", "layer2", "layer3", "layer4", "avgpool", "fc"]]
with torch.inference_mode():
logits = model(x)
for handle in handles:
handle.remove()
print("Total parameters:", sum(p.numel() for p in model.parameters()))
# A basic identity block returns ReLU(F(x) + x).
# A shape-changing block uses ReLU(F(x) + projection(x)).
# For ResNet-50, torchvision uses bottlenecks (1x1, 3x3, 1x1).Explain the change.
Test your reasoning.
- Find an identity shortcut and a projection shortcut. Explain why only one needs learned parameters.
- Compare ResNet-18 with ResNet-50. Where do bottlenecks change the structure?
- Double the input width and height. What changes in the feature maps, and why does the parameter count stay the same?