Transfer Learning: Choose What Learns
Freeze a real ResNet, choose the fine-tuning depth and see every trainable parameter. Turn your choices into PyTorch code.
How much of a pretrained network should your task be allowed to change?
A pretrained feature extractor can support a new classifier. Fine-tuning allows selected weights to adapt too. Trainable parameter counts, BatchNorm behavior and validation evidence help you describe and evaluate the configuration.
Make a prediction.
Then move a control.
Keep the knowledge.
Choose what can change.
Freeze and unfreeze a real ResNet, then count exactly what your optimizer can update.
The same forward path. A different update plan.
Choose which parameter groups can learn.
Every stage processes the image. Only selected groups can update their parameters.
BatchNorm running statistics stay fixed in this configuration.
Inspect the parameter accounting
Convolution weights + BatchNorm affine parameters + projection shortcuts + classifier. Running statistics are excluded. Torchvision ResNets use no convolution biases and include a classifier bias.
| Group | Main conv weights | Main BN parameters | Shortcut parameters | Total | Trainable |
|---|
Your configuration in PyTorch
Architecture counts are exact for the selected torchvision ResNet. The forward trace illustrates processing; it does not run image inference.
The idea,
in Python.
A compact implementation of the core concept. Open the code when you are ready to connect the visual to the algorithm.
Open the core implementation Python ↗
import torch
from torch import nn
from torchvision import models
# Downloads ImageNet weights on the first run.
model = models.resnet18(weights=models.ResNet18_Weights.DEFAULT)
model.fc = nn.Linear(model.fc.in_features, 3)
trainable_groups = ["fc"]
for name, parameter in model.named_parameters():
group = "stem" if name.split(".")[0] in {"conv1", "bn1"} else name.split(".")[0]
parameter.requires_grad = group in trainable_groups
def set_training_mode():
# Call at the START of EVERY training epoch.
model.train()
# Freeze BN buffers separately from affine weight/bias parameters.
for module in model.modules():
if isinstance(module, nn.modules.batchnorm._BatchNorm):
module.eval()
set_training_mode()
parameters = [p for p in model.parameters() if p.requires_grad]
optimizer = torch.optim.AdamW(parameters, lr=1e-4, weight_decay=1e-2)
print("Total:", sum(p.numel() for p in model.parameters()))
print("Trainable:", sum(p.numel() for p in parameters))
# Use model.eval() and torch.inference_mode() for validation/inference.
# Freezing weights does not remove their forward computation.Explain the change.
Test your reasoning.
- Start with a three-class ResNet-18 head. Explain why only 1,539 parameters are trainable.
- Unfreeze Stage 4. Why does one stage account for roughly 75% of the adapted model?
- Create two configurations with the same trainable groups but different BatchNorm statistics behavior. Explain what changes, then design a validation comparison.