Self-distillation methods transfer supervision within a network or across stages of its training process:

• Deep-to-shallow distillation transfers knowledge from deeper network sections to shallower sections.
• Self-attention distillation uses attention maps from later layers as distillation targets for earlier layers; the supplied context identifies lane detection as an application.
• Snapshot distillation uses an earlier training snapshot as the teacher for a later snapshot.
• Self-knowledge distillation matches predicted probabilities so that they reflect similarities in the model's feature-embedding space.