Feature-based knowledge is a knowledge distillation approach that uses the feature maps produced by a teacher model's intermediate layers, not just its final output, as knowledge to transfer to a student model. It builds on the concept of representation learning, since deep networks learn multiple levels of increasingly abstracted feature representations, and it is especially suitable for training thinner and deeper student networks.