Knowledge Distillation

Training a smaller “student” model to mimic a larger “teacher” model, transferring capability while reducing size and cost.