Model Distillation

A form of knowledge distillation in which a compact model is trained to reproduce the behavior of a larger, more capable model.