Tag: jsd

All the articles with the tag "jsd".

• Notes on: On-Policy Distillation of Language Models (GKD)

  6 Sep, 2026

  The actual GKD objective — a λ-weighted mix of fixed-dataset and student-sampled sequences, a tunable JSD(β) divergence family, and where each choice wins across T5-family experiments.