ByteTop-k OPD: バイト列表現に基づく語彙の異なる LLM 間のオンポリシ蒸留

摘要

On-policy distillation trains a student model on the teacher’s predictions over contexts the student itself generates, but the two next-token distributions cannot be compared directly when teacher and student use different vocabularies. Existing methods restrict the comparison to tokens shared by both vocabularies, or to spans of the generated text that align as strings, so a candidate the student assigns high probability to can fall outside the comparison entirely. This paper proposes ByteTop-k OPD, which represents the student’s top-k candidates as byte strings and maps them onto the token sequences that realize them in the teacher’s vocabulary, yielding a common set of comparison labels.