OPSD Self-Distillation

What to do without a stronger teacher: one model plays both teacher and student – the difference is only in context. The teacher sees privileged information (ground truth answer, documentation) and “explains the solution” (easier than solving it oneself), outputting a word-for-word target distribution; the student only sees the task and pulls towards the teacher on its own trajectories. Sources of privileged information are broader than verifiable rewards. The limit: the ceiling is the ability of the student himself; if the model cannot clearly explain the solution even with the answer in hand – there is no signal.

Related: On-Policy Distillation, [RLVR and verifiable rewards]