如何從可教前綴中學習:多語言推理蒸餾
Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation
Teachable On-Policy Self-Distillation(TOPSD)透過偵測並限制教師漂移,改善多語言數學推理蒸餾,尤其針對低資源語言。TOPSD 根據師生似然選取監督前綴,並驗證教師能否從該前綴得出正確答案,只對通過驗證的前綴進行蒸餾。涵蓋十種語言和三種模型規模的實驗顯示,TOPSD 的表現持續優於傳統 OPSD,尤其在低、中資源語言上。
Published on Nov 5
Authors:
,
,
,
,
,
Abstract
Teachable On-Policy Self-Distillation improves multilingual mathematical reasoning by detecting and limiting teacher drift during student training, especially for lower-resource languages.
On-Policy Self-Distillation (OPSD) improves reasoning by training a student on its own reasoning, using the same model conditioned on reference solutions as a teacher. Applying OPSD to multilingual reasoning requires the teacher to provide useful feedback across languages, even when the student makes mistakes in intermediate steps. We observe a phenomenon we call Teacher Drift: as these mistakes accumulate, the teacher may follow the student's incorrect steps instead of correcting them. Our analysis shows that this shift tends to occur earlier in lower-resource languages. To account for these differences across languages, we propose Teachable On-Policy Self-Distillation (TOPSD), which determines where and whether teacher supervision should be applied to each reasoning trajectory. TOPSD uses teacher-student likelihood to estimate where drift begins and select a prefix for supervision. It then verifies whether the teacher can recover the correct answer from that prefix, applying distillation only to prefixes that pass verification. Experiments on multilingual mathematical reasoning across ten languages and three model scales show that TOPSD consistently outperforms conventional OPSD, particularly in low- and medium-resource languages.
View arXiv page View PDF Add to collection
Get this paper in your agent:
hf papers read 2611.08179
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2611.08179 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2611.08179 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2611.08179 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co