跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 20 天前AI 評分43

P²-MOPD:以參數先驗加速多教師 on-policy 蒸餾

P^2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors

AI 導讀

P²-MOPD 將凍結專家模型的 task vectors 作為可學習參數先驗,與學生權重及係數 α 聯合訓練,以加速多教師 on-policy 蒸餾。Qwen3-1.7B 和 4B 實驗中,最終平均恢復率分別達 94.7% 和 94.9%,高於 MOPD 的 80.9% 和 83.1%,並以一半訓練步數超越 MOPD 的最終表現。

正文

🔥 Multi-teacher on-policy distillation (MOPD) is now a standard way to consolidate domain experts into one model, yet the student still has to climb to every teacher on its own, and some domains are left behind.

What if the experts' weights could guide the student, not only their logits?

P²-MOPD writes every student module as its own learnable weights plus a weighted sum of the frozen expert task vectors (expert minus base model), and trains the weights and the mixing coefficients α jointly under the same MOPD loss. The experts become parameter priors: one learnable scalar per module moves the student along a whole expert direction, while the student weights remain free to go beyond any merge. The two channels, student weights and coefficients, share one backward pass, α is folded into the weights, and the trained model is a standard checkpoint with no change at inference.

Key results (Qwen3-1.7B / 4B; medicine, law, finance, instruction following):

  • Half the steps: P²-MOPD surpasses the final average recovery of MOPD after half the training steps, at ≤0.55× of its wall-clock.
  • Higher final recovery: 94.7% vs 80.9% (1.7B) and 94.9% vs 83.1% (4B).
  • Balanced recovery: worst-domain recovery rises from 32% / 47% to 88% / 89%, and the across-domain std falls from 28 / 22 to about 5.
  • A learned, reproducible mixture: across 3 training seeds the mean coefficient of every teacher agrees to within 0.01, and the mixture consistently weights the lagging teacher the most.

Why it works:
MOPD's gap is one of reachability, not of cross-domain conflict. Student weight-only steps barely move along the task vector of the domain they fail to recover, while P²-MOPD moves 13–15× farther along it through a single scalar per module; random directions in place of the task vectors give no gain, so what matters is the expert direction itself. The coefficients also turn cross-domain interference into a contest over a few interpretable scalars, which the gradient settles in favour of the lagging domain. Neither channel suffices alone: coefficients only (a learned merge) stay far below, weights only (MOPD) leave a domain behind, and the combination (P²-MOPD) gives both the best average and the best worst-domain recovery.

In one line: expert weights, not only expert predictions, can supervise distillation. Kept as learnable priors throughout training, they guide where the student moves.

💻 Code: https://github.com/SinuoWang/P2-MOPD
We welcome feedback and discussion!

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co