miniTE:診斷並彌合原生 NVFP4 強化學習的執行差距
miniTE: Diagnosing and Bridging the Execution Gap in Native NVFP4 Reinforcement Learning
miniTE 開源框架針對原生 NVFP4(W4A4)強化學習,診斷並縮小 learner 與 rollout engine 的數值執行差異,旨在提升訓練穩定性與效率。
🚀 Can we really train LLMs with native FP4 reinforcement learning — without sacrificing stability?
Excited to share miniTE, our open-source framework for stable and efficient native NVFP4 (W4A4) RL!
One surprising finding: Same weights + same precision ≠ same policy. Even when both the learner and rollout engine use NVFP4, subtle differences in numerical execution can introduce significant training–inference mismatch and eventually destabilize RL.
With miniTE, we tackle this challenge from both numerical and systems perspectives:
🔬 Diagnose & align: Identify cross-engine execution differences and bring training–inference log-probability gaps down to ~1e-8 in our strict-alignment setting.
⚡ Make FP4 actually fast: Optimize native NVFP4 kernels, scaling, and grouped GEMMs, achieving up to 1.39× BF16 learner throughput on 8×B300 GPUs.
📈 Keep learning stable: Demonstrate long-horizon RL on Qwen3-30B-A3B with downstream performance close to BF16.
Beyond the results, we're releasing the framework, kernels, diagnostics, and training recipes to make native low-bit RL easier to reproduce and extend.
FP4 is not just about using fewer bits — it's about making every bit work. 🔥
📄 Paper: https://arxiv.org/abs/2610.23928
💻 Details: https://infix.io/research/miniTE
We'd love to hear your thoughts, feedback, and ideas for pushing low-precision RL further!
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co