跳到正文
DEV Community· Yoane Ngassam·· 13 天前AI 評分41

「數學表現提升 6 倍」的説法需要 Gemma 對照組

The “6 better at math” claim needs the Gemma control

AI 導讀

在所報告的測試設定中,Qwen3.5-0.8B 使用 50 次前向傳播後,MATH-500 評分由 7.9% 升至 48.1%,答案觸及 token 上限的比例則由 93.0% 降至 33.1%。Gemma 3 270M 的評分基準為 6.8%,使用 25 次及 50 次前向傳播後分別為 7.7% 和 7.0%,增幅尚未確立。結果顯示,解除生成瓶頸不等於證明數學能力有所提升。

正文

Yoane Ngassam

POIDS, episode 2 of 6. French video series; English technical notes.
The Qwen benchmark jump is real within the reported protocol, but the control matters. On Qwen3.5-0.8B, answers hitting the token limit fall from 93.0% to 33.1%, while MATH-500 rises from 7.9% to 48.1% at a 50-forward-pass budget.
For Gemma 3 270M, the table reports 6.8% at baseline, 7.7% with 25 forward passes, and 7.0% with 50. The gain is described as not established. The 7.7% figure in the video belongs to the 25-pass setting.
The result supports a narrow interpretation: removing a generation bottleneck is different from demonstrating a new mathematical capability. It does not establish that weight editing can never improve capabilities. Different models and sampling protocols also limit direct comparisons.

References and scope

Sparks of Weight-Space Programming — primary author report. MIT CSAIL, November 2, 2026. These are preliminary reported results, not a guarantee of generalization.

Watch and follow

French video series on YouTube · Episode 1
Video images and voice are AI-generated.

來源:DEV Community · dev.to