80.52 短片:為何 65B 模型勝過 501B 模型,仍需要影片佐證
The 80.52 Short: why a 65B beating a 501B needed a video to be believed
一個本地運行的 65B 模型在評測中取得 80.52,分數高於同一輪測試的 501B 模型;作者拍攝操作演示,展示評測過程。兩者回答相同的 200 道選擇題,温度設為 0,但推理路徑不同;65B 模型在 20 個類別中勝出 14 個。重現 501B 模型的 API 測試每輪約需 $12,65B 模型則需約 40 分鐘 GPU 時間及電費。
I spent last week trying to explain that parameter count stopped being the whole story. A 65B model I run locally scored 80.52 on our eval suite; a 501B frontier model scored lower on the same run. The number alone sounds like a typo, so I filmed a walkthrough instead of arguing in comments.
The Short exists because the gap is real but invisible without seeing the harness. The concrete detail I show on screen: both models got the exact same 200-question multiple-choice set, temperature 0, no system-prompt changes. The only difference was inference path — the 65B ran quantized at Q4_K_M on a single consumer GPU while the 501B ran through its hosted API with defaults. Same questions, same scoring script, and the smaller model won 14 of 20 categories.
What it costs the operator: reproducing the 501B side isn't free. Each full run through that API burns about twelve dollars in tokens, so anyone verifying the number has to pay the toll again. The local 65B run costs electricity and roughly forty minutes of GPU time — the trade I actually prefer.
I packaged the walkthrough as https://www.youtube.com/shorts/niEWIgSUopU
來源:DEV Community · dev.to