跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 28 天前AI 評分36

超越估計:Wiener-informed Decoding 讓像素空間 Flow 模型更易訓練

Beyond the Estimate: Wiener-Informed Decoding for Easier Training of Pixel-Space Flow Models

AI 導讀

Wiener-informed Decoding(WiD)將維納統計估計融入像素解碼器,透過統計量初始化新路徑,並與模型一同訓練,以改善像素空間 Flow 模型的學習。WiD 適用於類別條件及文字生成圖像,並支援不同解像度;統計量以約 50,000 張圖像估算,使用 GPU 加速約需一分鐘。

正文

Reading papers like JiT and PixelDiT got me thinking about why learning velocity directly in pixel space is so difficult. Some approaches tackle this by choosing an easier prediction target, such as x-prediction; others design architectures that separate semantic modeling from dense pixel prediction.

At first, my question was quite simple: what if we could first make a coarse estimate of the velocity from the noisy sample, and then let the model learn the remaining correction? Could that make the learning problem easier?

Inspired by Wiener filtering, we can construct an analytic estimator that gives us such a velocity prediction using the second-order statistics from data. In our experiments, we observed that simply adding this fixed prediction and letting the model learn the residual already led to improvements. Its benefits were similar to those of pixel decoders, but the two were not entirely redundant.

A frozen statistical estimator has limited flexibility, though. And when added to an architecture that already has a pixel decoder, its extra benefits tend to diminish as training progresses. So I wondered: why not make the estimator part of the architecture itself?

The pixel decoder turns out to provide a natural interface for this. Rather than using only the estimator’s prediction, we also use the way it organizes computation. We introduce modules and pathways within the decoder that interact with its existing structure, with the new parameters trained jointly with the model, and use the statistics to initialize these pathways.

We call this Wiener-informed Decoding (WiD). We’ve found it effective for both class-conditional and text-to-image generation, and across different resolutions. In our experiments, we estimate the statistics from around 50,000 images, which takes about only one minute with GPU acceleration. The method also adds relatively few parameters.

Statistical denoisers have a long history, but I think they are still able to inspire us in modern denoising models. In this first step of our exploration, we’ve demonstrated the idea in architectures with pixel decoders. In the future, we’d like to explore how it might apply to other architectures and generation tasks.

3

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co