跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 16 天前AI 評分46

MACaQ:面向低精度權重與激活值的快速、模型無關校準量化

MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations

AI 導讀

MACaQ 是一種 MXFP4 W4A4 後訓練量化方法,透過逐層獨立量化及 Hessian 感知修正,降低成本並保留語言與擴散模型的準確度。在 Qwen3-32B 上,透過 vLLM 推理的端到端額外開銷僅為 2.3%;在 Llama-3-8B 上,量化時間減少 11 倍,困惑度已達 DuQuant++ 和 MR-GPTQ 的最佳水平。

正文

Published on Nov 7

Authors:

,

,

,

,

Abstract

MACaQ is an MXFP4 W4A4 post-training quantization method that quantizes each linear layer independently using online activation transforms and Hessian-aware refinement, reducing cost while preserving accuracy across language and diffusion models.

Weight-activation quantization can reduce the memory and compute cost of neural-network inference, but recent methods often require expensive optimization and architecture-specific propagation of basis transforms across layers. We introduce MACaQ, an MXFP4 W4A4 post-training quantization method that treats every linear layer independently. MACaQ applies a per-channel scale and a block-Hadamard transform online to each layer's activations, while absorbing the corresponding inverse transforms into its weights. Retaining these transforms online avoids cross-layer dependencies and enables a simple streaming implementation of the quantization process, in which only one layer must reside in memory at a time. To improve over round-to-nearest quantization, we introduce Flip-Refinement, a Hessian-aware coordinate-descent procedure that greedily moves individual FP4 codes to adjacent representable values. Despite executing the activation transforms at inference time, MACaQ adds only 2.3% end-to-end overhead over pure MXFP4 on Qwen3-32B in vLLM. MACaQ reaches lower perplexity than DuQuant++ and MR-GPTQ, and already matches their best perplexity on Llama-3-8B in 11 times less quantization time. We further apply MACaQ without architecture-specific modifications to an image diffusion model, illustrating the generality of its layer-local design and show quantization of a 122B parameter model in 64 minutes, using only 32.5GB GPU RAM.

View arXiv page View PDF Project page GitHub 3 Add to collection

Get this paper in your agent:

hf papers read 2611.10649

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.10649 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.10649 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.10649 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co