跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 17 天前AI 評分45

AutoCompress:硬件引導的模型壓縮

AutoCompress: Automated Hardware-Guided Model Compression

AI 導讀

AutoCompress 透過硬件引導的可微分架構搜尋及服務引擎成本模型壓縮 LLM,生成符合延遲、吞吐量和記憶體限制的異質架構,同時保留準確度。與 Minitron 相比,它在八項基準測試的平均準確度,在 RTX 5070、RTX 5080 和 H100 上分別提升 6.57、2.35 和 0.57 個百分點。

正文

Published on Nov 5

Authors:

,

,

,

,

,

Abstract

AutoCompress is a hardware-guided compression framework that uses differentiable architecture search and serving-engine cost models to produce heterogeneous LLM architectures meeting latency, throughput, and memory constraints while preserving accuracy.

LLM pruning must preserve quality while meeting hardware- and workload-specific constraints on latency, throughput, and memory. Existing pruning workflows often target parameter counts, requiring practitioners to manually select model sizes that meet latency, throughput, and GPU memory limits. However, parameter count alone does not predict these metrics, which depend on the model architecture, workload, hardware, and serving engine. We introduce AutoCompress, a hardware-guided compression framework that directly incorporates such constraints into differentiable architecture search. AutoCompress jointly optimizes global and per-layer structure to produce heterogeneous architectures, guided by serving-engine cost models validated against end-to-end latency, throughput, and memory measurements. We evaluate AutoCompress on long-context local coding, batched datacenter serving, and latency-sensitive in-game interaction. Compared to the state-of-the-art Minitron pruning framework, AutoCompress improves average accuracy across eight benchmarks by 6.57, 2.35, and 0.57 percentage points on RTX 5070, RTX 5080, and H100, respectively. AutoCompress also compresses Nemotron 3.5 Super from 120B to 74B parameters for single-B200 BF16 serving at 128K context, retaining 95% of its mean benchmark score.

View arXiv page View PDF Add to collection

Get this paper in your agent:

hf papers read 2611.08290

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.08290 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.08290 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.08290 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co