跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-16AI 評分42

N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models

N-Gram Memories at Two Bits: Quantizing Lookup Tables in Large Language Models

AI 導讀

Conditional Program(CP)以共享重建程序、短碼和局部尺度,按輸入需求重建語言模型嵌入表的資料列,將儲存量減少 75%。CP 的兩位元編碼相較兩位元最接近值取整,令輸出 KL 降低 41--63%。應用於 DeepSeek 時,查找表儲存量由 203GB 減至 51GB。

正文

Published on Oct 16

·

Submitted by

Junhoo

on Oct 20

Authors:

,

,

,

Abstract

Conditional Program compresses language-model embedding tables by reconstructing only requested rows via shared codes and local scales, reducing storage by 75% while improving fidelity.

Conditional memory expands language-model capacity through selective lookup, but the full embedding table must remain available for every possible input. We introduce Conditional Program (CP), a compressed representation that reconstructs only the rows an input requests. CP represents each row by a short code and a local scale, with a reconstruction program shared across the table. We instantiate this representation with a Gaussian trellis, norm restoration and independently decodable row records. CP's two-bit code lowers output KL by 41--63\% relative to 2-bit round-to-nearest. Analysis across three trained n-gram memory families shows that Gaussian-like coordinate statistics coexist with learned relationships among rows; at matched storage budgets, the shared trellis outperforms the fitted codebooks and low-rank residual variants we test. For DeepSeek, it reduces table storage from 203GB to 51GB, leaving room for larger workloads when the table is kept on GPU.

View arXiv page View PDF Add to collection

Get this paper in your agent:

hf papers read 2610.21810

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.21810 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.21810 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.21810 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co