跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-19AI 評分33

SpecMRL:在參數化頻譜中重新設計 Matryoshka 表徵學習

SpecMRL: Redesign Matryoshka Representation Learning within Parametric Spectrum

AI 導讀

SpecMRL透過按特徵重要性排列的參數導出頻譜基底重組特徵空間,改善壓縮嵌入表現。MTEB實驗顯示,SpecMRL在所有受測的縮減維度下均勝過標準MRL,僅用一半嵌入維度便超越全維度MRL。

正文

Published on Oct 19

Authors:

,

,

,

,

Abstract

SpecMRL improves compressed embeddings by reorganizing features via a learned spectral basis that prioritizes informative directions, outperforming standard MRL at reduced dimensions.

Matryoshka Representation Learning (MRL) is designed to support flexible representation sizes by jointly optimizing nested prefixes. While effective, this design lacks an explicit constraint to ensure that the leading portion provides a suitable compact encoding. To address this, we propose SpecMRL, which reorganizes the feature space to enable more effective prefix readout. Specifically, we extend MRL with a parameter-derived spectral basis organized with feature importance. SpecMRL otherwise follows standard MRL, with prefix readout performed in the learned spectral space. Experiments on MTEB show that SpecMRL outperforms MRL at all evaluated reduced dimensions, with larger gains under more aggressive compression. These results suggest that the learned spectral basis better prioritizes informative directions, enabling more effective representation compression. Notably, SpecMRL extends the performance frontier of MRL, surpassing full-dimensional MRL with only half the embedding dimensions.

View arXiv page View PDF GitHub 1 Add to collection

Get this paper in your agent:

hf papers read 2610.26824

Don't have the latest CLI?

curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.26824 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.26824 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.26824 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co