跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 28 天前AI 評分39

GPT-4o mini 指令容量:可控負載下語言模型的行為與退化

Instruction Capacity: Behavior and Degradation of Language Models under Controlled Load

AI 導讀

研究以無語義的逐詞替換規則作為負載,測試 GPT-4o mini 的指令容量及在可控負載下的退化,考察文本長度、答案中的詞語位置、替換密度、規則數量和規則排序位置。規則數量與品質呈對數線性關係,替換密度符合飽和定律,規則位置符合冪律;文本長度與答案中的詞語位置則互相制約。研究據此建立模型「護照」,其中的數值均可根據已發布數據重新計算。

正文

IEC studies the behavior and degradation of a language model under controlled load.

The load is set by the simplest independent instructions: rules that replace one word with another
in a text without meaning. A program checks every replacement, and a statistical model recovers how
the quality depends on each kind of load — the text length, the word position in the answer, the
replacement density, the number of rules and the position of a rule in the list.

On GPT-4o mini each of these dependences proved regular: linear in the logarithm for the number of
rules, a saturation law for the replacement density and a power law for the position of a rule. The
text length and the word position act as two interacting constraints, and the collapse of the whole
answer obeys its own laws, determined by the load of the text and the progress through the answer
rather than by the number of instructions. The laws and the capacities derived from them form the
passport of the model.

Every number of the passport can be recomputed from the published data. Questions and criticism are welcome.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co