論文提出用於後訓練的校準式語言適應配對,讓模型配合用戶語言變體
Answering in the User's Variety: Calibrated Accommodation Pairs for Post-Training
研究提出校準式適應配對(CAP),以偏好資料獎勵模型配合用戶的方言特徵密度,同時讓標準英語用戶維持標準英語回覆。研究追蹤 10 個開放模型系列,發現後訓練會令回覆趨向標準化;調整候選回覆原有可能性後,DPO 在 10 個系列中仍較偏好英式及澳洲拼字,而非 AAE 和 Singlish。
Published on Oct 20
Authors:
,
,
,
,
,
Abstract
People adapt how they talk to the person in front of them. However, chat models tend to respond in Standard English regardless of the dialect used by the user. A uniform answer can hide whether training treats these users alike, and output audits cannot tell as well. We trace how each post-training stage, from supervised fine-tuning (SFT) to preference optimization (DPO) and reinforcement learning, changes a model's tendency to match the user, and read what each update prefers from two content-matched replies, one matching the user and one not. Across 10 open model lineages, post-training standardizes replies to every informal cue, yet after adjusting for how likely each reply already was, the DPO update still favours matching British and Australian spelling over AAE and Singlish on 10 lineages, and models still produce AAE features when asked. We then introduce calibrated accommodation pairs (CAP), preference data that reward matching users' dialect density while preserving Standard English for Standard-English users. On a fully open model, CAP raises convergence on features it was not trained on without a detectable increase in dialect for Standard-English users or a cost in instruction following. In contrast, one-sided pairs reach a similar gain only by overshooting the user and imposing dialect. Our findings show that uniform replies can hide selective updates and audits should read the update as well as the output. Code: https://github.com/kouzhizhuo/cap-accommodation.
View arXiv page View PDF GitHub Add to collection
Get this paper in your agent:
hf papers read 2610.28474
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2610.28474 in a model README.md to link it from this page.
Datasets citing this paper 1
alan1027/cap-accommodation
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2610.28474 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.
來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co