跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 2026-10-14AI 評分42

Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR

Equal CER, Unequal Damage: Severity-Weighted Character Error Rate for OCR

AI 導讀

研究提出 semCER,利用 LLM 評審按嚴重程度為字素編輯加權,令 OCR 和視覺語言模型的評分比標準 CER 更貼近人類評分。semCER 保留 CER 的對齊方式和分母,無需分詞,並將錯誤拆解為具名編輯。研究公開 61 million 個經評審的編輯、彙總讀者評分及蒸餾評分器。

正文

Published on Oct 14

Authors:

Abstract

A new metric called semCER uses an LLM judge to weight grapheme edits by severity, improving alignment with human ratings over standard character error rate for OCR and vision-language models.

Character Error Rate (CER), the standard metric for OCR and vision-language models (VLMs), charges a typo as much as a wrong name. We introduce semCER, which keeps CER's alignment and denominator and charges each grapheme edit a severity from an LLM judge, with no word segmentation. Normalized CER keeps the score of a whole transcription, where it tracks readers as well as semCER or better, and semCER breaks that score down into named edits. The charges track readers' ratings edit by edit in English, Vietnamese, and Chinese and put the edits readers rate most severe near the top of a sorted list. On pairs drawn where CER and semCER disagreed, readers side with semCER more often than with normalized CER in English and Chinese, and equally often in Vietnamese. Across 16 languages, the judge charges a wrong diacritic or tone mark almost nothing in Spanish but nearly a wrong letter's severity in Telugu. Where we can count real words, a general VLM's wrong letter forms one far above chance while a classical pipeline's stays near it. We release the 61 million judged edits, the aggregated reader ratings, and the distilled scorer at https://github.com/nguyenx2t/semcer.

View arXiv page View PDF GitHub 11 Add to collection

Models citing this paper 1

nguyenx2t/semcer-qwen3.5-0.8b

0.8B •

Updated 38 minutes ago

•

19

Datasets citing this paper 1

nguyenx2t/semcer-judged-edits

Viewer •

Updated 38 minutes ago

•

63.5M

•

110

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.20461 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co