跳到正文
HuggingFace Daily Papers(社區熱門論文)·· 10 天前AI 評分53

LuthorSQL 提出單次前向傳播 Text-to-SQL 模型,為較慢 LLM Agent 擔任 System 1

LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent

AI 導讀

研究提出 LuthorSQL,一款非自回歸 7B Text-to-SQL 模型,作為較慢 LLM Agent 的快速 System 1 閘門,只在不確定時才把問題升級給 Agent。

正文

Published on Nov 13

Authors:

,

,

Abstract

Frontier LLM agents that query databases with tools are among the most accurate text-to-SQL systems, but per-query calls to a hosted model are slow, costly, and expose sensitive organizational queries to external APIs. For example, a Claude Opus 5.5 agent serving 10,000 queries a day costs over $70,000 a year at list price. To what extent can local hardware already owned by an organization offload this burden without compromising the agent's accuracy? We present LuthorSQL, a non-autoregressive 7B text-to-SQL model that serves as a rapid System 1 gate for a slower LLM agent (System 2), escalating queries only when uncertain. LuthorSQL makes a single forward pass to predict slot scores over a bounded grammar for query plans, compiles and runs up to 32 candidate plans, and chooses an answer using a trained validator. On its own, LuthorSQL reaches 68.03% execution accuracy on BIRD's hidden test set at 0.84 s per question on a single GPU, maintains its accuracy in 9 GB with 8-bit quantized weights, and transfers zero-shot to Snowflake, BigQuery, MySQL, PostgreSQL, and DuckDB within 3.1 points of its native SQLite accuracy. Combined with Claude Opus 5.5 in a pre-registered confidence cascade over 227 questions on databases published after Opus's training cutoff, LuthorSQL answers nearly half of all questions locally and cuts API cost by 33.3% at the same accuracy. Finally, contamination probing shows that Opus reproduces 48-92% of public benchmark gold queries verbatim without seeing the schema; however, its tool-assisted accuracy holds on renamed schemas and reaches 95.6-97.4% on post-cutoff databases. Code and data: https://github.com/IAMIbrahimmemon/luthorsql

View arXiv page View PDF GitHub 0 Add to collection

Models citing this paper 1

IAMIbrahim/luthorsql-bird

Updated about 6 hours ago

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.17780 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.17780 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.

來源:HuggingFace Daily Papers(社區熱門論文) · huggingface.co