Two Surfaces of Ambiguity: Complementary Detection for Text-to-SQL

Abstract

Natural-language interfaces to relational databases increasingly rely on LLMs, but real-world queries are routinely ambiguous, admitting multiple valid SQL interpretations. In interactive deployment, the system should detect this ambiguity and ask for clarification rather than commit to a plausible but wrong query. Two black-box strategies are typically treated as alternatives: prompting the model to flag ambiguous terms in the question (introspection), or sampling candidate SQLs and using their disagreement as an ambiguity signal (sampling-based detection). We show that they are complementary but not disjoint on BIRD-Interact-Lite, a Text-to-SQL benchmark with 300 tasks, annotated ambiguities, and an LLM-based user simulator. Introspection catches what is visible in the prompt: vague terms, missing formulas, and unclear schema references. Sampling, by contrast, exposes choices the model silently commits to during SQL generation, such as null handling and join type. A union of both strategies' clarification questions raises recall by at least 6 pp over either method across three open-weight models, and by 4.8 pp on one proprietary model, using only three LLM calls. In the downstream agentic setting, the same union improves normalized reward across all three open-weight models; on the non-interactive AMBROSIA benchmark, it also improves coverage of valid SQL interpretations. As a byproduct, we trace an apparent recall gap on masked-knowledge ambiguities to the single-label limitation in BIRD-Interact's user-simulator encoder.