PRECISE-LLM: Formal Ambiguity Detection and Interpretation Enumeration for Text-to-SQL

Abstract

Large language models translate natural-language questions into SQL with remarkable fluency, but they fail silently on ambiguous questions: rather than signalling uncertainty, the model commits to one reading and returns a syntactically perfect yet semantically wrong query. We present PRECISE-LLM, a system that detects ambiguity before generation using a purely formal, deterministic and parameter-free detector — three rules over the dependency tree and the database schema, one per ambiguity type — then asks a targeted clarification question and generates one SQL query per reading. On AMBROSIA, the benchmark dedicated to ambiguous text-to-SQL, the detector is a high-recall signal: F1 0.675 on the calibration split and 0.657 on held-out data, with recall 0.731 overall and 0.844/0.918 on the two structural ambiguity types. Evaluated on all 1149 ambiguous questions of the benchmark with a local 7B generator, PRECISE-LLM recovers every gold interpretation of a question in 24.2% of cases, against 0.4% for the same generator deprived of enumeration (×60) and 1.9% for the best prompted LLM reported on the benchmark (×12.7), at identical evaluation scope. A controlled ablation isolates the detection step: with generation held fixed, formal rules reach coverage 0.463 against 0.287 for an LLM-based detector and 0.237 without detection — while requiring no API call and remaining fully auditable. We position formal detection as an explainable, zero-cost complement to probabilistic detectors, not as a universal replacement.