Public Database Leakage Distorts Model Rankings in Real-Spectra NMR Structure Elucidation

Abstract

Abstract Sequence models that translate NMR spectra into molecular structures report up to 96% top-1 accuracy, but they are pretrained on simulated spectra and tested on public databases that can overlap those corpora. We build a leakage-controlled benchmark on nmrshiftdb2 and find that 26.9% of public molecules are exact training-corpus matches under a stated identity rule, with substantial training-set proximity remaining after exact-match removal. On the wider exact-match-removed real-spectrum cohort, released-model top-1 accuracy is 24.4% (95% CI 23.3–25.6%). A training-free rerank using molecular formula and 13C-peak count consistency raises it to 31.1% (29.9–32.4%), supporting leakage-controlled evaluation and decode-time consistency checks before claims of experimental generalization for simulation-pretrained inverse models.