How to train your slop cannon: A short guide to using large language models for research in the natural sciences

Abstract

Large Language Models are transforming mathematics and the physical sciences, allowing scientists to explore ideas quickly and raise their sights to grand challenges. But they also produce plausible but wrong text at industrial scale. Staying afloat in this torrent of slop requires scrutinising everything the model produces. We describe a number of workflows that improve and verify model output. These include using adversarial verification protocols, in tandem with Lamport's structured proof format. This exposes logical dependencies, and, by refining claims into sub-claims, spans a spectrum of rigour, from prose to formal verification. Because an LLM performs best when its input only contains what its next step needs, we use wikis and notebooks that the model reads selectively. We try to avoid techniques which merely patch flaws in current models, instead aiming to amplify LLMs as they improve. We explain enough of the architecture so users can diagnose common failure modes such as context rot, hyperfixation, and sycophancy. This guide is intended for scientists whose experience of large language models is the chat window, which conceals many of the levers that can make LLMs more reliable. The goal is to make LLM output more open to inspection, without losing sight of AI's broader dangers. Source (LaTeX), PDF and figure are maintained at https://github.com/Open-Science-Ledger/how-to-train-your-slop-cannon.