Astronomical spectra as language: Order-agnostic foundation model for low-SNR reconstruction and stellar parameter prediction

Abstract

Large-scale spectroscopic surveys contain numerous low-SNR stellar spectra that are difficult to analyze with traditional pipelines but remain valuable for Galactic archaeology, chemical-evolution studies, and population-level stellar characterization. We propose an order-agnostic foundation model that represents astronomical spectra as discrete tokens and learns intrinsic spectral correlations through arbitrary masking, enabling effective modeling of incomplete and noisy observations. The model is pretrained from scratch on physically motivated synthetic spectra in a self-supervised reconstruction task, and is then fine-tuned on observational LAMOST spectra to predict stellar parameters, with emphasis on metallicity ([Fe/H]). Experiments show that the model reconstructs low-SNR spectra with high fidelity and improves parameter estimation under noisy conditions. In the tested low-SNR setting, it achieves a metallicity prediction error of approximately 0.38 dex; even at extremely low-SNR values of about 1–2, it still provides useful information from spectra that are otherwise difficult to exploit.