Bayesian Neural Networks versus deep ensembles for uncertainty quantification in machine learning interatomic potentials

Abstract

Abstract Neural-network-based machine learning interatomic potentials have emerged as powerful tools for predicting atomic energies and forces, enabling accurate and efficient simulations in atomistic modeling. A key limitation of traditional deep learning approaches, however, is their inability to provide reliable estimates of predictive uncertainty. Such uncertainty quantification is critical for assessing model reliability, especially in materials science, where often the model is applied on out-of-distribution data. Different strategies have been proposed to address this challenge, with deep ensembles (DE) and Bayesian neural networks (BNN) being among the most widely used. In this work, we introduce an implementation of Bayesian neural networks with variational inference in the \texttt{\ae net-PyTorch} framework. To evaluate their applicability to machine learning interatomic potentials, we systematically compare the performance of variational BNNs and deep ensembles on a dataset of 7,815 TiO\textsubscript{2} structures, and further assess the generalisability of our conclusions on the chemically more diverse QM7 dataset of 7,165 small organic molecules. The models are trained on both a full dataset and a smaller subset to assess how variations in data representation influence predictive accuracy and uncertainty estimation. Our findings reveal that, despite its more empirical and less theory-grounded nature and in addition to its lower inference cost and implementation simplicity, Deep Ensembles outperform variational Bayesian Neural Networks on most accuracy and uncertainty-quantification metrics across data regimes, the main exceptions being the calibration error and the energy accuracy on the chemically diverse QM7 dataset, where the BNNs are competitive or better. This analysis provides insights into the strengths and limitations of each approach, offering practical guidance for the development and evaluation of uncertainty-aware machine learning interatomic potentials.