Subspace Langevin Monte Carlo

Abstract

Abstract. Sampling from high-dimensional distributions has wide applications in data science and machine learning but poses significant computational challenges. We introduce subspace Langevin Monte Carlo (SLMC), a novel and efficient sampling method that generalizes random-coordinate LMC and preconditioned LMC by projecting the Langevin update onto subsampled eigenblocks of a time-varying preconditioner at each iteration. The advantage of SLMC is its superior adaptability and computational efficiency compared to traditional LMC and preconditioned LMC. Using coupling arguments, we establish error guarantees for SLMC and demonstrate its practical effectiveness through a few experiments on sampling from ill-conditioned distributions.