Estimating the Number of Components in Finite Mixture Models via Variational Approximation

Abstract

This work introduces a new method for selecting the number of components in finite mixture models (FMMs) using variational Bayes, inspired by the large-sample properties of the Evidence Lower Bound (ELBO) derived from mean-field (MF) variational approximation. Specifically, we establish matching upper and lower bounds for the ELBO without assuming conjugate priors, suggesting the consistency of model selection for FMMs based on maximizing the ELBO. As a by-product, we show that the MF approximation inherits the stable behavior of the posterior distribution, which benefits from model singularity and tends to eliminate the extra components under model over-specification. This stable behavior also leads to the n−1/2 convergence rate for parameter estimation, up to a logarithmic factor, under model over-specification. Empirical experiments are conducted to validate our theoretical findings and compare with other advanced methods for selecting the number of components in FMMs.