Non-asymptotic oracle inequalities for the Lasso in high-dimensional mixture of experts

Abstract

Abstract We study Lasso estimation for high-dimensional softmax-gated Gaussian mixture-of-experts (SGMoE) regression models, where the predictor dimension P may exceed the sample size N . Under bounded parameters, we establish a non-asymptotic Kullback–Leibler (KL) oracle inequality with an explicit sufficient regularization level and explicit dependence on N , P , response dimension Q , number of experts K , and boundedness constants. We derive sparse well-specified and approximation-space rates, showing how deterministic SGMoE approximation error transfers into statistical KL-risk bounds. The theory extends to random covariates and, under misspecification, controls the best Gaussian SGMoE approximation in the candidate class. Reproducible experiments validate KL decay and density recovery, compare Lasso with SCAD when $$P>N$$ P > N , evaluate model-selection criteria, handle multivariate covariance structures, and assess high-dimensional biological benchmarks relevant to heterogeneous genotype–phenotype mapping, omics-to-trait prediction, and AI-enabled virtual-cell modelling in systems biology.