The Structured Latent Basis: Feature Engineering as Basis Selection

Abstract

We introduce the Structured Latent Basis (SLB) framework, a perspective that compares feature engineering, meta-learning, and representation learning through a common diagnostic question: does the chosen representation make a regularized linear predictor sufficient under a declared evaluation protocol? We make two formal observations and define an empirical program: 1. Conditional optimality: For a fixed feature matrix and \alpha>0, the regularized linear objective is strictly convex and Ridge gives its unique minimizer. 2. Approximation: An analytic cosine-decay component plus finitely many jumps admits a quantitative cosine-plus-sigmoid approximation bound. 3. Diagnostic protocol: Compare a regularized linear probe with specified nonlinear baselines after all learned preprocessing is fit inside the training fold. The current text implementation establishes the fold-local Ridge path and valid-token feature extraction; nonlinear baselines remain part of the prospective protocol, and the required embedding caches must be regenerated before numerical claims are reinstated. The unifying insight is narrow: feature construction and model fitting should be evaluated jointly. The basis can be hand-designed, inherited from pre-training, or learned across tasks. A companion paper studies learned feature maps with closed-form Ridge adaptation and explicitly relates that mechanism to R2-D2. The present paper does not claim that a suitable basis always exists at practical dimension or that Ridge is globally optimal among model classes. Keywords: feature engineering, basis selection, meta-learning, spectral methods, Ridge regression, few-shot learning, representation learning Maturity: Working Paper. Target venue: Zenodo preprint; journal venue to be determined. Part of The Latent research program. Related papers in this program: Universal.