Compressing Exponential Weights for Online Logistic Regression: Exact Identities, Update-Closed Surrogates, and a Geometric Bottleneck

Abstract

Near-optimal regret for online logistic regression is known, yet the most direct exponential-weights (EW) predictors remain computationally expensive precisely in the regimes where their statistical guarantees are most attractive. We ask a narrower question than whether logarithmic regret is possible: can the EW predictor itself be represented and evaluated from an update-closed state whose dependence on the horizon is lightweight? The answer is partly positive and partly geometric. We derive exact posterior-score identities that recover repeated-direction EW predictions from posterior means and counts; show why several natural Gaussian, projection, and fixed-moment compressions fail; and exploit the decomposition of logistic loss into a linear label term plus an even log-cosh term to obtain an additive, update-closed polynomial-tensor state. We establish a bounded-domain Bayesian-mixture regret bound, an approximation-to-log-loss stability bridge, exact complex-curvature control for the logistic loss, and a curvature-height localization inequality. These ingredients yield a practical one-dimensional adaptive quadrature implementation whose prediction error remains below 5.06 × 10⁻⁵ across twenty stress cases through T = 2048, with absolute cumulative loss tax below 2.14 × 10⁻⁴ and at most 64 quadrature nodes in the tested regime. In higher dimensions we isolate a precise Hessian-metric volume condition sufficient for polylogarithmic fixed-dimensional integration, prove structured cases, and identify unrestricted directional packing as the remaining obstacle. The result is a reproducible reduction from posterior computation to a concrete geometric question rather than a claim that the general open problem is solved.