Dictionary-KAN: Resolving the Optimization Paradox of Kolmogorov-Arnold Networks via Complex RKHS, Machine-Verified Theory, and Discrete Hierarchical Refinement

Abstract

Kolmogorov-Arnold Networks (KANs) replace dense weight matrices with learnable univariate edge functions, but continuous spline-based formulations suffer from an “optimization paradox”: vanishing gradients, knot-sorting collapses, dense-grid memory costs, and the reintroduction of the curse of dimensionality via multi-variable gating. We introduce Dictionary-KAN (DKAN), an architecture that grounds every edge function in a fixed magnitude-RBF dictionary with complex coefficients, lifts inputs into the complex plane so that analytic base terms (z, z2) natively isolate multivariate cross-products, grows capacity through Discrete Hierarchical Refinement (DHR) with exact zero catastrophic forgetting, and regularizes weight curvature with a discrete total-variation (TV) penalty. Crucially, the core optimization theory—exact quadratic loss expansion, strict convexity under a full-rank dictionary, stationarity ⇔ normal equations, unique minimizers, gradient-descent convergence under a Polyak– Lojasiewicz condition, the DHR zero-forgetting identity, and the TV curvature characterization—is machine-verified in Lean 4 / Mathlib [13] with explicit assumptions. Empirically, DKAN outperforms dense RBF-KAN baselines by ∼ 20× on coupled multivariate regression, recovers the damped-pendulum ODE and the Burgers PDE coefficients (< 0.3% error) via complex-manifold feature pruning, avoids the out-ofmemory failure of dense KANs when scaling width, and compiles a trained network into a ∼50-line closed-form GLSL BRDF shader. We report negative results with equal prominence:smooth dictionaries exhibit Gibbs-like ringing on discontinuities, and weight-space TV does not control function-space overshoot. DKAN is positioned as an interpretable, refinable, hardware-efficient alternative to both MLPs and continuous KANs.