Error Analysis of Deep Learning

Abstract

This review examines deep learning theory through the lens of error decomposition, systematically addressing approximation, statistical, and optimization errors. We trace the field's methodological evolution from traditional partial error analyses to emerging unified frameworks that integrate all three algorithmic errors. By utilizing nonparametric regression and the Deep Ritz Method as dual running examples, we demonstrate how this theoretical framework operationalizes across fundamentally different loss structures. Finally, we use this decomposition template to evaluate the theoretical maturity of front-line applications, such as scientific machine learning, generative learning, reinforcement learning, representation learning, and large language models, highlighting both what has been achieved and where complete three-error analyses remain open. This unified approach bridges the historical gap between statistical learning and optimization theories, offering a clear roadmap for future theoretical development.