A Dual-Stage Metaheuristic-Adam Optimization Framework for Structural LSTM Pruning: Application to Solar Energy Forecasting

Abstract

Long short-term memory (LSTM) networks, a specialized class of Recurrent Neural Networks (RNNs), excel at capturing temporal dependencies in time-series data but face significant performance trade-offs due to manual architectural design and parameter redundancy. This study introduces a dual-stage architecture optimization framework designed to enhance computational efficiency and structural compactness in LSTMs. The proposed framework combines metaheuristic search techniques specifically, Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Simulated Annealing (SA) with Hadamard-based binary mask pruning during candidate search, which is followed by physical sub-network extraction for deployment. Coupled with the Adam optimizer for weight updates, the methodology is validated through an empirical application to solar energy forecasting using real-world photovoltaic plant data. Experimental results demonstrate that the proposed framework physically prunes active network capacity down to 23.6–28.3% (reducing the parameter count from 1051 K to 248 K and FLOPs from 2.10 to 0.49 MFLOPs) while attaining superior predictive precision (R2=0.979, MAE =116.04 kW over 30 independent seeds) and sub-millisecond inference latencies (≤0.92 ms). By outperforming standard baselines and modern time-series architectures, the framework establishes a highly efficient solution for real-time edge sequential prediction in renewable energy systems.