Physically Constrained Temporal Fusion Transformer with LLM-Assisted Hyperparameter Optimization for Multi-Scenario Energy Time Series Forecasting

Abstract

Accurate forecasting of battery state of health (SOH), wind power, and photovoltaic (PV) power is challenged by non-stationarity, multiscale dependencies, and physically implausible outputs. This study proposes a physically constrained Temporal Fusion Transformer (PC-TFT) that coordinates enhanced temporal representation, physical regularization, and LLM-assisted Bayesian optimization. Mogrifier-LSTM and a feature-enhanced attention module capture transient fluctuations and long-range dependencies, while spectral normalization stabilizes training under physical constraints. Scenario-specific differentiable penalties discourage unrealistic battery SOH rebounds, wind-power boundary violations, and inconsistent PV irradiance–power relationships. An LLM incorporates domain knowledge to refine the hyperparameter search space, while numerical optimization determines the final configuration. Experiments on three energy datasets show that, relative to the standard TFT, PC-TFT reduces RMSE by 62.7%, 13.8%, and 15.8% for battery SOH, wind-power, and photovoltaic-power forecasting, respectively, while achieving R2 values above 0.97. These results demonstrate improvements in forecasting accuracy and physical consistency across the evaluated datasets, highlighting the potential of PC-TFT for forecasting-assisted energy management.