Real-time electricity load forecasting in South Africa using SOM-enriched deep learning ensembles

Abstract

Accurate short-term electrical demand forecasting is critical for maintaining operational efficiency and energy security, especially in power-constrained systems like South Africa's Eskom. Statistical methods like autoregressive integrated moving average (ARIMA) and exponential smoothing often fail to represent nonlinear and regime-dependent trends in power demand. This study presents a dynamic ensemble that combines deep neural networks (DNN) and long short-term memory (LSTM) architectures, which are both augmented by self-organising maps (SOM)-based clustering. The proposed method divides historical hourly load data from the Drakensberg generation plant into discrete temporal regimes using SOM, then trains the DNN and LSTM architectures within each regime, and dynamically combines their predictions. Shapley additive explanations (SHAP) are used to improve the interpretability of the impact of each cluster and time hierarchies, while mean square error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) measures are used to assess prediction performance. The ensemble architecture delivers a higher accuracy, lowering MAPE to 2.20% while consistently outperforming individual benchmark architectures. The deployment on Amazon Web Services (AWS) proves the model's scalability and appropriateness for real-time applications. Although performance degrades in irregular demand clusters, adaptive re-clustering may alleviate this constraint. Overall, the combined DNN-LSTM-SOM strategy is a reliable, interpretable, and scalable solution for short-term load forecasting, enabling better operational planning and grid dependability in developing energy systems.