Basic statistical estimation outperforms machine learning in monthly prediction of seasonal climatic parameters
Loading...
Date
2021
Journal Title
Journal ISSN
Volume Title
Publisher
MPDI
Abstract
Machine learning (ML) has been utilized to predict climatic parameters, and many successes
have been reported in the literature. In this paper, we scrutinize the effectiveness of five widely
used ML algorithms in the monthly prediction of seasonal climatic parameters using monthly image
data. Specifically, we quantify the predictive performance of these algorithms applied to five climatic
parameters using various combinations of features. We compare the predictive accuracy of the
resulting trained ML models to that of basic statistical estimators that are computed directly from the
training data. Our results show that ML never significantly outperforms the statistical baseline, and
underperforms for most feature sets. Unlike previous similar studies, we provide error bars for the
relative performance of different predictors based on jackknife estimates applied to differences in
predictive error magnitudes. We also show that the practice of shuffling data sequences which was
employed in some previous references leads to data leakage, resulting in over-estimated performance.
Ultimately, the paper demonstrates the importance of using well-grounded statistical techniques
when producing and analyzing the results of ML predictive models.
Description
Keywords
Geophysical image data, Statistical modeling, Data leakage, Jackknife, Machine learning
Citation
Hussein, E. A. et al. (2021). Basic statistical estimation outperforms machine learning in monthly prediction of seasonal climatic parameters. Atmosphere,12 (5),1-20. https://doi.org/10.3390/atmos12050539