Enhancing hydrological modeling with transformers: a case study for 24-h streamflow prediction.
deep learning
flood forecasting
machine learning
rainfall-runoff modeling
streamflow forecasting
transformers
Journal
Water science and technology : a journal of the International Association on Water Pollution Research
ISSN: 0273-1223
Titre abrégé: Water Sci Technol
Pays: England
ID NLM: 9879497
Informations de publication
Date de publication:
May 2024
May 2024
Historique:
received:
23
10
2023
accepted:
22
03
2024
medline:
15
5
2024
pubmed:
15
5
2024
entrez:
15
5
2024
Statut:
ppublish
Résumé
In this paper, we address the critical task of 24-h streamflow forecasting using advanced deep-learning models, with a primary focus on the transformer architecture which has seen limited application in this specific task. We compare the performance of five different models, including persistence, long short-term memory (LSTM), Seq2Seq, GRU, and transformer, across four distinct regions. The evaluation is based on three performance metrics: Nash-Sutcliffe Efficiency (NSE), Pearson's
Identifiants
pubmed: 38747952
pii: wst_2024_110
doi: 10.2166/wst.2024.110
doi:
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
2326-2341Informations de copyright
© 2024 The Authors This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (CC BY 4.0), which permits copying, adaptation and redistribution, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/).
Déclaration de conflit d'intérêts
The authors declare there is no conflict.
Références
Ahmed A. M., Deo R. C., Feng Q., Ghahramani A., Raj N., Yin Z. & Yang L. 2021 Deep learning hybrid model with Boruta-Random forest optimiser algorithm for streamflow forecasting with climate mode indices, rainfall, and periodicity. Journal of Hydrology 599, 126350.
Alabbad Y. & Demir I. 2022 Comprehensive flood vulnerability analysis in urban communities: Iowa case study. International Journal of Disaster Risk Reduction 74, 102955.
Arnold J. 1994 SWAT-Soil and Water Assessment Tool.
Arnold J. G., Moriasi D. N., Gassman P. W., Abbaspour K. C., White M. J., Srinivasan R., Santhi C., Harmel R. D., Van Griensven A., Van Liew M. W. & Kannan N. 2012 SWAT: Model use, calibration, and validation. Transactions of the ASABE 55 (4), 1491–1508.
Banholzer S., Kossin J. & Donner S. 2014 The impact of climate change on natural disasters. In: Singh, A. & Zommers, Z. (eds.) Reducing Disaster: Early Warning Systems for Climate Change. Springer, Dordrecht, The Netherlands, pp. 21–49.
Bayar S., Demir I. & Engin G. O. 2009 Modeling leaching behavior of solidified wastes using back-propagation neural networks. Ecotoxicology and Environmental Safety 72 (3), 843–850.
pubmed: 18068228
Beven K. J. & Kirkby M. J. 1979 A physically based, variable contributing area model of basin hydrology/Un modèle à base physique de zone d'appel variable de l'hydrologie du bassin versant. Hydrological Sciences Journal 24 (1), 43–69.
Castangia M., Grajales L. M. M., Aliberti A., Rossi C., Macii A., Macii E. & Patti E. 2023 Transformer neural networks for interpretable flood forecasting. Environmental Modelling & Software 160, 105581.
Chen Z., Lin H. & Shen G. 2023 TreeLSTM: A spatiotemporal machine learning model for rainfall-runoff estimation. Journal of Hydrology: Regional Studies 48, 101474.
Cho K., Van Merriënboer B., Bahdanau D. & Bengio Y. 2014 On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. arXiv preprint arXiv:1409.1259.
Davenport F. V., Burke M. & Diffenbaugh N. S. 2021 Contribution of historical precipitation change to US flood damages. Proceedings of the National Academy of Sciences 118 (4).
pmcid: PMC7848586
Demir I. & Beck M. B. 2009 GWIS: A prototype information system for Georgia watersheds. In Georgia Water Resources Conference: Regional Water Management Opportunities, Athens, GA, USA.
Demir I., Xiang Z., Demiray B. & Sit M. 2022 WaterBench-Iowa: A large-scale benchmark dataset for data-driven streamflow forecasting. Earth System Science Data 14 (12), 5605–5616.
Demiray B. Z., Sit M. & Demir I. 2023 EfficientTempNet: Temporal Super-Resolution of Radar Rainfall. arXiv preprint arXiv:2303.05552.
Devia G. K., Ganasri B. P. & Dwarakish G. S. 2015 A review on hydrological models. Aquatic Procedia 4, 1001–1007.
Diffenbaugh N. S., Singh D., Mankin J. S., Horton D. E., Swain D. L., Touma D., Charland A., Liu Y., Haugen M., Tsiang M. & Rajaratnam B. 2017 Quantifying the influence of global warming on unprecedented extreme climate events. Proceedings of the National Academy of Sciences 114 (19), 4881–4886.
pmcid: PMC5441735
Feng D., Fang K. & Shen C. 2020 Enhancing streamflow forecast and extracting insights using long-short term memory networks with data integration at continental scales. Water Resources Research 56 (9), e2019WR026793.
Frame J. M., Kratzert F., Klotz D., Gauch M., Shalev G., Gilon O., Qualls L. M., Gupta H. V. & Nearing G. S. 2022 Deep learning rainfall–runoff predictions of extreme events. Hydrology and Earth System Sciences 26 (13), 3377–3392.
Granata F., Gargano R. & De Marinis G. 2016 Support vector regression for rainfall-runoff modeling in urban drainage: A comparison with the EPA's storm water management model. Water 8 (3), 69.
Guo Y., Yu X., Xu Y. P., Chen H., Gu H. & Xie J. 2021 AI-based techniques for multi-step streamflow forecasts: Application for multi-objective reservoir operation optimization and performance assessment. Hydrology and Earth System Sciences 25 (11), 5951–5979.
Hochreiter S. & Schmidhuber J. 1997 Long short-term memory. Neural Computation 9 (8), 1735–1780.
pubmed: 9377276
Honorato A. G. D. S. M., Silva G. B. L. D. & Guimaraes Santos C. A. 2018 Monthly streamflow forecasting using neuro-wavelet techniques and input analysis. Hydrological Sciences Journal 63 (15–16), 2060–2075.
Ibrahim K. S. M. H., Huang Y. F., Ahmed A. N., Koo C. H. & El-Shafie A. 2022 A review of the hybrid artificial intelligence and optimization modelling of hydrological streamflow forecasting. Alexandria Engineering Journal 61 (1), 279–303.
Krajewski W. F., Ghimire G. R., Demir I. & Mantilla R. 2021 Real-time streamflow forecasting: AI vs. Hydrologic insights. Journal of Hydrology X 13, 100110.
Kratzert F., Klotz D., Brenner C., Schulz K. & Herrnegger M. 2018 Rainfall–runoff modelling using long short-term memory (LSTM) networks. Hydrology and Earth System Sciences 22 (11), 6005–6022.
Krause P., Boyle D. P. & Bäse F. 2005 Comparison of different efficiency criteria for hydrological model assessment. Advances in Geosciences 5, 89–97.
Lee T. H. & Georgakakos K. P. 1996 Operational rainfall prediction on Meso-
Li Z. & Demir I. 2022 A comprehensive web-based system for flood inundation map generation and comparative analysis based on height above nearest drainage. Science of the Total Environment 828, 154420.
pubmed: 35276151
Li Z. & Demir I. 2023 U-net-based semantic classification for flood extent extraction using SAR imagery and GEE platform: A case study for 2019 central US flooding. Science of the Total Environment 869, 161757.
pubmed: 36690091
Lin T., Wang Y., Liu X. & Qiu X. 2022 A survey of transformers. AI Open 3, 111–132.
Liu C., Liu D. & Mu L. 2022 Improved transformer model for enhanced monthly streamflow predictions of the Yangtze River. IEEE Access 10, 58240–58253.
Mosavi A., Ozturk P. & Chau K. W. 2018 Flood prediction using machine learning models: Literature review. Water 10 (11), 1536.
Munich Re 2022 Hurricanes, Cold Waves, Tornadoes: Weather Disasters in USA Dominate Natural Disaster Losses in 2021. Available from: https://www.munichre.com/en/company/media-relations/media-information-and-corporate-news/media-information/2022/natural-disaster-losses-2021.html.
NDRCC 2021 2020 Global Natural Disaster Assessment Report. Available from: https://reliefweb.int/report/china/2020-global-natural-disaster-assessment-report
NOAA National Centers for Environmental Information (NCEI) 2022 US Billion-Dollar Weather and Climate Disasters. Available from: https://www.ncei.noaa.gov/access/monitoring/billions/. doi:10.25921/stkw-7w73.
Ren-Jun Z. 1992 The Xinanjiang model applied in China. Journal of Hydrology 135 (1–4), 371–381.
Salas J. D., Markus M. & Tokar A. S. 2000 Streamflow forecasting based on artificial neural networks. In: Govindaraju, R. S. & Rao, A. R. (eds.) Artificial Neural Networks in Hydrology. Springer, Dordrecht, The Netherlands,
Sharma P. & Machiwal D. 2021 Advances in Streamflow Forecasting: From Traditional to Modern Approaches. Elsevier, Amsterdam, The Netherlands.
Sit M., Demiray B. & Demir I. 2021a Short-term Hourly Streamflow Prediction with Graph Convolutional GRU Networks. arXiv preprint arXiv:2107.07039.
Sit M., Seo B. C. & Demir I. 2021b Iowarain: A Statewide Rain Event Dataset Based on Weather Radars and Quantitative Precipitation Estimation. arXiv preprint arXiv:2107.03432.
Sit M., Demiray B. Z. & Demir I. 2022a A Systematic Review of Deep Learning Applications in Streamflow Data Augmentation and Forecasting. EarthArxiv 3617. Available from: https://doi.org/10.31223/X5HM08
Sit M., Demiray B. Z. & Demir I. 2022b A Systematic Review of Deep Learning Applications in Interpolation and Extrapolation of Precipitation Data. EarthArxiv 4715. Available from: https://doi.org/10.31223/X57H2H
Sit M., Seo B. C., Demiray B. Z. & Demir I. 2023a Efficientrainnet: Smaller Neural Networks Based on Efficientnetv2 for Rainfall Nowcasting. EarthArxiv 5232. Available from: https://doi.org/10.31223/X5VQ1S
Sit M., Demiray B. Z. & Demir I. 2023b Spatial downscaling of streamflow data with attention based spatio-temporal graph convolutional networks. EarthArxiv 5227. Available from: https://doi.org/10.31223/X5666M
Strauss B. H., Kopp R. E., Sweet W. V. & Bittermann K. 2016 Unnatural Coastal Floods: Sea Level Rise and the Human Fingerprint on US Floods Since 1950. Climate Central.
Tabari H. 2020 Climate change impact on flood and extreme precipitation increases with water availability. Scientific Reports 10 (1), 1–10.
pmcid: PMC6959339
pubmed: 31913322
Tay Y., Dehghani M., Bahri D. & Metzler D. 2022 Efficient transformers: A survey. ACM Computing Surveys 55 (6), 1–28.
Trenberth K. E., Cheng L., Jacobs P., Zhang Y. & Fasullo J. 2018 Hurricane Harvey links to ocean heat content and climate change adaptation. Earth's Future 6 (5), 730–744.
Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser Ł. & Polosukhin I. 2017 Attention is all you need. Advances in Neural Information Processing Systems 30, 5998–6008.
Wen Q., Zhou T., Zhang C., Chen W., Ma Z., Yan J. & Sun L. 2022 Transformers in Time Series: A Survey. arXiv preprint arXiv:2202.07125.
World Meteorological Organization (WMO) 2021 The Atlas of Mortality and Economic Losses From Weather, Climate and Water Extremes (1970–2019).
Wu H., Xu J., Wang J. & Long M. 2021 Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in Neural Information Processing Systems 34, 22419–22430.
Xiang Z. & Demir I. 2021 High-Resolution Rainfall-Runoff Modeling Using Graph Neural Network. arXiv preprint arXiv:2110.10833.
Xiang Z. & Demir I. 2022a Real-Time Streamflow Forecasting Framework, Implementation and Post-analysis Using Deep Learning. EarthArxiv 3162. https://doi.org/10.31223/X5BW6R
Xiang Z. & Demir I. 2022b Fully Distributed Rainfall-Runoff Modeling Using Spatial-Temporal Graph Neural Network. EarthArxiv 3018. https://doi.org/10.31223/X57P74
Xiang Z., Demir I., Mantilla R. & Krajewski W. F. 2021 A Regional Semi-Distributed Streamflow Model Using Deep Learning. EarthArxiv 2152. https://doi.org/10.31223/X5GW3V
Yan J., Jin J., Chen F., Yu G., Yin H. & Wang W. 2018 Urban flash flood forecast using support vector machine and numerical simulation. Journal of Hydroinformatics 20 (1), 221–231.
Yaseen Z. M., El-Shafie A., Jaafar O., Afan H. A. & Sayl K. N. 2015 Artificial intelligence based models for stream-flow forecasting: 2000–2015. Journal of Hydrology 530, 829–844.
Yaseen Z. M., Ebtehaj I., Bonakdari H., Deo R. C., Mehr A. D., Mohtar W. H. M. W., Diop L., El-Shafie A. & Singh V. P. 2017 Novel approach for streamflow forecasting using a hybrid ANFIS-FFA model. Journal of Hydrology 554, 263–276.
Yaseen Z. M., Awadh S. M., Sharafati A. & Shahid S. 2018 Complementary data-intelligence model for river flow simulation. Journal of Hydrology 567, 180–190.
Yaseen Z. M., Sulaiman S. O., Deo R. C. & Chau K. W. 2019 An enhanced extreme learning machine model for river flow forecasting: State-of-the-art, practical applications in water resource engineering area and future research direction. Journal of Hydrology 569, 387–408.
Yildirim E. & Demir I. 2022 Agricultural flood vulnerability assessment and risk quantification in Iowa. Science of The Total Environment 826, 154165.
pubmed: 35231508
Zhou H., Zhang S., Peng J., Zhang S., Li J., Xiong H. & Zhang W. 2021 Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, No. 12, pp. 11106–11115.
Zhou T., Ma Z., Wen Q., Wang X., Sun L. & Jin R. 2022 Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, PMLR, pp. 27268–27286.