Indicators for evaluation of model performance: irrigation hydraulics applications
Abstract
Several mathematical models have been developed for applications in the hydraulics of irrigation systems and several performance indicators of these models are used and suggested by the literature. Thus, the objective of this work was to investigate the performance of statistical indicators for the evaluation of models in irrigation hydraulics. For this, three case studies which represent typical irrigation hydraulics modeling were used to assess the indicators. A set of indicators were analyzed: a) difference-based: mean absolute error, mean square error, root mean square error, scaled root mean square error, and percent mean absolute error; b) efficiency-based: Nash-Sutcliffe and Legates-McCabe; c) correlation coefficient ( ); d) coefficient of determination ( ); e) index of agreement index ( ); f) Camargo and Sentelhas index ( ); and g) graphical methods: regression error characteristic curve based on relative absolute error and 1:1 scatter plot. For the evaluated cases, which are physical phenomena, differentiable indicators are similar measures and it is appropriate to report either or both indices. The assessment of models must also be supported by graphical analysis, which shows the real scenario of errors in the model evaluation processes. Efficiency-based indicators, , , , and are not recommended and should be avoided in modeling of irrigation hydraulics.
Downloads
References
Alazba, A. A., Mattar, M. A., ElNesr, M. N., & Amin, M. T. (2012). Field assessment of friction head loss and friction correction factor equations. Journal of Irrigation and Drainage Engineering, 138(2), 166-176. DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0000387
Alexandrov, G. A., Ames, D., Bellocchi, G., Michael, B., Crout, N., Erechtchoukova, M., … Samaniego, L. (2011). Technical assessment and evaluation of environmental models and software: Letter to the Editor. Environmental Modelling & Software, 26(3), 328-336. DOI: https://doi.org/10.1016/j.envsoft.2010.08.004
Al-Ghobari, H. M., El-Marazky, M. S., Dewidar, A. Z., & Mattar, M. A. (2018). Prediction of wind drift and evaporation losses from sprinkler irrigation using neural network and multiple regression techniques. Agricultural Water Management, 195, 211-221. DOI: https://doi.org/10.1016/j.agwat.2017.10.005
Ali, M. H., & Abustan, I. (2014). A new novel index for evaluating model performance. Journal of Natural Resources and Development, 4, 1-9. DOI: https://doi.org/10.5027/jnrd.v4i0.01
Bachour, R., Walker, W. R., Ticlavilca, A. M., & McKee, M. (2014). Estimation of spatially distributed evapotranspiration using remote sensing and a relevance vector machine. Journal of Irrigation and Drainage Engineering, 140(8). DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0000754
Bellocchi G., Acuit, M., Fila, G., & Donatelli, M. (2002). An indicator of solar radiation model performance based on a fuzzy expert system. Agronomy Journal, 94(6), 1222-1233. DOI: https://doi.org/10.2134/agronj2002.1222
Camargo, A. P., & Sentelhas, P. C. (1997). Avaliação do desempenho de diferentes métodos de estimativa da evapotranspiração potencial no estado de São Paulo, Brasil. Revista Brasileira Meteorologia, 5(1), 89-97.
Cano, N. D., Camargo, A. P., Muniz, G. K., Oliveira, J., Dalfré Filho, J. G., & Frizzone, J. A. (2021). Performance of models to determine flowrate using orifice plates. Revista Brasileira de Engenharia Agrícola e Ambiental, 25(1), 10-16. DOI: https://doi.org/10.1590/1807-1929/agriambi.v25n1p10-16
Chatterjee, S., & Simonoff, J. (2013). Handbook of regression analysis. Hoboken, NJ: John Wiley and Sons.
Colebrook, C. F., & White, C. M. (1937). Experiments with fluid friction in roughened pipes. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 161(906), 367-381. DOI: https://doi.org/10.1098/rspa.1937.0150
Elbana, M., Ramírez de Cartagena, F., & Puig-Bargués, J. (2013). New mathematical model for computing head loss across sand media filter for microirrigation systems. Irrigation Science, 31, 343-349. DOI: https://doi.org/10.1007/s00271-011-0310-4
Fox, D. G. (1981). Judging air quality model performance. Bulletin of the American Meteorological Society, 62(5), 599-609. DOI: https://doi.org/10.1175/1520-0477(1981)062<0599:JAQMP>2.0.CO;2
Gaj, N., & Madramootoo, C. A. (2020). Effects of perforation geometry on pipe drainage in agricultural lands. Journal of Irrigation and Drainage Engineering, 146(7). DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0001482
García Nieto, P. J., García-Gonzalo, E., Bové, J., Arbat, G., Duran-Ros, M., & Puig-Bargués, J. (2017). Modeling pressure drop produced by different filtering media in microirrigation sand filters using the hybrid ABC-MARS-based approach, MLP neural network and M5 model tree. Computers and Electronics in Agriculture, 139, 65-74. DOI: https://doi.org/10.1016/j.compag.2017.05.008
Hallak, R., & Pereira Filho, A. J. (2011). Metodologia para análise de desempenho de simulações de sistemas convectivos na região metropolitana de São Paulo com o modelo ARPS: sensibilidade a variações com os esquemas de advecção e assimilação de dados. Revista Brasileira de Meteorologia, 24(4), 591-608. DOI: https://doi.org/10.1590/S0102-77862011000400009
Hatiye, S. D., Prasad, K. S. H., & Ojha, C. S. P. (2018). Deep percolation under irrigated water-intensive crops. Journal of Irrigation and Drainage Engineering, 144(8), 1-13. DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0001326
International Organization for Standardization [ISO]. (2003). ISO 5167-2 - Measurement of fluid flow by means of pressure differential devices inserted in circular-cross section conduits running full – Part 2: Orifice plates. Genebra, SW: ISO.
Katsurayama, G. T., Sobenko, L. R., Camargo, A. P., Botrel, T. A., Frizzone, J. A., & Duarte, S. N. 2020. A mathematical model for hydraulic characterization of microtube emitters using dimensional analysis. Journal of Agricultural Science and Technology, 22(4), 1123-1135.
Legates, D. R., & McCabe, G. J. (1999). Evaluating the use of “goodness-of fit” measures in hydrologic and hydroclimatic model validation. Water Resources Research, 35(1), 233-241. DOI: https://doi.org/10.1029/1998WR900018
McCuen, R. H., Knight, Z., & Cutter, A. G. (2006). Evaluation of the Nash–Sutcliffe efficiency index. Journal of Hydrologic Engineering, 11(6), 597-602. DOI: https://doi.org/10.1061/(ASCE)1084-0699(2006)11:6(597)
Montgomery, D. C., & Runger, G. C. (2013). Applied statistics and probability for engineers. (6th ed.). Hoboken, NJ: Wiley.
Moriasi, D. N., Arnold, J. G., Van Liew, M. W., Bingner, R. L., Harmel, R. D., & Veith, T. L. (2007). Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Transactions of ASABE, 50(3), 885-900. DOI: https://doi.org/10.13031/2013.23153
Nash, J. E., & Sutcliffe, J. V. (1970). River flow forecasting through conceptual models part I – A discussion of principles. Journal of Hydrology, 10(3), 282-290. DOI: https://doi.org/10.1016/0022-1694(70)90255-6
Najafzadeh, M., Shiri, J., Sadeghi, G., & Ghaemi, A. (2018). Prediction of the friction factor in pipes using model tree. Journal of Hydraulic Engineering, 24(1), 9-15. DOI: https://doi.org/10.1080/09715010.2017.1333926
Oke, I. A., Ojo, S. O., & Adeosun, O. O. (2015). Performance evaluation for Darcy friction factor formulae using Colebrook-White as reference. Ife Journal of Science, 17(1), 75-86.
Pimenta, B. D., Robaina, A. D., Peiter, M. X., Mezzomo, W., Kirchner, J. H., & Ben, L. H. B. (2018). Performance of explicit approximations of the coefficient of head loss for pressurized conduits. Revista Brasileira de Engenharia Agrícola e Ambiental, 22(5), 301-307. DOI: https://doi.org/10.1590/18071929/agriambi.v22n5p301-307
Provenzano, G., Di Dio, P. M., & Leone, R. (2014). Assessing a local losses evaluation procedure for low-pressure lay-flat drip laterals. Journal of Irrigation and Drainage Engineering, 140(6). DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0000731
Provenzano, G., Alagna, V., Autovino, D., Juarez, J. M., & Rallo, G. (2016). Analysis of geometrical relationships and friction losses in small-diameter lay-flat polyethylene pipes. Journal of Irrigation and Drainage Engineering, 142(2), 1-9. DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0000958
Sarwar, A., Peters, R. T., & Mohamed, A. Z. (2019). Linear mixed modeling and artifcial neural network techniques for predicting wind drift and evaporation losses under moving sprinkler irrigation systems. Irrigation Science, 38, 177-188. DOI: https://doi.org/10.1007/s00271-019-00659-x
Shaikh, M. M., Massan S. R., & Wagan, A. I. (2015). A new explicit approximation to Colebrook’s friction factor in rough pipes under highly turbulent cases. International Journal of Heat and Mass Transfer, 88, 538-543. DOI: https://doi.org/10.1016/j.ijheatmasstransfer.2015.05.006
Shiri, J., & Kisi, Ö. (2011). Application of artificial intelligence to estimate daily pan evaporation using available and estimated climatic data in the Khozestan Province (South Western Iran). Journal of Irrigation and Drainage Engineering, 137, 412-425. DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0000315
Sobenko, L. R., Bombardelli, W. W. A., Camargo, A. P., Frizzone, J. A., & Duarte, S. N. (2020). Minor losses through start connectors in microirrigation laterals: dimensional analysis and artificial neural networks approaches. Journal of Irrigation and Drainage Engineering, 146(5), 1-13. DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0001466
Souza, R. O. R. M., & Botrel, T. A. (2004). Modeling for the design of microtubes in trickle irrigation. Revista Brasileira de Engenharia Agrícola e Ambiental, 8(1), 16-22. DOI: https://doi.org/10.1590/S1415-43662004000100003
Swamee, P. K., & Jain, A. K. (1976). Explicit equations for pipe flow problems. Journal of the Hydraulics Division, 102, 657-664.
Thompson, M. J., Hathaway, J. M., & Schwartz, J. S. (2018). Three-dimensional modeling of the hydraulic function and channel stability of regenerative stormwater conveyances. Journal of Sustainable Water in the Built Environment, 4(3). DOI: https://doi.org/10.1061/ JSWBAY.0000861
Willmott, C. J. (1981). On the validation of models. Physical Geography, 2, 184-194. DOI: https://doi.org/10.1080/02723646.1981.10642213
Willmott, C. J., Robeson, S. M., & Matsuura, K. (2011). A refined index of model performance. International Journal of Climatology, 32(13), 2088–2094. DOI: https://doi.org/10.1002/joc.2419
Zhang, Z., Chai, J., Li, Z., Xu, Z., & Li, P. (2019). Discharge coefficient of a spillway with a riser perforated by rectangular orifices. Journal of Irrigation and Drainage Engineering, 145(11). DOI: https://doi.org/10.1061/(ASCE)IR.1943-4774.0001425
DECLARATION OF ORIGINALITY AND COPYRIGHTS
I Declare that current article is original and has not been submitted for publication, in part or in whole, to any other national or international journal.
The copyrights belong exclusively to the authors. Published content is licensed under Creative Commons Attribution 4.0 (CC BY 4.0) guidelines, which allows sharing (copy and distribution of the material in any medium or format) and adaptation (remix, transform, and build upon the material) for any purpose, even commercially, under the terms of attribution.
