gmddGeoscientific Model Development DiscussionsgmddDGeosci. Model Dev. Discuss.1991-962XCopernicus PublicationsGöttingen, Germany10.5194/gmd-2019-17What do we do with model simulation crashes? Recommendations for global sensitivity analysis of earth and environmental systems modelsSheikholeslamiRazi12RazaviSaman123HaghnegahdarAmin121School of Environment and Sustainability, University of Saskatchewan, Saskatoon, Canada2Global Institute for Water Security, University of Saskatchewan, Saskatoon, Canada3Department of Civil, Geological, and Environmental Engineering, University of Saskatchewan, Saskatoon, Canada040220192019132This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/This article is available from https://www.geosci-model-dev-discuss.net/gmd-2019-17.htmlThe full text article is available as a PDF file from https://www.geosci-model-dev-discuss.net/gmd-2019-17.pdf

<p>Complex, software-intensive, technically advanced, and computationally demanding models, presumably with ever-growing realism and fidelity, have been widely used to simulate and predict the dynamics of the Earth and environmental systems. The parameter-induced simulation crash (failure) problem is typical across most of these models, despite considerable efforts that modellers have directed at model development and implementation over the last few decades. A simulation failure mainly occurs due to the violation of the numerical stability conditions, non-robust numerical implementations, or errors in programming. However, the existing sampling-based analysis techniques such as global sensitivity analysis (GSA) methods, which require running these models under many configurations of parameter values, are ill-equipped to effectively deal with model failures. To tackle this problem, we propose a novel approach that allows users to cope with failed designs (samples) during the GSA, without knowing where they took place and without re-running the entire experiment. This approach deems model crashes as missing data and uses strategies such as median substitution, single nearest neighbour, or response surface modelling to fill in for model crashes. We test the proposed approach on a 10-paramter HBV-SASK rainfall-runoff model and a 111-parameter MESH land surface-hydrology model. Our results show that response surface modelling is a superior strategy, out of the data filling strategies tested, and can scale well to the dimensionality of the model, sample size, and the ratio of number of failures to the sample size. Further, we conduct a "failure analysis" and discuss some possible causes of the MESH model failure.</p>