Evaluation of Performance Measures for Comparing Flood Models With Satellite Observations

Selecting appropriate performance measures is crucial for calibrating numerical flood models with satellite images, as differences between performance measures can significantly impact model predictions. The present study reports a step-by-step methodology to guide the selection of performance measures to compare numerical simulations of a flood event with a satellite image, for purposes of model calibration or validation. Faced with a set of performance measures, practitioners need to assess which are most suitable for comparing observed and simulated flood maps. First, a correlation analysis was conducted to determine the relationships between various performance measures. A series of criteria were then applied to evaluate the sensitivity of the performance measures with respect to flood extent, satellite characteristics (position and orientation), and errors in measurement or processing (raw satellite values or extraction of flood maps). The performance measures meeting the most criteria were then ranked based on the identifiability of parameter values with regard to observation errors. The methodology is discussed numerically in the context of a single flood event (on the Garonne River in France in February 2021), using a distribution of water depths simulated from a shallow-water model (parameterized by an uncertain friction field). The methodology was shown to be useful in the study case in identifying 4 performance measures out of 24. The proposed methodology is intended to be generic, and therefore, applicable to any flood model or flood event.