The indicators calculated by the Comparator object are based on observed and simulated values. Since the Comparator object is able to compare variables of type flow, height, altitude, power and True/False, the descriptions below only refer to these 2 variables.
9.1 Nash coefficient
The Nash-Sutcliffe criteria (Nash and Sutcliffe 1970) is used to assess the predictive power of hydrological models (Ajami et al. 2004; Schaefli et al. 2005; Jordan 2007; Viviroli et al. 2009; García Hernández 2011). It is defined as presented in Equation 9.1.
\[ Nash = 1 - \frac{\sum_{t = t_{i}}^{t_{f}}{(X_{sim,t} - X_{ref,t})^{2}}}{\sum_{t = t_{i}}^{t_{f}}{(X_{ref,t} - {\overline{X}_{ref}})^{2}}} \tag{9.1}\]
with \(Nash\): Nash-Sutcliffe model efficiency coefficient [-]; \(X_{sim,t}\): simulated variable (discharge [L3/T] or height [L]) at time \(t\); \(X_{ref,t}\) : observed variable (discharge [L3/T] or height [L]) at time \(t\); \(\overline{X}_{ref}\): average observed variable (discharge [L3/T] or height [L]) for the considered period.
It varies from -∞ to 1, with 1 representing the best performance of the model and zero the same performance than assuming the average of all the observations at each time step.
9.2 Nash coefficient for logarithm values
The Nash-Sutcliffe coefficient for logarithm flow values (\(Nash-ln\)) is used to assess the hydrological models performance for low values (Krause, Boyle, and Bäse 2005; Nóbrega et al. 2011). It is defined as presented in Equation 9.2.
\[ \text{Nash-ln} = 1 - \frac{\sum_{t = t_{i}}^{t_{f}}{(ln(X_{sim,t}) - ln(X_{ref,t}))^{2}}}{\sum_{t = t_{i}}^{t_{f}}{(ln(X_{ref,t}) - ln({\overline{X}_{ref}}))^{2}}} \tag{9.2}\]
with \(Nash-ln\): Nash-Sutcliffe coefficient for log values [-].
It varies from -∞ to 1, with 1 representing the best performance of the model.
9.3 Pearson Correlation Coefficient
The Pearson correlation coefficient shows the covariability of the simulated and observed values without penalizing for bias (Aghakouchak and Habib 2010; Wang et al. 2011). It is defined as presented in Equation 9.3.
\[ Pearson = \frac{\sum_{t = t_{i}}^{t_{f}}{(X_{sim,t} - {\overline{X}_{sim}}) \cdot (X_{ref,t} - {\overline{X}_{ref}})}}{\sqrt{\sum_{t = t_{i}}^{t_{f}}{(X_{sim,t} - {\overline{X}_{sim}})^{2}} \cdot \sum_{t = t_{i}}^{t_{f}}{(X_{ref,t} - {\overline{X}_{ref}})^{2}}}} \tag{9.3}\]
with \(Pearson\): Pearson Correlation Coefficient [-]; \({\overline{X}_{sim}}\): average simulated variable (discharge [L3/T] or height [L]) for the considered period.
It varies from -1 to 1, with 1 representing the best performance of the model.
9.4 Kling-Gupta Efficiency
The Kling-Gupta efficiency (Gupta et al. 2009) provides an indicator which facilitates the global analysis based on different components (correlation, bias and variability) for hydrological modelling issues.
Kling, Fuchs, and Paulin (2012) proposed a revised version of this indicator, to ensure that the bias and variability ratios are not cross-correlated. This update is proposed as indicator in RS MINERVE (Equation 9.4):
\[ KGE' = 1 - \sqrt{(r - 1)^{2} + (\beta - 1)^{2} + (\gamma - 1)^{2}} \tag{9.4}\]
\[ \beta = \frac{\mu_{s}}{\mu_{o}} \tag{9.5}\]
\[ \gamma = \frac{{CV}_{s}}{{CV}_{o}} = \frac{\sigma_{s}/\mu_{s}}{\sigma_{o}/\mu_{o}} \tag{9.6}\]
with \(KGE'\): modified KGE-statistic [-]; \(r\): correlation coefficient between simulated and reference values [-]; \(\beta\): bias ratio [-]; \(\gamma\): variability ratio [-]; \(\mu\): mean discharge [L3/T]; \(CV\): coefficient of variation [-]; \(\sigma\): standard deviation of discharge [L3/T]; the indices \(s\) and \(o\) indicate respectively simulated and observed discharge values.
It varies from 0 to 1, with 1 representing the best performance.
9.5 Bias Score
The Bias Score (\(BS\)) is a symmetric estimation of the match between the average simulation and average observation (Wang et al. 2011). It is defined as presented in Equation 9.7.
\[ BS = 1 - \bigg( max \Big(\frac{{\overline{X}}_{sim}}{{\overline{X}}_{ref}};\frac{{\overline{X}}_{ref}}{{\overline{X}}_{sim}} \Big) - 1 \bigg)^2 \tag{9.7}\]
with \(BS\): Bias Score [-].
It varies from -∞ to 1, with 1 representing the best performance of the model.
9.6 Relative Root Mean Square Error
The Relative Root Mean Square Error (\(RRMSE\)) is defined as the RMSE normalized to the mean of the observed values (Feyen et al. 2000; El-Nasr et al. 2005; Heppner et al. 2006) and is presented in Equation 9.8.
\[ RRMSE = \frac{\sqrt{\frac{\sum_{t = t_{i}}^{t_{f}}{(X_{sim,t} - X_{ref,t})^{2}}}{n}}}{{\overline{X}}_{ref}} \tag{9.8}\]
with \(RRMSE\): relative \(RMSE\) [-]; \(n\): number of values [-].
It varies from 0 to +∞. The smaller \(RRMSE\), the better the model performance is.
9.7 Relative Volume Bias
The Relative Volume Bias (\(RVB\)), sometimes called differently, corresponds in this case to the relative error between the simulated and the observed volumes during the studied period (Ajami et al. 2004; Schaefli et al. 2005; Moriasi et al. 2007; Aghakouchak and Habib 2010) according to Equation 9.9. This indicator is envisaged for the comparison between observed and simulated discharges.
\[ RVB = \frac{\sum_{t = t_{i}}^{t_{f}}{(X_{sim,t} - X_{ref,t})}}{\sum_{t = t_{i}}^{t_{f}}{(X_{ref,t})}} \tag{9.9}\]
with \(RVB\): relative volume bias between forecast and observation for the considered period [-]; \(X\) usually corresponding to the discharge variable.
The \(RVB\) varies from -1 to +∞. An index near to zero indicates a good performance of the simulation. Negative values are returned when simulated variable is, in average, smaller than the average of the observed one (deficit model), while positive values mean the opposite (overage model).
9.8 Normalized Peak Error
The Normalized Peak Error (\(NPE\)) indicates the relative error between the simulated and the observed maximum values (Masmoudi and Habaieb 1993; Sun et al. 2000; Ajami et al. 2004; Gabellani et al. 2007). It is computed according to Equation 9.10 to Equation 9.12.
\[ NPE = \frac{S_{\max} - R_{\max}}{R_{\max}} \tag{9.10}\]
\[ S_{\max} = \overset{t_{f}}{\underset{t = t_{i}}{\vee}}Q_{sim,t} \tag{9.11}\]
\[ R_{\max} = \overset{t_{f}}{\underset{t = t_{i}}{\vee}}Q_{ref,t} \tag{9.12}\]
with \(NPE\): relative error between simulated and observed peak value [-]; \(S_{max}\): maximum simulated value (discharge [L3/T] or height [L]) for the studied period; \(R_{max}\) : maximum observed value (discharge [L3/T] or height [L]) for the studied period.
The NPE varies from -1 to +∞. Negative values are returned when maximum simulated value is below the observed one, while positive values mean the opposite. Values near to zero indicate a good performance of simulated peaks regarding observed ones.
The indicator is computed over the entire simulation period and the absolute maximum of the simulated and the observed peaks are considered! This indicator should therefore be used with care when simulating over long periods of time.
9.9 Peirce Skill Score
The Peirce Skill Score (\(PSS\)) indicates the performance of the model to reproduce the overrun of a threshold (Peirce 1884; Manzato 2007). Based on a contingency table definging the number of cases where the simulation and the obseration exceed or not the threshold, the \(PSS\) is computed according to Equation 9.13.
\[ PSS = \frac{ad - bc}{(a + c)(b + d)} \tag{9.13}\]
with \(a\): the number of cases when both simulation and observation exceed the threshold defined in the Comparator (event); \(b\): the number of cases when the simulation exceeds the threshold but not the obseration (false); \(c\): the number of cases when the observation exceeds the threshold but not the simulation (miss); \(d\): the number of cases when both simulation and obseration are below the threshold (no event).
If the denominerator equals 0 (division by 0), a value of 0 is returned for the PSS.
9.10 Overall Accuracy
The Overall Accurary (\(OA\)) indicates the performance of the model to reproduce the overrun of a threshold (Parajka and Blöschl 2008). Based on the same contingency table as the Pierce Skill Score, the \(OA\) is computed according to Equation 9.14.
\[ OA = \frac{a + d}{a + b + \ c + d} \tag{9.14}\]