Evaluation of watershed-level runoff against river gauge data
Miquel De Cáceres / María González
2025-03-28
Source:vignettes/evaluation/WatershedLevelEvaluation.Rmd
WatershedLevelEvaluation.RmdIntroduction
Goal
The aim of this article is to provide an assessment of the
performance of spwb_land() for the prediction of watershed
outflow. To this aim, we simulate hydrological processes in a set of
benchmark watersheds and compare the model predictions of watershed
outflow against measurements obtained using river gauges at watershed
outlets.
Simulation procedure
For each watershed, the following procedure has been conducted:
- Initial warm-up simulation for a specified number of years
- Simulation for the period with observed data before calibrating watershed parameters
- Manual calibration of watershed parameters (to be replaced with automatized calibration)
- Final simulation for the period with observed data after calibrating watershed parameters.
Goodness-of-fit statistics
The following goodness of fit statistics are calculated using package
hydroGOF:
- Nash-Sutcliffe Efficiency (NSE): This coefficient is sensitive to extreme values and might yield sub-optimal results when the dataset contains large outliers.
- Kling–Gupta efficiency (KGE): Provides a decomposition of the Nash-Sutcliffe efficiency, which facilitates the analysis of the importance of different components (bias, correlation and variability).
- Index of agreement (d): Initially proposed by Willmott (1981) to overcome the drawbacks of the R2, such as the differences in observed and predicted means and variances (Legates and McCabe, 1999). d is also dimensionless and bounded between 0 and 1 and can be interpreted similarly to R2.
- Volumetric efficiency index (VE): Originally proposed by Criss and Winston (2008) to circumvent some of the NSE flaws. VE values are also bounded [0, 1] and represent the fraction of water delivered at the proper time.
- Root mean squared error (RMSE): The usual estimation of average model error (i.e. the square root of mean squared errors).
Watershed (TETIS) parameters
The following table contains the set of TETIS parameters employed in
spwb_land() simulations on all watersheds, before and after
calibration:
| medfateland | medfate | model | watershed | Calibration | R_localflow | R_interflow | R_baseflow | n_interflow | n_baseflow | num_daily_substeps | rock_max_infiltration | deep_aquifer_loss |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2.5.1 | 4.8.0 | tetis | aiguadora | before | 1 | 50 | 5 | 1.0 | 1.0 | 4 | 10 | 0 |
| 2.5.1 | 4.8.0 | tetis | aiguadora | after | 1 | 8 | 1 | 0.5 | 0.7 | 4 | 10 | 5 |
| 2.5.1 | 4.8.0 | tetis | aiguadora | before | 1 | 50 | 5 | 1.0 | 1.0 | 4 | 10 | 0 |
| 2.5.1 | 4.8.0 | tetis | aiguadora | after | 1 | 8 | 1 | 0.5 | 0.7 | 4 | 10 | 5 |
| 2.5.2 | 4.8.0 | tetis | aiguadora | before | 1 | 50 | 5 | 1.0 | 1.0 | 4 | 10 | 0 |
| 2.5.2 | 4.8.0 | tetis | aiguadora | after | 1 | 8 | 1 | 0.5 | 0.7 | 4 | 10 | 5 |
Evaluation results
AIGUADORA watershed with TETIS and version 2.4.6
Goodness-of-fit
| Scale | Calibration | NSE | KGE | d | VE | RMSE |
|---|---|---|---|---|---|---|
| Daily | Before | -10.432 | -2.136 | 0.614 | -3.571 | 2.487 |
| Daily | After | -0.748 | 0.132 | 0.306 | -0.224 | 0.973 |
| Monthly | Before | -21.961 | -3.604 | 0.652 | -3.483 | 2.352 |
| Monthly | After | -1.519 | -0.209 | 0.325 | -0.054 | 0.779 |
Hydrological analysis
Density distribution
Percentiles
| Observed | Uncalibrated | Calibrated | |
|---|---|---|---|
| 1% | 0.040 | 0.980 | 0.051 |
| 5% | 0.060 | 1.129 | 0.095 |
| 10% | 0.070 | 1.254 | 0.134 |
| 15% | 0.080 | 1.377 | 0.169 |
| 25% | 0.110 | 1.580 | 0.250 |
| 50% | 0.250 | 2.201 | 0.590 |
| 75% | 0.520 | 3.088 | 1.279 |
| 85% | 0.820 | 3.749 | 1.765 |
| 90% | 1.150 | 4.262 | 2.126 |
| 95% | 1.724 | 5.175 | 2.719 |
| 99% | 3.370 | 7.220 | 4.049 |
AIGUADORA watershed with TETIS and version 2.5.1
Goodness-of-fit
| Scale | Calibration | NSE | KGE | d | VE | RMSE |
|---|---|---|---|---|---|---|
| Daily | Before | -7.688 | -1.721 | 0.543 | -2.851 | 2.257 |
| Daily | After | -0.388 | 0.306 | 0.466 | 0.061 | 0.902 |
| Monthly | Before | -16.128 | -3.002 | 0.592 | -2.804 | 2.096 |
| Monthly | After | -0.396 | 0.139 | 0.515 | 0.246 | 0.598 |
Hydrological analysis
Density distribution
Percentiles
| Observed | Uncalibrated | Calibrated | |
|---|---|---|---|
| 1% | 0.04 | 0.861 | 0.021 |
| 5% | 0.06 | 0.995 | 0.057 |
| 10% | 0.07 | 1.111 | 0.089 |
| 15% | 0.08 | 1.227 | 0.116 |
| 25% | 0.12 | 1.404 | 0.178 |
| 50% | 0.26 | 2.006 | 0.474 |
| 75% | 0.56 | 2.910 | 1.146 |
| 85% | 0.89 | 3.576 | 1.618 |
| 90% | 1.22 | 4.132 | 2.018 |
| 95% | 1.82 | 5.035 | 2.582 |
| 99% | 3.53 | 7.443 | 4.298 |
AIGUADORA watershed with TETIS and version 2.5.2
Goodness-of-fit
| Scale | Calibration | NSE | KGE | d | VE | RMSE |
|---|---|---|---|---|---|---|
| Daily | Before | -7.688 | -1.721 | 0.543 | -2.851 | 2.257 |
| Daily | After | -0.388 | 0.306 | 0.466 | 0.061 | 0.902 |
| Monthly | Before | -16.128 | -3.002 | 0.592 | -2.804 | 2.096 |
| Monthly | After | -0.396 | 0.139 | 0.515 | 0.246 | 0.598 |
Hydrological analysis
Density distribution
Percentiles
| Observed | Uncalibrated | Calibrated | |
|---|---|---|---|
| 1% | 0.04 | 0.861 | 0.021 |
| 5% | 0.06 | 0.995 | 0.057 |
| 10% | 0.07 | 1.111 | 0.089 |
| 15% | 0.08 | 1.227 | 0.116 |
| 25% | 0.12 | 1.404 | 0.178 |
| 50% | 0.26 | 2.006 | 0.474 |
| 75% | 0.56 | 2.910 | 1.146 |
| 85% | 0.89 | 3.576 | 1.618 |
| 90% | 1.22 | 4.132 | 2.018 |
| 95% | 1.82 | 5.035 | 2.582 |
| 99% | 3.53 | 7.443 | 4.298 |