SPAEF version 1.0 with histogram match
Mehmet Cüneyd Demirel · Zenodo (CERN European Organization for Nuclear Research) · 2018
Developing new metrics for spatial pattern comparison of observed and simulated variables has been planned as major part of the first work package (WP1) of the SPACE project. This goal was achieved by developing SPAtial EFficiency metric (SPAEF) described below. For that first we have extensively investigated the current state-of-art spatial comparison metrics. We tested available metrics such as coefficient of variation, correlation coefficient, Goodman and Kruskal's lambda [1], Mapcurves [2], agreement coefficient [3], FSS, Theil's Uncertainty, EOF and Cramér's V [4]–[6] in calibration of a distributed hydrologic model. However, after many calibration attempts we found them inadequate due to the unexpected resultant visual patterns resulted in high correlation but too low standard deviation or highly separate groups. Therefore, we introduced a brand-new multi-component SPAEF metric in Demirel et al. [7] which showed the utility of SPAEF in an ensemble model calibration case. For further comparison of the SPAEF with other metrics and particularly analyzing each component separately in a calibration framework please refer to the subsequent study by Koch et al.[8]. Illustrative examples and source codes are available in Python, R and Matlab [9]. SPAEF=1-[(A-1)^2+(B-1)^2+(C-1)^2]^(1/2) A=Correlation Coefficient (O,S) B=COV(O) / COV(S) C=Histogram Match (O,S) References: [1] L. A. Goodman and W. H. Kruskal, “Measures of Association for Cross Classifications*,” J. Am. Stat. Assoc., vol. 49, no. 268, pp. 732–764, Dec. 1954, doi: 10.1080/01621459.1954.10501231. [2] W. W. Hargrove, F. M. Hoffman, and P. F. Hessburg, “Mapcurves: a quantitative method for comparing categorical maps,” J. Geogr. Syst., vol. 8, no. 2, pp. 187–208, Jul. 2006, doi: 10.1007/s10109-006-0025-x. [3] L. Ji and K. Gallo, “An agreement coefficient for image comparison,” Photogramm. Eng. Remote Sensing, vol. 72, no. 7, pp. 823–833, 2006. [4] H. Cramér, Mathematical Methods of Statistics. Princeton University Press, 1946. [5] J. Koch, K. H. Jensen, and S. Stisen, “Toward a true spatial model evaluation in distributed hydrological modeling: Kappa statistics, Fuzzy theory, and EOF-analysis benchmarked by the human perception and evaluated against a modeling case study,” Water Resour. Res., vol. 51, no. 2, pp. 1225–1246, Feb. 2015, doi: 10.1002/2014WR016607. [6] W. G. Rees, “Comparing the spatial content of thematic maps,” Int. J. Remote Sens., vol. 29, no. 13, pp. 3833–3844, Jul. 2008, doi: 10.1080/01431160701852088. [7] M. C. Demirel, J. Mai, G. Mendiguren, J. Koch, L. Samaniego, and S. Stisen, “Combining satellite data and appropriate objective functions for improved spatial pattern performance of a distributed hydrologic model,” Hydrol. Earth Syst. Sci., vol. 22, no. 2, pp. 1299–1315, Feb. 2018, doi: 10.5194/hess-22-1299-2018. [8] J. Koch, M. C. Demirel, and S. Stisen, “The SPAtial EFficiency metric (SPAEF): multiple-component evaluation of spatial patterns for optimization of hydrological models,” Geosci. Model Dev., vol. 11, no. 5, pp. 1873–1886, May 2018, doi: 10.5194/gmd-11-1873-2018. [9] M. C. Demirel, J. Koch, and S. Stisen, “SPAEF: SPAtial EFficiency,” GitHub. GEUS, Copenhagen, 25-Jan-2017, doi: 10.5281/ZENODO.1158890.