Enhancing of accuracy assessment for forest above-ground biomass estimates obtained from remote sensing via hypothesis testing and overfitting evaluation

Autor(es)

Valbuena, R. ^{[1, 2]} ; Hernando, A. ^[3] ; Manzanera, J. A. ^[3] ; Gorgens, E. B. ^[4] ; Almeida, D. R. A. ^[5] ; Mauro, F. ^[3] ; Garcia-Abril, A. ^[3] ; Coomes, D. A. ^[1]

Afiliação do(s) autor(es)

^[1] Univ Cambridge, Dept Plant Sci Forest Ecol & Conservat, Downing St, Cambridge CB2 3EA - England

^[2] Univ Eastern Finland, Fac Forest Sci, POB 111, Joensuu - Finland

^[3] Univ Politecn Madrid, Coll Forestry & Nat Environm, Res Grp SILVANET, Ciudad Univ, E-28040 Madrid - Spain

^[4] Univ Fed Vales Jequitinhonha & Mucuri, Dept Forestry, Campus JK, Diamantina - Brazil

^[5] Univ Sao Paulo, Dept Forest Sci, Luiz de Queiroz Coll Agr, Av Padua Dias 11, BR-13418900 Piracicaba - Brazil

• Tipo de documento: Artigo Científico
• Fonte: ECOLOGICAL MODELLING; v. 366, p. 15-26, DEC 24 2017.
• Citações Web of Science: 14

Resumo

The evaluation of accuracy is essential for assuring the reliability of ecological models. Usually, the accuracy of above-ground biomass (AGB) predictions obtained from remote sensing is assessed by the mean differences (MD), the root mean squared differences (RMSD), and the coefficient of determination (R-2) between observed and predicted values. In this article we propose a more thorough analysis of accuracy, including a hypothesis test to evaluate the agreement between observed and predicted values, and an assessment of the degree of overfitting to the sample employed for model training. Using the estimation of forest AGB from LIDAR and spectral sensors as a case study, we compared alternative prediction and variable selection methods using several statistical measures to evaluate their accuracy. We showed that the hypothesis tests provide an objective method to infer the statistical significance of agreement. We also observed that overfitting can be assessed by comparing the inflation in residual sums of squares experienced when carrying out a cross-validation. Our results suggest that this method may be more effective than analysing the deflation in R-2. We proved that overfitting needs to be specifically addressed since, in light of MD, RMSD and R-2 alone, predictions may apparently seem reliable even in clearly unre-alistic circumstances, for instance when including too many predictor variables. Moreover, Theil's partial inequality coefficients, which are employed to resolve the proportions of the total errors due to the unexplained variance, the slope and the bias, may become useful to detect averaging effects common in remote sensing predictions of AGB. We concluded that statistical measures of accuracy, precision and agreement are necessary but insufficient for model evaluation. We therefore advocate for incorporating evaluation measures specifically devoted to testing observed-versus-predicted fit, and to assessing the degree of overfitting. (C) 2017 Elsevier B.V. All rights reserved. (AU)

• Processo FAPESP: 16/05219-9 - Monitoramento de programas de restauração de paisagens florestais por meio de sensoriamento remoto LiDAR.
• Beneficiário: Danilo Roberti Alves de Almeida
• Modalidade de apoio: Bolsas no Brasil - Doutorado