| Full text | |
| Author(s): Show less - |
Machicao, J.
;
Ben Abbes, A.
;
Meneguzzi, L.
;
Correa, P. L. P.
;
Specht, A.
;
David, R.
;
Subsol, G.
;
Vellenich, D.
;
Devillers, R.
;
Stall, S.
;
Mouquet, N.
;
Chaumont, M.
;
Berti-Equille, L.
;
Mouillot, D.
Total Authors: 14
|
| Document type: | Journal article |
| Source: | EARTH AND SPACE SCIENCE; v. 9, n. 8, p. 16-pg., 2022-08-01. |
| Abstract | |
The challenges of Reproducibility and Replicability (R & R) in computer science experiments have become a focus of attention in the last decade, as efforts to adhere to good research practices have increased. However, experiments using Deep Learning (DL) remain difficult to reproduce due to the complexity of the techniques used. Challenges such as estimating poverty indicators (e.g., wealth index levels) from remote sensing imagery, requiring the use of huge volumes of data across different geographic locations, would be impossible without the use of DL technology. To test the reproducibility of DL experiments, we report a review of the reproducibility of three DL experiments which analyze visual indicators from satellite and street imagery. For each experiment, we identify the challenges found in the data sets, methods and workflows used. As a result of this assessment we propose a checklist incorporating relevant FAIR principles to screen an experiment for its reproducibility. Based on the lessons learned from this study, we recommend a set of actions aimed to improve the reproducibility of such experiments and reduce the likelihood of wasted effort. We believe that the target audience is broad, from researchers seeking to reproduce an experiment, authors reporting an experiment, or reviewers seeking to assess the work of others. (AU) | |
| FAPESP's process: | 18/24017-3 - Building new tools for data sharing and re-use through a transnational investigation of the socioeconomic impacts of protected areas (PARSEC) |
| Grantee: | Pedro Luiz Pizzigatti Corrêa |
| Support Opportunities: | Regular Research Grants |
| FAPESP's process: | 20/03514-9 - Evaluation the effects of Brazilian protected areas in local communities based on the use and re-use of biological, environmental and socioeconomic data |
| Grantee: | Marina Jeaneth Machicao Justo |
| Support Opportunities: | Scholarships in Brazil - Post-Doctoral |