3-D scene understanding for agriculture using deep neural networks

• Grant number: 19/07863-0
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Effective date (Start): July 01, 2019
• Effective date (End): February 28, 2021
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Acordo de Cooperação: IBM Brasil
• Principal Investigator: Thiago Teixeira Santos
• Grantee: Marcos Gabriel Barboza Dure Diaz
• Host Institution: Embrapa Informática Agropecuária. Empresa Brasileira de Pesquisa Agropecuária (EMBRAPA). Ministério da Agricultura, Pecuária e Abastecimento (Brasil). Campinas , SP, Brazil
• Host Company: Ministério da Agricultura, Pecuária e Abastecimento (Brasil). Empresa Brasileira de Pesquisa Agropecuária (EMBRAPA). Embrapa Informática Agropecuária
• Associated research grant: 17/19282-7 - Ambient-awareness in agriculture: 3-D structure and reasoning in the crop field (AACr3), AP.PITE

Abstract

To operate autonomously in an unstructured, complex environment like agricultural crop fields, machines need 3-D scene understanding: a machine reasoning-based process to detect structures of interest and their three-dimensional spatial organization and pose. The recent success of deep neural networks (DNN) in several artificial intelligence problems has made some researchers to say that /Deep Learning is the master of perception/ in current computer vision research. However, beside the large number of works in tasks involving images, audio or text, there is a lack of deep learning-based solution for recognition tasks in 3-D data processing like three-dimensional point clouds. Recently, Qi et al. proposed a novel neural net architecture for classification and segmentation tasks in point clouds, named PointNet, possibly one of the first works for segmentation and classification of three-dimensional structures represented as point clouds. This Scientific Initiation (IC) grant has as goal introduce an undergraduate student to DNN-based supervised learning. The student will help the AACr3 team on (I) developing an annotation tool for 3-D data annotation; (II) train a deep neural network architecture, based on PointNet, for segmentation and classification problems in 3-D point clouds of real crop plots and (III) evaluate the solution using test data.