3D human pose estimation based on monocular RGB images and domain adaptation

• Grant number: 22/07055-4
• Support Opportunities: Scholarships abroad - Research Internship - Master's degree
• Effective date (Start): September 30, 2022
• Effective date (End): February 28, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Aparecido Nilceu Marana
• Grantee: João Renato Ribeiro Manesco
• Supervisor: Stefano Berretti
• Host Institution: Faculdade de Ciências (FC). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil
• Research place: Università degli Studi di Firenze, Italy
• Associated to the scholarship: 21/02028-6 - 3D human pose estimation based on monocular RGB images and domain adaptation, BP.MS

Abstract

Monocular human pose estimation is an important Computer Vision problem, aimed to estimate the human body shape based on a single RGB image. Currently, methods that employ deep learning techniques excel in the task of 2D human pose estimation, which can be used in a diverse set of applications. However, the use of 3D poses can bring more accurate and robust results. As 3D pose annotations are difficult to obtain, fully convolutional methods tend to perform poorly, therefore, a solution of estimating 3D poses based on the 2D pose previously obtained was proposed, offering improved performance by delegating the exploration of images features to more mature 2D pose estimation techniques. Due to database acquisition constraints, this performance improvement is only observed in controlled scenarios, therefore domain adaptation techniques can be used to increase the generalization capability of the system by inserting new actions and camera angles from external databases. The goal of this project is to propose a domain adaptation framework to work with 2D-based 3D pose estimation, to simplify the use of synthetic datasets during training, thus improving the generalization capability of the network as well as its real-life performance. (AU)