Alignment of 3D medical models in Microsoft Hololens

• Grant number: 21/12132-5
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: April 01, 2022
• End date: March 31, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Paulo Sergio Silva Rodrigues
• Grantee: Guilherme Vieira Silva Gonçalves
• Host Institution: Centro Universitário FEI (UNIFEI). Campus de São Bernardo do Campo. São Bernardo do Campo , SP, Brazil

Abstract

Since its launch in 2016, Microsoft Hololens has led the market as the most advanced Augmented Reality hardware available to the general public. This generation boosted the AR area in several applications. The medical field, for example, has applied these techniques with great enthusiasm, both for teaching and within surgical centers. However, automatic alignment between authentic scene images and 3D models is still a challenge due to the high computational processing. The main obstacle to this alignment is detecting fiducial points in the scene and their correspondence with the projected 3D model. This project proposes the use of new algorithms to detect fiducial points, both based on CNN (Convolutional Neural Network) and compared with the traditional SITF (Scale Invariant Feature Transform) algorithm.(AU)