Development of a platform for specific patient surgery training using augmented reality and additive manufacturing

• Grant number: 20/05306-4
• Support Opportunities: Research Grants - Innovative Research in Small Business - PIPE
• Start date: February 01, 2021
• End date: January 31, 2023
• Field of knowledge: Health Sciences - Medicine - Surgery
• Principal Investigator: Carlo Rondinoni
• Grantee: Carlo Rondinoni
• Company: TagmaVisions Treinamentos Digitais Ltda
• CNAE: Impressão de materiais para outros usos ; Desenvolvimento e licenciamento de programas de computador não-customizáveis
• Associated researchers: Antonio Adilton Oliveira Carneiro ; Helio Rubens Machado ; Helton Luiz Aparecido Defino ; Oswaldo Baffa Filho
• Associated research grant: 18/25348-3 - Technical-commercial viability analysis of an augmented reality and additive manufacturing application for surgery training in patient-specific anatomy, AP.PIPE
• Associated scholarship(s): 21/13288-9 - Development of a platform for training in specific patient surgeries using augmented reality and additive manufacturing, BP.TT; 21/01951-5 - Development of a platform for training in specific patient surgeries using augmented reality and additive manufacturing, BP.TT; 21/01887-5 - Development of a platform for training in specific patient surgeries using augmented reality and additive manufacturing, BP.PIPE

Abstract

The creation of new forms of specialized and interactive medical training is important when considering the scenario related to mortality linked to preventable events. According to the 2nd Yearbook of Hospital Assistance Safety in Brazil, in 2017, approximately 98 patients died each day due to serious and preventable events. During PIPE Phase I, we applied a questionnaire to find the real market demand for this type of product. In addition, we asked the experts' opinion regarding interaction tools available in a prototype with simple functionalities. In this next phase, the team aims to completely develop a product based on digital visualization and additive manufacturing (MA) techniques to create a complex surgery training platform. Therefore, a patient's anatomy will be reproduced simultaneously in a 3D prototype and in an interactive digital visualization. On the one hand, augmented reality (AR) provides indicators for the surgeon to visualize the internal anatomy to act externally. On the other hand, the use of additive manufacturing allows the mimicking of individual cases in training, including aspects of a single anatomy of a specific patient to be treated. Our goal is to develop the aspects of industrial, commercial and marketing design of a product that is based on the application of augmented reality and additive manufacturing in training in endoscopic surgery of the human spine. Thus, complex cases can serve to train a large number of students, reducing the need for corpses. To this end, we present a development plan that includes: 1. A 3D printed physical training platform that uses augmented reality on mobile devices; 2. A digital platform that uses instructional design to integrate knowledge into the training platform; 3. A market survey in the specific spine surgery training area; and 4. A business model design capable of managing knowledge at a level of sufficient complexity to meet this demand with a high degree of specialization. The commercial project should consider the best way to combine the reproduction of the anatomy of the bones of the spine both in a physical and virtual three-dimensional model viewed in superposition to the former. During the project, we will seek subsidies to develop an intelligent pipeline that accelerates the processing of images and the production of anatomical models through 3D printing. This pipeline should be able to generate three-dimensional models of bone anatomy in a semi-automatic way using machine learning models. The further processing will then depend not only on the entry of medical image data but also on the information and knowledge generated in the briefing with the client. The next step will be the integration of this segmentation with the augmented reality application, which will receive the three-dimensional meshes and integrate them with the printed model. The products of each iterative phase of development will be tested for their presentation as teaching resources in the 2020 classes of the Continuing Education Endoscopy Course at USP in Ribeirão Preto. In each module of the course, consecutive cycles of building - measuring - learning will be carried out with the least possible effort and development time possible. (AU)