Prototype of a sensory environment capable of providing an immersive and monitored experience based on internet of things and augmented reality

• Grant number: 20/11995-7
• Support Opportunities: Research Grants - Innovative Research in Small Business - PIPE
• Duration: April 01, 2021 - March 31, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Roberto Celestino Pereira
• Grantee: Roberto Celestino Pereira
• Host Company: NTT Data Brasil Consultoria em TI & Soluções Ltda
• CNAE: Desenvolvimento de programas de computador sob encomenda ; Desenvolvimento e licenciamento de programas de computador customizáveis ; Suporte técnico, manutenção e outros serviços em tecnologia da informação
• City: São Paulo
• Associated researchers: Cristiano Saito de Souza ; Douglas Lima Dantas
• Associated research grant: 18/22405-6 - Development of a sensory environment capable of immersing the client of a banking agency in a digitized environment, based on the principles of ubiquitous computing, Internet of Things and tangible user interfaces, AP.PIPE

Abstract

With the emergence of the Internet of Services, the user now has access to different services that were previously only offered in person, such as Internet Banking and applications, thus, the user began to consider the bank branch to be too bureaucratic and undesirable. However, several procedures such as manually filling documents and contracts still need to be carried out at the agency, as the presence of the user brings the possibility of facilitating the sale of products and services. Then, the evolution of this paradigm led to the emergence of the Internet of Things (IoT), enabling advanced services by interconnecting things (physical and virtual) based on existing and evolving interoperable information and communication technologies, as well as the emergence of a new paradigm where the Internet starts to focus not only on the communication between people and their actions, but also between machines. This network of objects, added to technologies from areas such as Artificial Intelligence, can allow the detection of user needs without making them explicit, obtaining ubiquitous computing. Meanwhile, in the context of the human-computer interface, technologies such as Tangible User Interfaces (TUIs) are present, which use physical elements to represent and manipulate digital data, increasing the degree of user immersion. The technologies described have the potential to improve the user experience and generate different data and metrics that previously would not be possible. To make this happen, an arrangement and a way of how these technologies should be used based on the goals are needed. In this context, the idea of Phygital journey (from English: Physical + Digital) arises, which deals with techniques on how to use different ubiquitous, tangible technologies, concepts of Digital Marketing and Analytics to provide the user with a journey of experiences. This project has the potential to integrate these different technologies within the scope of bank branches in order to provide a personalized phygital experience to users, allowing them to be-come more attractive, friendly, and increase the sales conversion of banking products and ser-vices, with the objective of developing a prototype from the results of the technical feasibility assessment carried out in Phase 1, which provides an immersive and personalized experience to the user, through a set of physical and digital (phygital) experiences that make use of challenging technologies such as Internet of Things (IoT) and mixed reality. Applied to bank branches, this project will bring an innovative opportunity to users that they can have a personalized phygital experience according to their individual profiles, leading to an improvement in the quality of service at bank branches, to a better knowledge of managers in relation to its users through the data resulting from the interactions and, consequently, to the conversion into acquisition of products or services. (AU)