Intelligent Multi-stakeholder market research classifier using BERTimbau

• Grant number: 21/13418-0
• Support Opportunities: Research Grants - Innovative Research in Small Business - PIPE
• Start date: October 01, 2023
• End date: June 30, 2024
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Pedro Ernesto Pereira Paro
• Grantee: Pedro Ernesto Pereira Paro
• Company: Humanizadas Desenvolvimento de Software Ltda
• CNAE: Pesquisas de mercado e de opinião pública
• City: Itu
• Associated researchers: Bruno Tonet ; Rosival Rodrigues do Nascimento Neto
• Associated research grant(s): 24/13522-0 - Sustainable Digital Transformation: AI Innovations for the Humanizadas Platform, AP.PIPE
• Associated scholarship(s): 23/13887-5 - Planning and implementation of Cloud Computing structure and Web front-end for a multi-stakeholder market research company., BP.TT; 23/12432-4 - Design and implementation of a BERTimbau algorithm for a multi-stakeholder market research company., BP.TT

Abstract

Market research companies often have essay (open) questions in their survey instruments that need to be sorted in order to make up a more robust part of the results analysis. To gain scalability, these companies need to automate this process as much as possible so as not to depend entirely on hiring a specialized team as their demand increases. Artificial intelligence algorithms, particularly NLP (Natural Language Processing), are being used to automate textual analysis, but they require a large amount of pre-sorted records to be efficient. New Research Companies (such as startups) hardly have enough data to use traditional NLP techniques. Recently, techniques called Transformers model have been developed, which show themselves as an efficient model for reducing data requirements and improving the performance of neural networks. The BERTimbau model allows the use of a generic pre-trained base in Brazilian Portuguese that can pass its parameters to a smaller base to perform a specific NLP task. Considering this scenario, it is intended to implement four NLP classifiers for the analysis of essay questions for market research companies. (AU)