A big data platform for media aggregation and intelligence

• Grant number: 17/12888-7
• Support type: Research Grants - Innovative Research in Small Business - PIPE
• Duration: April 01, 2018 - March 31, 2020
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal researcher: Thiago Barros Rodrigues Costa
• Grantee: Thiago Barros Rodrigues Costa
• Company: Nervera Serviços de Informática Ltda. - ME
• CNAE: Desenvolvimento e licenciamento de programas de computador customizáveis
• City: São Paulo
• Associated research grant: 14/22802-4 - A social media monitoring platform: large scale natural language processing using Hadoop, AP.PIPE

Abstract

This research project aims to develop a Big Data system for analyzing media: news articles, videos, images, and social media posts. This system constitutes the foundation of a complex product for media aggregation and curation that relies heavily on advanced analytics. With the financing from the first phase of PIPE and additional Nervera funding, we already built a distributed system that collects and analyzes media content in real time and leverages natural language processing to curate basic channels. In the second phase of PIPE, we plan to further develop this system and complement it with advanced intelligence tools focused on content and user monitoring. These tools depend on advancing our research in three interconnected fields: big data engineering, data science, and computational psychology. The main research challenge for the second phase will be that of developing a content recommendation system that is based on dynamic user behavior. The commercial application of our research is Catalyst, a product that integrates a B2C news aggregator that allows for advanced content customization for each user with a B2B clipping system that allows corporate and public sector clients to monitor specific issues and entities. (AU)