Pattern discovery and event highlight from heterogeneous sources

• Grant number: 22/13061-7
• Support Opportunities: Scholarships abroad - Research Internship - Doctorate (Direct)
• Start date: January 20, 2023
• End date: July 31, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Anderson de Rezende Rocha
• Grantee: José Dorivaldo Nascimento Souza Júnior
• Supervisor: Paolo Bestagini
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Institution abroad: Politecnico di Milano, Italy
• Associated to the scholarship: 20/02241-9 - Pattern discovery and event highlight from heterogenous sources, BP.DD

Abstract

When a forensic event of large scale happens, immediately visual content related to it is shared on social networks. These data might be important for a posterior forensic inspection, given that they can depict different views in different moments of the event. However, an analysis of social media imagery related to an event might be harmed by the abundant number of irrelevant items retrieved by a collection procedure, such as memes and images from previous events. Manually sanitizing these datasets is unfeasible, as they might contain thousands of items. To tackle this problem, we study designing and developing machine learning techniques to speed up the procedure and reduce the required human force. In detail, we target including humans in the loop of the machine learning pipeline, by employing instance selection and active learning techniques, and performing classification with few annotated samples, employing as starting point semi-supervised techniques that vary from graph-based to self-training methods. These two paths are promising and can improve the performance of methods for this task. (AU)