Advanced search
Start date

Déjà vu: feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events

Grant number: 17/12646-3
Support type:Research Projects - Thematic Grants
Duration: December 01, 2017 - November 30, 2022
Field of knowledge:Physical Sciences and Mathematics - Computer Science
Principal Investigator:Anderson de Rezende Rocha
Grantee:Anderson de Rezende Rocha
Home Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas, SP, Brazil
Co-Principal Investigators:Hélio Pedrini
Assoc. researchers: Adam Maciej Czajka ; Alexandre Mello Ferreira ; André Carlos Ponce de Leon Ferreira de Carvalho ; Eduardo Alves Do Valle Junior ; Edward John Delp Iii ; Fernanda Alcântara Andaló ; Gerberth Adín Ramírez Rivera ; Jacques Wainer ; Kevin Wilson Bowyer ; Kot Chi Chung Alex ; Lin Tzy Li ; Marcos André Gonçalves ; Paolo Bestagini ; Patrick Joseph Flynn ; Sandra Eliza Fontes de Avila ; Stefano Tubaro ; Walter Jerome Scheirer ; William Robson Schwartz ; Zanoni Dias
Associated scholarship(s):18/10204-6 - Fighting "fake news" through authorship attribution and phylogeny analysis, BP.DR
17/21957-2 - Learning visual clues of the passage of time, BP.DR
18/05668-3 - Feature-space-time coherence with heterogeneous data, BP.PD


In this research project, we focus on synchronizing specific events in space and time (X-coherence), fact-checking, and mining persons, objects and contents of interest from various and heterogeneous sources including - but not limited to - the internet, social media and surveillance imagery. For that, we seek to harness information from various media sources and synchronize the multiple textual and visual information pieces around the position of an event or object as well as order them so as to allow a better understanding about what happened before, during, and shortly after the event. After automatically organizing the data and understanding the order of the facts, we can devise and deploy solutions for mining persons or objects of interest for suspect analysis/tracking, fact checking, or even understanding the nature of the said event. Additionally, by exploring the possible existing links among different pieces of information, we aim at further designing and developing media integrity analytics tools to hint at existing forgeries, sensitive content (e.g., violent content, child pornography), and spreading patterns of multimedia objects online. With demanding and sophisticated crimes and terrorist threats becoming ever more pervasive, allied with the advent and spread of fake news, our objective is to use the developed solutions to help us answering the four most important questions in forensics regarding an event: "who," "in what circumstances," "why," and "how," thus identifying the characteristics and circumstances in which an event has taken place. (AU)