Advanced search
Start date
Betweenand

Déjà vu: feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events

Grant number: 17/12646-3
Support type:Research Projects - Thematic Grants
Duration: December 01, 2017 - November 30, 2022
Field of knowledge:Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
Principal Investigator:Anderson de Rezende Rocha
Grantee:Anderson de Rezende Rocha
Home Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
Co-Principal Investigators:Hélio Pedrini
Assoc. researchers: Adam Maciej Czajka ; Alexandre Mello Ferreira ; André Carlos Ponce de Leon Ferreira de Carvalho ; Eduardo Alves Do Valle Junior ; Edward John Delp Iii ; Fernanda Alcântara Andaló ; Gerberth Adín Ramírez Rivera ; Jacques Wainer ; Kevin Wilson Bowyer ; Kot Chi Chung Alex ; Lin Tzy Li ; Marcos André Gonçalves ; Paolo Bestagini ; Patrick Joseph Flynn ; Sandra Eliza Fontes de Avila ; Stefano Tubaro ; Walter Jerome Scheirer ; William Robson Schwartz ; Zanoni Dias
Associated scholarship(s):19/26283-5 - Learning visual clues of the passage of time, BP.PD
20/02211-2 - Provenance filtering and analysis, BP.DD
20/02241-9 - Pattern discovery and event highlight from heterogenous sources, BP.DD
+ associated scholarships 19/04053-8 - Event reconstruction from heterogeneous visual data, BP.DR
19/15825-1 - Mining persons, objects and places of interest from heterogeneous data sources, BP.DD
18/16214-3 - Heterogeneous data analysis for event detection, BP.MS
18/15864-4 - Scientific integrity: analysis of Misconduct in images of scientific papers, BP.IC
18/18264-8 - Monitoring of events in social networks for data collection, information analysis and temporal synchronization, BP.IC
18/10204-6 - Fighting "fake news" through authorship attribution and phylogeny analysis, BP.DR
18/16548-9 - Learning visual clues of the passage of time, BP.PD
17/21957-2 - Learning visual clues of the passage of time, BP.DR
18/05668-3 - Feature-space-time coherence with heterogeneous data, BP.PD - associated scholarships

Abstract

In this research project, we focus on synchronizing specific events in space and time (X-coherence), fact-checking, and mining persons, objects and contents of interest from various and heterogeneous sources including - but not limited to - the internet, social media and surveillance imagery. For that, we seek to harness information from various media sources and synchronize the multiple textual and visual information pieces around the position of an event or object as well as order them so as to allow a better understanding about what happened before, during, and shortly after the event. After automatically organizing the data and understanding the order of the facts, we can devise and deploy solutions for mining persons or objects of interest for suspect analysis/tracking, fact checking, or even understanding the nature of the said event. Additionally, by exploring the possible existing links among different pieces of information, we aim at further designing and developing media integrity analytics tools to hint at existing forgeries, sensitive content (e.g., violent content, child pornography), and spreading patterns of multimedia objects online. With demanding and sophisticated crimes and terrorist threats becoming ever more pervasive, allied with the advent and spread of fake news, our objective is to use the developed solutions to help us answering the four most important questions in forensics regarding an event: "who," "in what circumstances," "why," and "how," thus identifying the characteristics and circumstances in which an event has taken place. (AU)

Scientific publications (32)
(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)
ROBERTO E SOUZA, MARCOS; MAIA, HELENA DE ALMEIDA; VIEIRA, MARCELO BERNARDES; PEDRINI, HELIO. Survey on visual rhythms: A spatio-temporal representation for video sequences. Neurocomputing, v. 402, p. 409-422, AUG 18 2020. Web of Science Citations: 0.
ROBERTO E SOUZA, MARCOS; PEDRINI, HELIO. Visual rhythms for qualitative evaluation of video stabilization. EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING, v. 2020, n. 1 MAY 18 2020. Web of Science Citations: 1.
ALEXANDRINO, ALEXSANDRO OLIVEIRA; LINTZMAYER, CARLA NEGRI; DIAS, ZANONI. Sorting permutations by fragmentation-weighted operations. JOURNAL OF BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, v. 18, n. 2 APR 2020. Web of Science Citations: 0.
PINTO, ALLAN; GOLDENSTEIN, SIOME; FERREIRA, ALEXANDRE; CARVALHO, TIAGO; PEDRINI, HELIO; ROCHA, ANDERSON. Leveraging Shape, Reflectance and Albedo From Shading for Face Presentation Attack Detection. IEEE Transactions on Information Forensics and Security, v. 15, p. 3347-3358, 2020. Web of Science Citations: 0.
BRITO, KLAIRTON LIMA; JEAN, GERALDINE; FERTIN, GUILLAUME; OLIVEIRA, ANDRE RODRIGUES; DIAS, ULISSES; DIAS, ZANONI. Sorting by Genome Rearrangements on Both Gene Order and Intergenic Sizes. JOURNAL OF COMPUTATIONAL BIOLOGY, v. 27, n. 2 DEC 2019. Web of Science Citations: 0.
FERREIRA, ANSELMO; FELIPUSSI, SIOVANI C.; PIRES, RAMON; AVILA, SANDRA; SANTOS, GEISE; LAMBERT, JORGE; HUANG, JIWU; ROCHA, ANDERSON. Eyes in the Skies: A Data-Driven Fusion Approach to Identifying Drug Crops From Remote Sensing Images. IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, v. 12, n. 12, p. 4773-4786, DEC 2019. Web of Science Citations: 0.
MUNOZ, JAVIER VARGAS; GONCALVES, MARCOS A.; DIAS, ZANONI; TORRES, RICARDO DA S. Hierarchical Clustering-Based Graphs for Large Scale Approximate Nearest Neighbor Search. PATTERN RECOGNITION, v. 96, DEC 2019. Web of Science Citations: 0.
QUISPE, RODOLFO; PEDRINI, HELIO. Improved person re-identification based on saliency and semantic parsing with deep neural network models. Image and Vision Computing, v. 92, DEC 2019. Web of Science Citations: 0.
ROBERTO E SOUZA, MARCOS; PEDRINI, HELIO. Motion energy image for evaluation of video stabilization. VISUAL COMPUTER, v. 35, n. 12, p. 1769-1781, DEC 2019. Web of Science Citations: 0.
OLIVEIRA, ANDRE R.; JEAN, GERALDINE; FERTIN, GUILLAUME; DIAS, ULISSES; DIAS, ZANONI. Super short operations on both gene order and intergenic sizes. Algorithms for Molecular Biology, v. 14, n. 1 NOV 5 2019. Web of Science Citations: 0.
OLIVEIRA, ANDRE RODRIGUES; BRITO, KLAIRTON LIMA; DIAS, ULISSES; DIAS, ZANONI. On the Complexity of Sorting by Reversals and Transpositions Problems. JOURNAL OF COMPUTATIONAL BIOLOGY, v. 26, n. 11, p. 1223-1229, NOV 1 2019. Web of Science Citations: 1.
MENDOZA, JULIO; PEDRINI, HELIO. Detection and classification of lung nodules in chest X-ray images using deep convolutional neural networks. COMPUTATIONAL INTELLIGENCE, NOV 2019. Web of Science Citations: 0.
MORAES, THIAGO; AMORIM, PAULO; DA SILVA, JORGE VICENTE; PEDRINI, HELIO. Medical image interpolation based on 3D Lanczos filtering. COMPUTER METHODS IN BIOMECHANICS AND BIOMEDICAL ENGINEERING-IMAGING AND VISUALIZATION, NOV 2019. Web of Science Citations: 0.
KUEHLKAMP, ANDREY; PINTO, ALLAN; ROCHA, ANDERSON; BOWYER, KEVIN W.; CZAJKA, ADAM. Ensemble of Multi-View Learning Classifiers for Cross-Domain Iris Presentation Attack Detection. IEEE Transactions on Information Forensics and Security, v. 14, n. 6, p. 1419-1431, JUN 2019. Web of Science Citations: 4.
PIRES, RAMON; AVILA, SANDRA; WAINER, JACQUES; VALLE, EDUARDO; ABRAMOFF, MICHAEL D.; ROCHA, ANDERSON. A data-driven approach to referable diabetic retinopathy detection. ARTIFICIAL INTELLIGENCE IN MEDICINE, v. 96, n. SI, p. 93-106, MAY 2019. Web of Science Citations: 0.
OLIVEIRA, ALBERTO; OAKLEY, ERIC; TORRES, RICARDO DA SILVA; ROCHA, ANDERSON. Relevance prediction in similarity-search systems using extreme value theory. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, v. 60, p. 236-249, APR 2019. Web of Science Citations: 0.
MENINI, NATHALIA; ALMEIDA, ALEXANDRE E.; LAMPARELLI, RUBENS; LE MAIRE, GUERRIC; DOS SANTOS, JEFERSSON A.; PEDRINI, HELIO; HIROTA, MARINA; TORRES, RICARDO DA S. A Soft Computing Framework for Image Classification Based on Recurrence Plots. IEEE Geoscience and Remote Sensing Letters, v. 16, n. 2, p. 320-324, FEB 2019. Web of Science Citations: 1.
MOREIRA, DANIEL; AVILA, SANDRA; PEREZ, MAURICIO; MORAES, DANIEL; TESTONI, VANESSA; VALLE, EDUARDO; GOLDENSTEIN, SIOME; ROCHA, ANDERSON. Multimodal data fusion for sensitive scene localization. Information Fusion, v. 45, p. 307-323, JAN 2019. Web of Science Citations: 4.
ALEXANDRE FERREIRA; TIAGO CARVALHO; FERNANDA ANDALÓ; ANDERSON ROCHA. Counteracting the contemporaneous proliferation of digital forgeries and fake news. Anais da Academia Brasileira de Ciências, v. 91, p. -, 2019.
FERREIRA, ALEXANDRE; CARVALHO, TIAGO; ANDALO, FERNANDA; ROCHA, ANDERSON. Counteracting the contemporaneous proliferation of digital forgeries and fake news. Anais da Academia Brasileira de Ciências, v. 91, n. 1 2019. Web of Science Citations: 0.
MENDES JUNIOR, PEDRO RIBEIRO; BONDI, LUCA; BESTAGINI, PAOLO; TUBARO, STEFANO; ROCHA, ANDERSON. An In-Depth Study on Open-Set Camera Model Identification. IEEE ACCESS, v. 7, p. 180713-180726, 2019. Web of Science Citations: 0.
SANTOS MIRANDA, GUILHERME HENRIQUE; LINTZMAYER, CARLA NEGRI; DIAS, ZANONI. Sorting Permutations by lambda-Operations. JOURNAL OF UNIVERSAL COMPUTER SCIENCE, v. 25, n. 2, p. 98-121, 2019. Web of Science Citations: 0.
LI, HAOLIANG; HE, PEISONG; WANG, SHIQI; ROCHA, ANDERSON; JIANG, XINGHAO; KOT, ALEX C. Learning Generalized Deep Feature Representation for Face Anti-Spoofing. IEEE Transactions on Information Forensics and Security, v. 13, n. 10, p. 2639-2652, OCT 2018. Web of Science Citations: 13.
NAVARRO, LUIZ C.; NAVARRO, ALEXANDRE K. W.; GREGIO, ANDRE; ROCHA, ANDERSON; DAHAB, RICARDO. Leveraging ontologies and machine-learning techniques for malware analysis into Android permissions ecosystems. COMPUTERS & SECURITY, v. 78, p. 429-453, SEP 2018. Web of Science Citations: 4.
DE REZENDE, EDMAR R. S.; RUPPERT, GUILHERME C. S.; THEOPHILO, ANTONIO; TOKUDA, ERIC K.; CARVALHO, TIAGO. Exposing computer generated images by using deep convolutional neural networks. SIGNAL PROCESSING-IMAGE COMMUNICATION, v. 66, p. 113-126, AUG 2018. Web of Science Citations: 2.
AMORIM, PAULO; MORAES, THIAGO; FAZANARO, DALTON; SILVA, JORGE; PEDRINI, HELIO. Shearlet and contourlet transforms for analysis of electrocardiogram signals. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE, v. 161, p. 125-132, JUL 2018. Web of Science Citations: 5.
SOUZA, MARCOS R.; PEDRINI, HELIO. Digital video stabilization based on adaptive camera trajectory smoothing. EURASIP JOURNAL ON IMAGE AND VIDEO PROCESSING, MAY 30 2018. Web of Science Citations: 1.
NAVARRO, LUIZ C.; NAVARRO, ALEXANDRE K. W.; ROCHA, ANDERSON; DAHAB, RICARDO. Connecting the dots: Toward accountable machine-learning printer attribution methods. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, v. 53, p. 257-272, MAY 2018. Web of Science Citations: 3.
ODIR RODRIGUES MELO, CARLOS FERNANDO; NAVARRO, LUIZ CLAUDIO; DE OLIVEIRA, DIOGO NOIN; GUERREIRO, TATIANE MELINA; LIMA, ESTELA DE OLIVEIRA; DELAFIORI, JEANY; DABAJA, MOHAMED ZIAD; RIBEIRO, MARTA DA SILVA; DE MENEZES, MAICO; MARTINS RODRIGUES, RAFAEL GUSTAVO; MORISHITA, KAREN NODA; ESTEVES, CIBELE ZANARDI; LUCAS DE AMORIM, ALINE LOPES; AOYAGUI, CAROLINE TIEMI; PARISE, PIERINA LORENCINI; MILANEZ, GUILHERME PAIER; DO NASCIMENTO, GABRIELA MANSANO; RIBAS FREITAS, ANDRE RICARDO; ANGERAMI, RODRIGO; MARANHAO COSTA, FABIO TRINDADE; ARNS, CLARICE WEIS; RESENDE, MARIANGELA RIBEIRO; AMARAL, ELIANA; PASSINI JUNIOR, RENATO; RIBEIRO-DO-VALLE, CAROLINA C.; MILANEZ, HELAINE; MORETTI, MARIA LUIZA; PROENCA-MODENA, JOSE LUIZ; AVILA, SANDRA; ROCHA, ANDERSON; CATHARINO, RODRIGO RAMOS; NETWORK, ZIKA-UNICAMP. A Machine Learning Application Based in Random Forest for Integrating Mass Spectrometry-Based Metabolomic Data: A Simple Screening Method for Patients With Zika Virus. FRONTIERS IN BIOENGINEERING AND BIOTECHNOLOGY, v. 6, APR 11 2018. Web of Science Citations: 0.
VITORINO, PAULO; AVILA, SANDRA; PEREZ, MAURICIO; ROCHA, ANDERSON. Leveraging deep neural networks to fight child pornography in the age of social media. JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION, v. 50, p. 303-313, JAN 2018. Web of Science Citations: 8.
CORDOVA NEIRA, MANUEL ALBERTO; MENDES JUNIOR, PEDRO RIBEIRO; ROCHA, ANDERSON; TORRES, RICARDO DA SILVA. Data-Fusion Techniques for Open-Set Recognition Problems. IEEE ACCESS, v. 6, p. 21242-U24, 2018. Web of Science Citations: 3.
SHEN, BINGYU; FORSTALL, CHRISTOPHER W.; ROCHA, ANDERSON DE REZENDE; SCHEIRER, WALTER J. Practical Text Phylogeny for Real-World Settings. IEEE ACCESS, v. 6, p. 41002-41012, 2018. Web of Science Citations: 1.

Please report errors in scientific publications list by writing to: cdi@fapesp.br.