Multimedia Phylogeny Forest Reconstruction: Recovering the ancestry relationship of Images, Videos and Text Documents

• Grant number: 14/03535-5
• Support Opportunities: Scholarships in Brazil - Post-Doctoral
• Start date: June 01, 2014
• End date: February 29, 2016
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computer Systems
• Principal Investigator: Anderson de Rezende Rocha
• Grantee: Marina Atsumi Oikawa
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil

Abstract

Nowadays, several cases of tampered digital media exist, depicting events that did not happen exactly the way they were reported, with some retouching, content inclusion or removal, among other types of operations. Whether these modifications were performed only for entertainment purposes or to forge digital evidence in crime investigations, it is undeniable the increasing importance of multimedia forensics research. In some cases, to simply investigate if one document is a duplicate of another is not enough; knowledge about the history of modifications is also important, since it can give us hints about its original creator and a better understanding of how the content was transformed along its lifetime. From this demand, a new research field called multimedia phylogeny has arisen, aiming at investigating how to infer the relationships among objects that belong to the same population. These relationships, represented by means of directed acyclic graphs, resemble the phylogenetic trees used in Biology and are a convenient way to describe the order and the chain of transformations used to create the digital object under investigation. In this research project, we aim at broadening the current state-of-the-art of multimedia phylogeny algorithms, by improving its robustness and expanding them from the image domain to videos and text documents. As a first step, we will target improvements on the calculation of the dissimilarity among the related objects, followed by new approaches for reconstructing phylogeny forests, that is, cases in which we have multiple objects with similar semantic content, but that might not be directly related to each other, being necessary to correctly place them in different processing trees. Furthermore, since most of the approaches developed so far are mainly applied to images, it is paramount to develop new methods to attend the particularities of videos and, in special, text documents, a type of media least explored and whose properties differ in many levels to images and videos. Other expected contributions of this research include the creation of new datasets and development of additional evaluation metrics when necessary.

Scientific publications

(4)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

OIKAWA, MARINA A.; DIAS, ZANONI; ROCHA, ANDERSON; GOLDENSTEIN, SIOME. Distances in multimedia phylogeny. International Transactions in Operational Research, v. 23, n. 5, SI, p. 921-946, SEP 2016. (14/19401-8, 14/03535-5, 13/08293-7)

OIKAWA, MARINA A.; DIAS, ZANONI; ROCHA, ANDERSON DE REZENDE; GOLDENSTEIN, SIOME. Manifold Learning and Spectral Clustering for Image Phylogeny Forests. IEEE Transactions on Information Forensics and Security, v. 11, n. 1, p. 5-18, JAN 2016. (14/03535-5, 14/19401-8)

MARMEROLA, GUILHERME D.; OIKAWA, MARINA A.; DIAS, ZANONI; GOLDENSTEIN, SIOME; ROCHA, ANDERSON. On the Reconstruction of Text Phylogeny Trees: Evaluation and Analysis of Textual Relationships. PLoS One, v. 11, n. 12, DEC 19 2016. (14/19401-8, 15/19222-9, 14/13433-5, 13/08293-7, 14/03535-5)

OIKAWA, MARINA A.; DIAS, ZANONI; ROCHA, ANDERSON; GOLDENSTEIN, SIOME. Distances in multimedia phylogeny. International Transactions in Operational Research, v. 23, n. 5, p. 26-pg., 2016-09-01. (14/19401-8, 13/08293-7, 14/03535-5)