Event reconstruction from heterogeneous visual data

• Grant number: 19/04053-8
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: August 01, 2019
• End date: June 27, 2024
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Anderson de Rezende Rocha
• Grantee: Jing Yang
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Associated research grant: 17/12646-3 - Déjà vu: feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events, AP.TEM
• Associated scholarship(s): 22/05002-0 - Rethinking fake news detection for the real world, BE.EP.DR

Abstract

The widespread use of camera systems and the popularity of multimedia sharing websites such as Twitter, Flickr, and YouTube have resulted in an enormous amount of publicly available image and video data. Organizing and browsing such large-scale visual data is, more than ever, a pertinent problem in computer vision. As a subset of this problem, event reconstruction is of great significance because of its potential applications: it can be used for reviewing large-scale sport matches and concert in 3D contexts; experiencing augmented visual tour of historical moments; recovering large areas that were hit by man-made or natural disasters; and to enable crime scene reconstruction for investigators to hunt criminals. In this research project, we aim to investigate methods for organizing, integrating and visualizing visual data to reconstruct historic events. We divide the reconstruction into three stages: 1) collecting and organizing visual data; 2) integrating visual data, including image/video stitching and video sequences recomposing, and 3) visualizing diverse data, which includes exploring different means for 3D reconstruction, projecting visual data onto a 3D environment and designing a navigation system for investigators to understand an event faster and more effectively. After event reconstruction, investigators can further leverage the result for many other things, such as human/objects tracking, behavior analysis, fact-checking, or event understanding. (AU)

Scientific publications

(6)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

YANG, JING; ROCHA, ANDERSON. Take It Easy: Label-Adaptive Self-Rationalization for Fact Verification and Explanation Generation. 2024 IEEE INTERNATIONAL WORKSHOP ON INFORMATION FORENSICS AND SECURITY, WIFS 2024, v. N/A, p. 6-pg., 2024-01-01. (23/12865-8, 19/04053-8)

YANG, JING; GLOCKNER, MAX; ROCHA, ANDERSON; GUREVYCH, IRYNA. Self-Rationalization in the Wild: A Large-scale Out-of-Distribution Evaluation on NLI-related tasks. TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, v. 13, p. 29-pg., 2025-04-03. (19/04053-8, 23/12865-8, 22/05002-0)

YANG, JING; VEGA-OLIVEROS, DIDIER; SEIBT, TAIS; ROCHA, ANDERSON; IEEE. Scalable Fact-checking with Human-in-the-Loop. 2021 IEEE INTERNATIONAL WORKSHOP ON INFORMATION FORENSICS AND SECURITY (WIFS), v. N/A, p. 6-pg., 2021-01-01. (17/12646-3, 19/26283-5, 19/04053-8)

YANG, JING; VEGA-OLIVEROS, DIDIER; SEIBT, TAIS; ROCHA, ANDERSON; IEEE. EXPLAINABLE FACT-CHECKING THROUGH QUESTION ANSWERING. 2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), v. N/A, p. 5-pg., 2022-01-01. (19/26283-5, 17/12646-3, 19/04053-8)

NASCIMENTO, JOSE; CARDENUTO, JOAO PHILLIPE; YANG, JING; ROCHA, ANDERSON; IEEE. Few-shot Learning for Multi-modal Social Media Event Filtering. 2022 IEEE INTERNATIONAL WORKSHOP ON INFORMATION FORENSICS AND SECURITY (WIFS), v. N/A, p. 6-pg., 2022-01-01. (17/12646-3, 20/02241-9, 20/02211-2, 19/04053-8)

CARDENUTO, JOAO PHILLIPE; YANG, JING; PADILHA, RAFAEL; WAN, RENJIE; MOREIRA, DANIEL; LI, HAOLIANG; WANG, SHIQI; ANDALO, FERNANDA; MARCEL, SEBASTIEN; ROCHA, ANDERSON. The Age of Synthetic Realities: Challenges and Opportunities. APSIPA TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING, v. 12, n. 1, p. 62-pg., 2023-01-01. (22/05002-0, 17/12646-3, 19/04053-8, 20/02211-2)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

YANG, Jing. Combatendo desinformação com verificação factual eficiente e explicável. 2024. Doctoral Thesis - Universidade Estadual de Campinas (UNICAMP). Instituto de Computação Campinas, SP.