Harnessing high-level concepts, visual, and auditory features for violence detection in videos

Autor(es)

Peixoto, Bruno M. ^[1] ; Lavi, Bahram ^[1] ; Dias, Zanoni ^[1] ; Rocha, Anderson ^[1]

Afiliação do(s) autor(es)

^[1] Univ Estadual Campinas, Inst Comp, BR-13083852 Campinas, SP - Brazil

• Tipo de documento: Artigo Científico
• Fonte: JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION; v. 78, JUL 2021.
• Citações Web of Science: 0

Resumo

In detecting sensitive media, violence is one of the hardest to define objectively, and thus, a significant challenge to detect automatically. While many studies were conducted in detecting aspects of violence, very few try to approach the general concept. We propose a method that aims to enable machines to understand a high-level concept of violence by first breaking it down into smaller, more objective ones, such as fights, explosions, blood, and gunshots, to combine them later, leading to a better understanding of the scene. For this, we leverage characteristics of each individual sub-concept of violence (relying upon custom-tailored convolutional neural networks) to guide how they should be described. A fight scene should incorporate temporal features that a scene with blood does not need to describe. A scene with explosions or gunshots should weigh more on its audio features. With this multimodal approach, we trained visual and auditory feature detectors and later combined them into a decision neural network to give us a violence detector that considers several different aspects of the problem. This robust and modular approach allows different cultures and users to adapt the detector to their specific needs. (AU)

• Processo FAPESP: 17/12646-3 - Déjà vu: coerência temporal, espacial e de caracterização de dados heterogêneos para análise e interpretação de integridade
• Beneficiário: Anderson de Rezende Rocha
• Modalidade de apoio: Auxílio à Pesquisa - Temático
• Processo FAPESP: 18/05668-3 - Coerência espaço-temporal e de características a partir de dados heterogêneos
• Beneficiário: Bahram Lavi Sefidgari
• Modalidade de apoio: Bolsas no Brasil - Pós-Doutorado