Sensitive media analysis through deep learning architectures

• Grant number: 17/16246-0
• Support Opportunities: Regular Research Grants
• Start date: May 01, 2018
• End date: April 30, 2020
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Sandra Eliza Fontes de Avila
• Grantee: Sandra Eliza Fontes de Avila
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Associated researchers: Anderson de Rezende Rocha ; Virginia Nunes Leal Franqueira ; Zanoni Dias

Abstract

With the growth of video data generated by many devices, such as cameras, smartphones, and CCTVs, allied with the Internet as a fast spreading venue, smart and continuous content filtering becomes paramount. In this context, classification of sensitive media content (e.g., pornography, violence) retains a considerable amount of attention because of its applications: it can be used for detecting, via surveillance cameras, inappropriate behavior; blocking undesired content from being uploaded to general purpose websites (e.g., social networks, online learning platforms, forums), or from being viewed in some places (e.g., schools, workplaces); preventing children from accessing adult content on personal computers, smartphones or smart TVs; and avoiding that improper content is distributed over phones (e.g., sexting). In this project, we aim at designing, developing and deploying solutions for detecting sensitive content in digital images/videos such as pornography, violence and adult content. We will advance the state of the art by investigating deep networks architectures, by proposing new efficient and effective deep learning approaches, and by incorporating mobile solutions for sensitive media analysis. Specifically, we will focus our efforts on: (1) designing and developing novel deep learning approaches for automatically extracting discriminative space-temporal characteristics from videos; and (2) devising new approaches for dealing with sensitive content that incorporate the benefit of robust mid-level representations such as Fisher Vectors with the powerful Deep Learning characterization pipeline. We highlight that our group is at the forefront of such research worldwide being responsible for groundbreaking results associated with digital forensics, machine learning, and computer vision. (AU)

Scientific publications

(45)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

MIRANDA, GUILHERME HENRIQUE SANTOS; ALEXANDRINO, ALEXSANDRO OLIVEIRA; LINTZMAYER, CARLA NEGRI; DIAS, ZANONI. Approximation Algorithms for Sorting lambda-Permutations by lambda-Operations. ALGORITHMS, v. 14, n. 6, JUN 2021. (13/08293-7, 15/11937-9, 17/16871-1, 17/12646-3, 17/16246-0)

BRITO, KLAIRTON LIMA; JEAN, GERALDINE; FERTIN, GUILLAUME; OLIVEIRA, ANDRE RODRIGUES; DIAS, ULISSES; DIAS, ZANONI. Sorting by Genome Rearrangements on Both Gene Order and Intergenic Sizes. JOURNAL OF COMPUTATIONAL BIOLOGY, v. 27, n. 2, DEC 2019. (17/12646-3, 17/16246-0, 15/11937-9)

RODRIGUES, CAROLINE MAZINI; SORIANO-VARGAS, AUREA; LAVI, BAHRAM; ROCHA, ANDERSON; DIAS, ZANONI. Manifold Learning for Real-World Event Understanding. IEEE Transactions on Information Forensics and Security, v. 16, p. 2957-2972, 2021. (18/16548-9, 18/16214-3, 17/16246-0, 15/11937-9, 18/05668-3, 13/08293-7, 17/12646-3, 17/16871-1)

ALEXANDRINO, ALEXSANDRO OLIVEIRA; MIRANDA, GUILHERME HENRIQUE SANTOS; LINTZMAYER, CARLA NEGRI; DIAS, ZANONI. Length-weighted lambda-rearrangement distance. JOURNAL OF COMBINATORIAL OPTIMIZATION, v. 41, n. 3, p. 579-602, APR 2021. (17/16246-0, 15/11937-9, 17/12646-3)

VARGAS MUNOZ, JAVIER A.; DIAS, ZANONI; TORRES, RICARDO DA SILVA. A genetic programming approach for searching on nearest neighbors graphs. MULTIMEDIA TOOLS AND APPLICATIONS, v. 81, n. 16, p. 24-pg., 2022-03-18. (14/12236-1, 17/20945-0, 13/50155-0, 13/50169-1, 17/12646-3, 15/11937-9, 16/50250-1, 15/24494-8, 17/16246-0, 14/50715-9)

BISSOTO, ALCEU; FORNACIALI, MICHEL; VALLE, EDUARDO; AVILA, SANDRA; IEEE. (De)Constructing Bias on Skin Lesion Datasets. 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW 2019), v. N/A, p. 9-pg., 2019-01-01. (17/16246-0)

RIBEIRO, VINICIUS; AVILA, SANDRA; VALLE, EDUARDO; IEEE COMP SOC. Less is More: Sample Selection and Label Conditioning Improve Skin Lesion Segmentation. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW 2020), v. N/A, p. 10-pg., 2020-01-01. (17/16246-0, 19/05018-1)

PEREZ, FABIO; AVILA, SANDRA; VALLE, EDUARDO; IEEE. Solo or Ensemble? Choosing a CNN Architecture for Melanoma Classification. 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW 2019), v. N/A, p. 9-pg., 2019-01-01. (17/16246-0)

DE OLIVEIRA, GABRIEL BIANCHIN; PEDRINI, HELIO; DIAS, ZANONI; IEEE. Fusion of BLAST and Ensemble of Classifiers for Protein Secondary Structure Prediction. 2020 33RD SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI 2020), v. N/A, p. 8-pg., 2020-01-01. (17/12646-3, 19/20875-8, 15/11937-9, 17/16246-0)

DE OLIVEIRA, GABRIEL BIANCHIN; PEDRINI, HELIO; DIAS, ZANONI; XAVIER-JUNIOR, JC; RIOS, RA. Protein Molecular Function Annotation Based on Transformer Embeddings. INTELLIGENT SYSTEMS, PT II, v. 13654, p. 11-pg., 2022-01-01. (17/12646-3, 19/20875-8, 15/11937-9, 17/16246-0)

DANTAS, ANA PAULA S.; DE SOUZA, CID C.; DIAS, ZANONI. A heuristic for the convex recoloring problem in graphs. International Transactions in Operational Research, v. 29, n. 3, p. 1454-1478, MAY 2022. (17/16246-0, 17/12646-3, 15/11937-9, 18/04760-3)

OLIVEIRA, ANDRE RODRIGUES; JEAN, GERALDINE; FERTIN, GUILLAUME; BRITO, KLAIRTON LIMA; BULTEAU, LAURENT; DIAS, ULISSES; DIAS, ZANONI. Sorting Signed Permutations by Intergenic Reversals. IEEE-ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS, v. 18, n. 6, p. 2870-2876, NOV 1 2021. (13/08293-7, 19/27331-3, 17/16246-0, 17/12646-3)

ISHIKAWA, AKARI; BOLLIS, EDSON; AVILA, SANDRA; IEEE. Combating the Elsagate Phenomenon: Deep Learning Architectures for Disturbing Cartoons. 2019 7TH INTERNATIONAL WORKSHOP ON BIOMETRICS AND FORENSICS (IWBF), v. N/A, p. 6-pg., 2019-01-01. (17/16246-0)

TEIXEIRA, VINICIUS; BRAZ, LEODECIO; PEDRINI, HELIO; DIAS, ZANONI; PAIVA, AC; CONCI, A; BRAZ, G; ALMEIDA, JDS; FERNANDES, LAF. DuaLAnet: Dual Lesion Attention Network for Thoracic Disease Classification in Chest X-Rays. PROCEEDINGS OF THE 2020 INTERNATIONAL CONFERENCE ON SYSTEMS, SIGNALS AND IMAGE PROCESSING (IWSSIP), 27TH EDITION, v. N/A, p. 6-pg., 2020-01-01. (17/16246-0, 19/20875-8, 17/12646-3, 15/11937-9)

ALEXANDRINO, ALEXSANDRO OLIVEIRA; SANTOS MIRANDA, GUILHERME HENRIQUE; LINTZMAYER, CARLA NEGRI; DIAS, ZANONI. Approximation Algorithms for Sorting Permutations by Length-Weighted Short Rearrangements. ELECTRONIC NOTES IN THEORETICAL COMPUTER SCIENCE, v. 346, p. 12-pg., 2019-08-30. (17/12646-3, 15/11937-9, 13/08293-7, 17/16246-0, 17/16871-1)

BRITO, KLAIRTON LIMA; JEAN, GERALDINE; FERTIN, GUILLAUME; OLIVEIRA, ANDRE RODRIGUES; DIAS, ULISSES; DIAS, ZANONI; CAI, Z; SKUMS, P; LI, M. Sorting by Reversals, Transpositions, and Indels on Both Gene Order and Intergenic Sizes. BIOINFORMATICS RESEARCH AND APPLICATIONS, ISBRA 2019, v. 11490, p. 12-pg., 2019-01-01. (13/08293-7, 17/16246-0, 17/12646-3, 15/11937-9)

SANTOS MIRANDA, GUILHERME HENRIQUE; ALEXANDRINO, ALEXSANDRO OLIVEIRA; LINTZMAYER, CARLA NEGRI; DIAS, ZANONI; ALVES, R. Sorting lambda-Permutations by lambda-Operations. ADVANCES IN BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, BSB 2018, v. 11228, p. 13-pg., 2018-01-01. (17/12646-3, 15/11937-9, 13/08293-7, 17/16246-0, 17/16871-1)

DANTAS, ANA PAULA S.; DE SOUZA, CID C.; DIAS, ZANONI. A heuristic for the convex recoloring problem in graphs. International Transactions in Operational Research, NOV 2020. (15/11937-9, 17/12646-3, 18/04760-3, 17/16246-0)

VALLE, EDUARDO; FORNACIALI, MICHEL; MENEGOLA, AFONSO; TAVARES, JULIA; BITTENCOURT, FLAVIA VASQUES; LI, LIN TZY; AVILA, SANDRA. Data, depth, and design: Learning reliable models for skin lesion analysis. Neurocomputing, v. 383, p. 303-313, MAR 28 2020. (19/05018-1, 17/16246-0)

OLIVEIRA, ANDRE R.; JEAN, GERALDINE; FERTIN, GUILLAUME; DIAS, ULISSES; DIAS, ZANONI. Super short operations on both gene order and intergenic sizes. Algorithms for Molecular Biology, v. 14, n. 1, NOV 5 2019. (17/12646-3, 17/16246-0, 13/08293-7, 15/11937-9)

OLIVEIRA, ANDRE RODRIGUES; BRITO, KLAIRTON LIMA; DIAS, ULISSES; DIAS, ZANONI. On the Complexity of Sorting by Reversals and Transpositions Problems. JOURNAL OF COMPUTATIONAL BIOLOGY, v. 26, n. 11, p. 1223-1229, NOV 1 2019. (13/08293-7, 17/16246-0, 15/11937-9, 17/12646-3)

MUNOZ, JAVIER VARGAS; GONCALVES, MARCOS A.; DIAS, ZANONI; TORRES, RICARDO DA S.. Hierarchical Clustering-Based Graphs for Large Scale Approximate Nearest Neighbor Search. PATTERN RECOGNITION, v. 96, DEC 2019. (14/50715-9, 16/50250-1, 17/16246-0, 17/20945-0, 14/12236-1, 13/50169-1, 15/11937-9, 17/12646-3, 13/50155-0, 15/24494-8)

DIAS, ULISSES; OLIVEIRA, ANDRE RODRIGUES; BRITO, KLAIRTON LIMA; DIAS, ZANONI; KOWADA, L; DEOLIVEIRA, D. Block-Interchange Distance Considering Intergenic Regions. ADVANCES IN BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, BSB 2019, v. 11347, p. 12-pg., 2020-01-01. (17/12646-3, 15/11937-9, 17/16246-0)

DE OLIVEIRA, GABRIEL BIANCHIN; PEDRINI, HELIO; DIAS, ZANONI; PAIVA, AC; CONCI, A; BRAZ, G; ALMEIDA, JDS; FERNANDES, LAF. Ensemble of Bidirectional Recurrent Networks and Random Forests for Protein Secondary Structure Prediction. PROCEEDINGS OF THE 2020 INTERNATIONAL CONFERENCE ON SYSTEMS, SIGNALS AND IMAGE PROCESSING (IWSSIP), 27TH EDITION, v. N/A, p. 6-pg., 2020-01-01. (17/16246-0, 19/20875-8, 17/12646-3, 15/11937-9)

BISSOTO, ALCEU; VALLE, EDUARDO; AVILA, SANDRA; IEEE COMP SOC. Debiasing Skin Lesion Datasets and Models? Not So Fast. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW 2020), v. N/A, p. 10-pg., 2020-01-01. (13/08293-7, 17/16246-0, 19/05018-1)

OLIVEIRA, ANDRE RODRIGUES; JEAN, GERALDINE; FERTIN, GUILLAUME; DIAS, ULISSES; DIAS, ZANONI; ALVES, R. Super Short Reversals on Both Gene Order and Intergenic Sizes. ADVANCES IN BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, BSB 2018, v. 11228, p. 12-pg., 2018-01-01. (17/12646-3, 15/11937-9, 13/08293-7, 17/16246-0, 17/16871-1)

OLIVEIRA, ANDRE RODRIGUES; BRITO, KLAIRTON LIMA; DIAS, ZANONI; DIAS, ULISSES; ALVES, R. Sorting by Weighted Reversals and Transpositions. ADVANCES IN BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, BSB 2018, v. 11228, p. 12-pg., 2018-01-01. (17/12646-3, 15/11937-9, 13/08293-7, 17/16246-0, 17/16871-1)

DE MELLO, FELIPE LOPES; DE OLIVEIRA, GABRIEL BIANCHIN; PEDRINI, HELIO; DIAS, ZANONI; RUTKOWSKI, L; SCHERER, R; KORYTKOWSKI, M; PEDRYCZ, W; TADEUSIEWICZ, R; ZURADA, JM. Prediction of Protein Molecular Functions Using Transformers. ARTIFICIAL INTELLIGENCE AND SOFT COMPUTING, ICAISC 2022, PT II, v. 13589, p. 9-pg., 2023-01-01. (17/16246-0, 19/20875-8, 17/12646-3, 15/11937-9)

PEIXOTO, BRUNO MALVEIRA; AVILA, SANDRA; DIAS, ZANONI; ROCHA, ANDERSON; ACM. Breaking down violence: A deep-learning strategy to model and classify violence in videos. 13TH INTERNATIONAL CONFERENCE ON AVAILABILITY, RELIABILITY AND SECURITY (ARES 2018), v. N/A, p. 7-pg., 2019-01-01. (17/12646-3, 15/11937-9, 13/08293-7, 17/16246-0, 17/16871-1)

RODRIGUES, CAROLINE MAZINI; PEREIRA, LUIS; ROCHA, ANDERSON; DIAS, ZANONI; IEEE. Image Semantic Representation for Event Understanding. 2019 IEEE INTERNATIONAL WORKSHOP ON INFORMATION FORENSICS AND SECURITY (WIFS), v. N/A, p. 6-pg., 2019-01-01. (17/12646-3, 15/11937-9, 13/08293-7, 17/16246-0, 18/16548-9, 18/16214-3, 17/16871-1)

PEIXOTO, BRUNO; LAVI, BAHRAM; PEREIRA MARTIN, JOAO PAULO; AVILA, SANDRA; DIAS, ZANONI; ROCHA, ANDERSON; IEEE. TOWARD SUBJECTIVE VIOLENCE DETECTION IN VIDEOS. 2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), v. N/A, p. 5-pg., 2019-01-01. (17/12646-3, 15/11937-9, 13/08293-7, 17/16246-0, 17/16871-1)

DOS SANTOS DANTASA, ANA PAULA; DE SOUZAA, CID CARVALHO; DIAS, ZANONI. A GRASP for the Convex Recoloring Problem in Graphs. ELECTRONIC NOTES IN THEORETICAL COMPUTER SCIENCE, v. 346, p. 13-pg., 2019-08-30. (17/12646-3, 18/04760-3, 15/11937-9, 13/08293-7, 17/16246-0)

OLIVEIRA, GABRIEL B.; PEDRINI, HELIO; DIAS, ZANONI. TEMPROT: protein function annotation using transformers embeddings and homology search. BMC Bioinformatics, v. 24, n. 1, p. 16-pg., 2023-06-08. (17/16246-0, 19/20875-8, 17/12646-3, 15/11937-9)

OLIVEIRA, ANDRE RODRIGUES; JEAN, GERALDINE; FERTIN, GUILLAUME; BRITO, KLAIRTON LIMA; DIAS, ULISSES; DIAS, ZANONI; MARTINVIDE, C; VEGARODRIGUEZ, MA; WHEELER, T. A 3.5-Approximation Algorithm for Sorting by Intergenic Transpositions. ALGORITHMS FOR COMPUTATIONAL BIOLOGY (ALCOB 2020), v. 12099, p. 13-pg., 2020-01-01. (13/08293-7, 17/12646-3, 15/11937-9, 17/16246-0)

TEIXEIRA, MARCOS; AVILA, SANDRA; ASSOC COMP MACHINERY. What should we pay attention to when classifying violent videos?. ARES 2021: 16TH INTERNATIONAL CONFERENCE ON AVAILABILITY, RELIABILITY AND SECURITY, v. N/A, p. 10-pg., 2021-01-01. (13/08293-7, 17/16246-0)

ALEXANDRINO, ALEXSANDRO OLIVEIRA; LINTZMAYER, CARLA NEGRI; DIAS, ZANONI. Sorting permutations by fragmentation-weighted operations. JOURNAL OF BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, v. 18, n. 2, APR 2020. (17/12646-3, 17/16871-1, 17/16246-0, 15/11937-9)

DAVID, LUCAS; PEDRINI, HELIO; DIAS, ZANONI; ROCHA, ANDERSON; IEEE. Connoisseur: Provenance Analysis in Paintings. 2021 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI 2021), v. N/A, p. 8-pg., 2021-01-01. (17/12646-3, 15/11937-9, 17/16246-0)

DE OLIVEIRA, GABRIEL BIANCHIN; PEDRINI, HELIO; DIAS, ZANONI. nsemble of Template-Free and Template-Based Classifiers for Protein Secondary Structure Predictio. INTERNATIONAL JOURNAL OF MOLECULAR SCIENCES, v. 22, n. 21, NOV 2021. (19/20875-8, 17/12646-3, 17/16246-0)

SANTOS, THIAGO T.; DE SOUZA, LEONARDO L.; DOS SANTOS, ANDREZA A.; AVILA, SANDRA. Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association. COMPUTERS AND ELECTRONICS IN AGRICULTURE, v. 170, MAR 2020. (13/08293-7, 17/16246-0)

SIQUEIRA, GABRIEL; BRITO, KLAIRTON LIMA; DIAS, ULISSES; DIAS, ZANONI. euristics for Genome Rearrangement Distance With Replicated Gene. IEEE-ACM TRANSACTIONS ON COMPUTATIONAL BIOLOGY AND BIOINFORMATICS, v. 18, n. 6, p. 2094-2108, NOV 1 2021. (17/16246-0, 15/11937-9, 17/12646-3)

BRITO, KLAIRTON LIMA; ALEXANDRINO, ALEXSANDRO OLIVEIRA; OLIVEIRA, ANDRE RODRIGUES; DIAS, ULISSES; DIAS, ZANONI. Reversals and transpositions distance with proportion restriction. JOURNAL OF BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, v. 19, n. 4, AUG 2021. (19/27331-3, 17/12646-3, 17/16246-0, 13/08293-7, 15/11937-9)

ALEXANDRINO, ALEXSANDRO OLIVEIRA; OLIVEIRA, ANDRE RODRIGUES; DIAS, ULISSES; DIAS, ZANONI. Genome Rearrangement Distance with Reversals, Transpositions, and Indels. JOURNAL OF COMPUTATIONAL BIOLOGY, v. 28, n. 3, p. 235-247, MAR 1 2021. (17/16246-0, 19/27331-3, 15/11937-9, 17/12646-3)

SIQUEIRA, GABRIEL; BRITO, KLAIRTON LIMA; DIAS, ULISSES; DIAS, ZANONI; MARTINVIDE, C; VEGARODRIGUEZ, MA; WHEELER, T. Heuristics for Reversal Distance Between Genomes with Duplicated Genes. ALGORITHMS FOR COMPUTATIONAL BIOLOGY (ALCOB 2020), v. 12099, p. 12-pg., 2020-01-01. (17/12646-3, 15/11937-9, 17/16246-0)

BOLLIS, EDSON; PEDRINI, HELIO; AVILA, SANDRA; IEEE COMP SOC. Weakly Supervised Learning Guided by Activation Mapping Applied to a Novel Citrus Pest Benchmark. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS (CVPRW 2020), v. N/A, p. 10-pg., 2020-01-01. (13/08293-7, 14/12236-1, 17/12646-3, 17/16246-0)

BRAZ, LEODECIO; TEIXEIRA, VINICIUS; PEDRINI, HELIO; DIAS, ZANONI; SETUBAL, JC; SILVA, WM. ImTeNet: Image-Text Classification Network for Abnormality Detection and Automatic Reporting on Musculoskeletal Radiographs. ADVANCES IN BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, BSB 2020, v. 12558, p. 12-pg., 2020-01-01. (19/20875-8, 17/12646-3, 15/11937-9, 17/16246-0)