Adversarial learning in natural language processing

• Grant number: 19/02205-5
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: May 01, 2019
• End date: July 31, 2021
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: João Paulo Papa
• Grantee: Gustavo Henrique de Rosa
• Host Institution: Faculdade de Ciências (FC). Universidade Estadual Paulista (UNESP). Campus de Bauru. Bauru , SP, Brazil
• Associated research grant: 14/12236-1 - AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction?, AP.TEM

Abstract

Machine learning techniques have been paramount in the latest years, mainly due to their high effectiveness in computer vision and Natural Language Processing (NLP) problems. Despite their significant discriminatory power, their generative capacities are far from having an ideal performance in specific applications, often related to text-data mining ones. The most significant setback occurs when those techniques are employed with discrete representations such as characters and words sequences. Although some novel works presented feasible solutions to the natural language generation problem, most could not generate a completely `natural´ language, lacking from either morphological, syntactical or semantical knowledge. Furthermore, another recurring issue in the NLP area concerns the scarcity of data to feed deep learning architectures. Popularly used in the image processing field for generating synthetical data, adversarial learning has not yet been developed to a satisfactory point in the text-data area, producing lots of repeated words and even some without semantical significance. Therefore, this present proposal endeavors on fostering forefront research in developing adversarial models in the context of natural language processing. Generative Adversarial Networks (GANs) and Recurrent Neural Networks (RNNs), with particular attention to Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTMs) architectures, will be studied. This proposal also comprises an internship at Stanford University, United States. (AU)

Scientific publications

(12)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

DE ROSA, GUSTAVO H.; PAPA, JOAO P.. A survey on text generation using generative adversarial networks. PATTERN RECOGNITION, v. 119, NOV 2021. (19/02205-5, 20/12101-0, 14/12236-1, 19/07665-4, 13/07375-0)

DE ROSA, GUSTAVO H.; RODER, MATEUS; PAPA, JOAO P.. Neighbour-based bag-of-samplings for person identification through handwritten dynamics and convolutional neural networks. EXPERT SYSTEMS, NOV 2021. (19/07665-4, 14/12236-1, 20/12101-0, 19/02205-5, 13/07375-0)

RODER, MATEUS; PASSOS, LEANDRO APARECIDO; DE ROSA, GUSTAVO H.; DE ALBUQUERQUE, VICTOR HUGO C.; PAPA, JOAO PAULO. Reinforcing learning in Deep Belief Networks through nature-inspired optimization. APPLIED SOFT COMPUTING, v. 108, SEP 2021. (19/07825-1, 18/21934-5, 17/25908-6, 19/07665-4, 19/02205-5, 13/07375-0, 14/12236-1)

RODER, MATEUS; DE ROSA, GUSTAVO HENRIQUE; PAPA, JOAO PAULO; BREVE, FABRICIO APARECIDO; IEEE. Fine-Tuning Temperatures in Restricted Boltzmann Machines Using Meta-Heuristic Optimization. 2020 IEEE CONGRESS ON EVOLUTIONARY COMPUTATION (CEC), v. N/A, p. 8-pg., 2020-01-01. (19/02205-5, 17/25908-6, 19/07825-1, 13/07375-0, 14/12236-1)

RODER, MATEUS; DE ROSA, GUSTAVO HENRIQUE; PASSOS, LEANDRO APARECIDO; PAPA, JOAO PAULO; DEBIASO ROSSI, ANDRE LUIS; IEEE. Harnessing Particle Swarm Optimization Through Relativistic Velocity. 2020 IEEE CONGRESS ON EVOLUTIONARY COMPUTATION (CEC), v. N/A, p. 8-pg., 2020-01-01. (19/02205-5, 17/25908-6, 19/07825-1, 13/07375-0, 19/07665-4, 14/12236-1)

DE ROSA, GUSTAVO H.; RODER, MATEUS; PAPA, JOAO PAULO; DOS SANTOS, CLAUDIO F. G.; IEEE. Improving Pre-Trained Weights through Meta-Heuristics Fine-Tuning. 2021 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI 2021), v. N/A, p. 8-pg., 2021-01-01. (13/07375-0, 19/07665-4, 14/12236-1, 19/02205-5, 20/12101-0)

FERREIRA, ALVARO R., JR.; DE ROSA, GUSTAVO H.; PAPA, JOAO P.; CARNEIRO, GUSTAVO; FARIA, FABIO A.; IEEE COMP SOC. Creating Classifier Ensembles through Meta-heuristic Algorithms for Aerial Scene Classification. 2020 25TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR), v. N/A, p. 8-pg., 2021-01-01. (14/12236-1, 18/23908-1, 17/25908-6, 19/07665-4, 19/02205-5)

RIBEIRO, LUIZ C. F.; DE ROSA, GUSTAVO H.; RODRIGUES, DOUGLAS; PAPA, JOAO P.. Convolutional neural networks ensembles through single-iteration optimization. SOFT COMPUTING, v. 26, n. 8, p. 12-pg., 2022-01-29. (14/12236-1, 19/07665-4, 13/07375-0, 17/25908-6, 19/02205-5, 18/21934-5)

OLIVEIRA, GUILHERME C.; NGO, QUOC C.; PASSOS, LEANDRO A.; PAPA, JOAO P.; JODAS, DANILO S.; KUMAR, DINESH. Tabular data augmentation for video-based detection of hypomimia in Parkinson's disease. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE, v. 240, p. 8-pg., 2023-10-01. (19/00585-5, 18/15597-6, 14/12236-1, 19/07665-4, 13/07375-0, 19/02205-5)

DE ROSA, GUSTAVO H.; PAPA, JOAO P.; YANG, XIN-SHE. A nature-inspired feature selection approach based on hypercomplex information. APPLIED SOFT COMPUTING, v. 94, SEP 2020. (13/07375-0, 16/19403-6, 14/12236-1, 17/25908-6, 17/02286-0, 19/02205-5)

DE ROSA, GUSTAVO H.; PAPA, JOAO P.. OPFython: A Python implementation for Optimum-Path Forest. SOFTWARE IMPACTS, v. 9, p. 3-pg., 2021-08-01. (19/02205-5, 20/12101-0, 14/12236-1, 19/07665-4, 13/07375-0)

DE ROSA, GUSTAVO H.; BREGA, JOSE R. F.; PAPA, JOAO P.. How optimizing perplexity can affect the dimensionality reduction on word embeddings visualization?. SN APPLIED SCIENCES, v. 1, n. 12, DEC 2019. (19/02205-5)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

ROSA, Gustavo Henrique de. Text generation analysis using similarity-based adversarial learning. 2022. Doctoral Thesis - Universidade Estadual Paulista (Unesp). Faculdade de Ciências. Bauru Bauru.