MMeaning - multimodal distributional semantic models

• Grant number: 16/13002-0
• Support Opportunities: Regular Research Grants
• Start date: October 01, 2016
• End date: October 31, 2018
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Helena de Medeiros Caseli
• Grantee: Helena de Medeiros Caseli
• Host Institution: Centro de Ciências Exatas e de Tecnologia (CCET). Universidade Federal de São Carlos (UFSCAR). São Carlos , SP, Brazil
• Associated researchers: Eloize Rossi Marques Seno ; Jander Moreira

Abstract

With the increasing availability of information on the web, processing and retrieving textual and visual information are essential activities in automatic knowledge generation. As most of the information available on the web is made up of text written in natural language and images, process them in a "smart" way necessarily involves understanding (interpretation) of the meaning of the information they convey. One of the most used forms for representing the semantic content are the distributional semantic models, which are based on the distributional hypothesis which states that the meaning of a word is given by its occurrence context. Although the main source for semantic knowledge extraction are the corpora, other sources of extra-linguistic information, such as images, should also be taken into account. The combination of multiple sources of information to generate semantic representations is called multimodal distributional semantic representation. In addition to this new research field, there is the recent interest in distributional representation models based on neural networks, also known as models of deep learning. In this context, this project aims to investigate the use of different sources of knowledge, such as parallel/comparable texts and images in the distributional semantic modeling of natural language texts to enrich the information used in Natural Language Processing and Information Retrieval applications. (AU)

Scientific publications

(8)

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

RODRIGUES DA SILVA, JESSICA; CASELI, HELENA DE M.. Sense representations for Portuguese: experiments with sense embeddings and deep neural language models. Language Resources and Evaluation, FEB 2021. (16/13002-0)

CASELI, HELENA DE MEDEIROS; INACIO, MARCIO LIMA; CALZOLARI, N; BECHET, F; BLACHE, P; CHOUKRI, K; CIERI, C; DECLERCK, T; GOGGI, S; ISAHARA, H; et al. NMT and PBSMT Error Analyses in English to Brazilian Portuguese Automatic Translations. PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020), v. N/A, p. 7-pg., 2020-01-01. (16/21317-0, 16/13002-0)

ITO, FERNANDO TADAO; CASELI, HELENA DE MEDEIROS; MOREIRA, JANDER; IEEE. The Effects of Underlying Mono and Multilingual Representations for Text Classification. 2018 7TH BRAZILIAN CONFERENCE ON INTELLIGENT SYSTEMS (BRACIS), v. N/A, p. 6-pg., 2018-01-01. (16/13002-0)

VIEIRA, MIGUEL G.; MOREIRA, JANDER; GARCIAGONCALVES, LM; BESERRAGOMES, R. Classification of E-commerce-related Images Using Hierarchical Classification with Deep Neural Networks. 2017 WORKSHOP OF COMPUTER VISION (WVC), v. N/A, p. 6-pg., 2017-01-01. (16/13002-0)

ITO, FERNANDO TADAO; CASELI, HELENA DE MEDEIROS; MOREIRA, JANDER; DECLERCK, T; CALZOLARI, N; CHOUKRI, K; CIERI, C; HASIDA, K; ISAHARA, H; MAEGAARD, B; et al. The Effects of Unimodal Representation Choices on Multimodal Learning. PROCEEDINGS OF THE ELEVENTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2018), v. N/A, p. 8-pg., 2018-01-01. (16/13002-0)

RODRIGUES DA SILVA, JESSICA; CASELI, HELENA DE M.. Sense representations for Portuguese: experiments with sense embeddings and deep neural language models. Language Resources and Evaluation, v. 55, n. 4, p. 24-pg., 2021-02-28. (16/13002-0)

INACIO, MARCIO LIMA; CASELI, HELENA DE MEDEIROS; QUARESMA, P; VIEIRA, R; ALUISIO, S; MONIZ, H; BATISTA, F; GONCALVES, T. Word Embeddings at Post-Editing. COMPUTATIONAL PROCESSING OF THE PORTUGUESE LANGUAGE, PROPOR 2020, v. 12037, p. 12-pg., 2020-01-01. (16/21317-0, 16/13002-0)

VELTRONI, WELLINGTON CRISTIANO; CASELI, HELENA DE MEDEIROS; VILLAVICENCIO, A; MOREIRA, V; ABAD, A; CASELI, H; GAMALLO, P; RAMISCH, C; OLIVEIRA, HG; PAETZOLD, GH. Text-Image Alignment in Portuguese News Using LinkPICS. COMPUTATIONAL PROCESSING OF THE PORTUGUESE LANGUAGE, PROPOR 2018, v. 11122, p. 11-pg., 2018-01-01. (16/13002-0)