Investigation of open set domain adaptation methods for computer vision tasks

• Grant number: 21/13348-1
• Support Opportunities: Scholarships abroad - Research Internship - Master's degree
• Start date: February 01, 2022
• End date: July 31, 2022
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Jurandy Gomes de Almeida Junior
• Grantee: Lucas Fernando Alvarenga e Silva
• Supervisor: Niculae Sebe
• Host Institution: Instituto de Ciência e Tecnologia (ICT). Universidade Federal de São Paulo (UNIFESP). Campus São José dos Campos. São José dos Campos , SP, Brazil
• Institution abroad: Universitá degli Studi di Trento, Italy
• Associated to the scholarship: 20/08770-3 - Open set methods based on deep networks for multimedia recognition, BP.MS

Abstract

Deep learning (DL) methods have brought revolutionary advances to many research areas due to their capacity of learning from raw data. Particularly, convolutional neural networks (CNNs) have substantially increased the development of intelligent multimedia systems, like natural language processing and computer vision systems. However, when those methods are applied to non-controllable environments, usually in real-world problems, many different factors can degrade the model's expected performance, for instance, when dealing with unlabeled datasets. Once labeling those examples is expensive, a naive approach is to train the model on similar annotated datasets, however, the examples can present some degrees of domain shift, degrading the model's performance. Another problem with inferring on unlabeled datasets occurs when the model is faced with test examples belonging to classes not seen during training, driving the model to make erroneous predictions. Each of these problems have been the goal of the research areas known as Unsupervised Domain Adaptation (UDA) and Open Set Classification (OS), respectively. We can have an even more challenging scenery by combining those real-world problems, which are investigated under the Open Set Domain Adaptation (OSDA) research area, by jointly dealing with UDA and OS assumptions when the training data are unlabeled and not all classes are known a priori. This research proposal aims to deal with this more challenging scenery, by learning from unlabeled data and recognizing unknown classes at inference time. For this, we intend to investigate Domain Alignment techniques to alleviate the domain shift across the sets of data into account and Contrastive Learning (CL) techniques to improve the discriminability between known and unknown classes. We believe that working jointly with the Multimedia and Human Understanding Group (MHUG) at the University of Trento, Italy, due to its state-of-the-art contributions on related topics, can help to define efficient ways to tackle OSDA problems. (AU)