Evaluation of few-shot learning models using performance estimates and ranking

• Grant number: 22/09913-8
• Support Opportunities: Scholarships abroad - Research Internship - Scientific Initiation
• Start date: March 01, 2023
• End date: June 30, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Moacir Antonelli Ponti
• Grantee: Luísa Balleroni Shimabucoro
• Supervisor: Timothy Hospedales
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Institution abroad: University of Edinburgh, Scotland
• Associated to the scholarship: 21/06462-2 - Representation learning of sketches and images for recognition, search and cross-domain synthesis, BP.IC

Abstract

Machine Learning has gained praise for its impressive performance in tasks where data is plentiful, like object detection and text generation. Many problems however do not have the privilege of having massive labelled data and collecting even small datasets in these domains often carries a large financial cost and requires a substantial time investment, which is the case in areas such as medicine and defense. For that reason, the development of techniques capable of dealing with data scarcity, such as Few-Shot Learning is crucial. Nonetheless, the development of methods capable of evaluating the performance of these models which lack large validation sets, which are crucial to the their deployment, is not given as much attention. Therefore, this project aims to explore different evaluation metrics within this data scarcity scenario so as to be able to analyse its behaviour and search for new methods which might be able to provide more reliable and robust evaluations of Few-Shot Learning models. (AU)