Incorporating contrastive learning in image segmentation by dynamic trees

• Grant number: 22/07877-4
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: August 01, 2022
• End date: November 30, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Alexandre Xavier Falcão
• Grantee: Ilan Francisco da Silva
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Associated research grant: 14/12236-1 - AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction?, AP.TEM
• Associated scholarship(s): 22/16491-2 - The role of contrastive learning for user-assisted cell segmentation correction, BE.EP.IC

Abstract

Dynamic Trees (DT) is an image segmentation method that grows optimum-path trees (regions) from root pixels (seeds). The seeds compete for the most strongly connected pixels, and the strength of connectedness between seed and pixel is defined during the optimum tree-growing process. DT uses an Image Foresting Transform (IFT) algorithm, which defines each object as an optimum-path forest rooted in its internal seeds. The method differs from other IFT-based segmentation approaches by the dynamic arc-weight estimation based on the image properties of the growing trees. Previous works have also shown that DT can outperform popular algorithms in different datasets, such as watershed, graph cut, deep-extreme cut, and graph Laplacian. In the previous Scientific Initiation project (FAPESP 2021/05704-2), the student studied and extended the DT algorithm for the differential reconstruction of the optimum-path forest - a fundamental property to provide interactive response time to the users' actions during the segmentation of images with millions of pixels, as the user adds and removes seeds to correct segmentation errors. This project continues the previous work investigating and incorporating contrastive learning techniques to compute the strength of connectedness. It will involve the construction of convolutional neural networks from image markers, using a recent technique, in combination with contrastive learning to adapt the convolutional pixel features to the object under segmentation. By incorporating machine learning, DT should become more easily adaptable to different situations. The project also considers an internship (BEPE) in collaboration with researchers of the CZ BioHub (a nonprofit research center in the USA), who are interested in the differential version of the DT algorithm for cell segmentation in 4D images of fluorescence microscopy.(AU)