The role of contrastive learning for user-assisted cell segmentation correction

• Grant number: 22/16491-2
• Support Opportunities: Scholarships abroad - Research Internship - Scientific Initiation
• Start date: April 01, 2023
• End date: July 31, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Alexandre Xavier Falcão
• Grantee: Ilan Francisco da Silva
• Supervisor: Loic Royer
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil
• Institution abroad: CZ Biohub, United States
• Associated to the scholarship: 22/07877-4 - Incorporating contrastive learning in image segmentation by dynamic trees, BP.IC

Abstract

Cell image segmentation of 3D/4D microscopy images is a complex task due to the lack of signal intensity in some parts of the image, splitting, and merging of cells. Methods based on Deep Learning have been developed to solve the problem automatically, but errors often occur, leading to two subproblems of interest: (1) how can we effectively detect the cells with segmentation errors? (2) how can we efficiently correct the errors? In this BEPE (Research Internship Abroad) project, we will collaborate with researchers from the Chan-Zuckerberg Biohub (CZ Biohub) on both subproblems. For (1), we propose a visualization tool based on contrastive learning and non-linear projections to more easily identify image blocks (patches) containing segmentation errors. For (2), by visualizing and selecting candidate blocks from a 2D projection, the user can efficiently correct the errors using a graph-based segmentation algorithm developed in a previous project (FAPESP 2021/05704-2) and improved in the current one (FAPESP 2022/07877-4). This algorithm, named Differential Dynamic Trees, now exploits contrastive learning to improve the contrast between object and background during object delineation. In this sense, the BEPE proposal will investigate the role of contrastive learning for interactive cell segmentation. The work consists of building the contrastive models for (1) and (2), incorporating them in the software tool under development at CZ Biohub, evaluating the results, and documenting the work. We expect to submit a paper about this in the last period after returning from BEPE. (AU)