Exploring high-level representations in image operator learning

• Grant number: 14/21692-0
• Support Opportunities: Scholarships abroad - Research Internship - Doctorate (Direct)
• Start date: January 20, 2015
• End date: January 19, 2016
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Roberto Hirata Junior
• Grantee: Igor dos Santos Montagner
• Supervisor: Stéphane Canu
• Host Institution: Instituto de Matemática e Estatística (IME). Universidade de São Paulo (USP). São Paulo , SP, Brazil
• Institution abroad: Institut National des Sciences Appliquées Rouen (INSA Rouen), France
• Associated to the scholarship: 11/23310-0 - Automatic design of image operators: extension and contextualization to not necessarily boolean lattices, BP.DD

Abstract

Designing Image Operators is a complex time-consuming task usually custom hand-made by image processing specialists. However, this task can also be interpreted as a Machine Learning task. An operator can be represented as a local function $\psi$ that outputs for each pixel $p$ in the input image its intensity on the output. This function takes as input the values of the pixels in a neighborhood (\emph{window}) around $p$. Function $\psi$ is estimated from a set of pairs of images containing an input image and its processed version. Most of the recent related works are on window selection methods, which are concerned in finding an optimal window within a large window domain, and two level operators, which combine the output of several image operators in order to achieve more robust classification. Although significant results have been obtained by operators trained between binary images, processing gray level images is still a challenge due to the much higher variability of the patterns. In this context, the use of small windows may result in operators that over fit the data and the existing techniques are not likely to work. We propose in this project to address this challenge by changing the representation of the window pattern and using an appropriate regularization penalty term. Instead of using raw pixels as features, we will study how to calculate a new feature vector from the window pattern in order to improve the performance of image operators. Two classes of techniques in which the change of representation will be studied: Kernel methods and Representation Learning. Both classes have state-of-the-art results in many different applications but calculate the new representation in very different ways. The theoretical relationships between these techniques and existing works will be explored, formalized and implemented. Their performance will be empirically assessed and compared to state-of-the-art methods for public datasets. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

HIRATA, NINA S. T.; MONTAGNER, IGOR S.; HIRATA, ROBERTO, JR.; ACM. Comics image processing: learning to segment text. PROCEEDINGS OF THE 1ST INTERNATIONAL WORKSHOP ON COMICS ANALYSIS, PROCESSING AND UNDERSTANDING (MANPU 2016), v. N/A, p. 6-pg., 2016-01-01. (11/23310-0, 15/01587-0, 15/17741-9, 14/21692-0)

MONTAGNER, IGOR S.; HIRATA, NINA S. T.; HIRATA, ROBERTO, JR.; CANU, STEPHANE; IEEE. Kernel approximations for W-operator learning. 2016 29TH SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI), v. N/A, p. 8-pg., 2016-01-01. (11/50761-2, 15/01587-0, 11/23310-0, 14/21692-0, 15/17741-9)

MONTAGNER, IGOR S.; HIRATA, NINA S. T.; HIRATA, JR., ROBERTO. Staff removal using image operator learning. PATTERN RECOGNITION, v. 63, p. 310-320, MAR 2017. (11/23310-0, 15/17741-9, 11/00325-1, 14/21692-0)