Patterns and randomness in networks for computer vision: from graphs to neural networks

Author(s)

Leonardo Felipe dos Santos Scabini

• Document type: Doctoral Thesis
• Press: São Carlos.
• Institution: Universidade de São Paulo (USP). Instituto de Física de São Carlos (IFSC/BT)
• Defense date: 2023-07-24

Examining board members

Odemir Martinez Bruno; Antoine Manzanera; Gilberto Medeiros Nakamura; Eraldo Ribeiro Junior; Francisco Aparecido Rodrigues

• Advisor: Odemir Martinez Bruno; Bernard De Baets

Abstract

Complex networks pervade various aspects of nature, society, and science. One of the most discussed types of network is a neural network, particularly in the last decade with the advances in artificial intelligence (AI). However, little is known about the structure of artificial neural networks (ANNs) in terms of topology and dynamics, from a network science point of view. Moreover, these models are known for being black-box systems, and may also exhibit unexpected behavior. This thesis focuses on AI networks, neural or not, for computer vision (CV) - the sub-field of AI that deals with visual information. Our study explores several aspects of network structure, patterns, and randomness, and their potential for understanding, enhancing, and developing new network-based CV systems. Firstly, we propose a novel network science-based framework for examining ANNs, focusing on their neuronal centrality. This approach, named Bag-of-Neurons (BoN), uncovers the relationship between structural patterns within trained ANNs and their performance and proved to be a promising approach for understanding these systems. These findings led to a new ANN random initialization technique, named Preferential Attachment Rewiring (PARw), which enhances performance and accelerates the training process of shallow and deep ANNs. By leveraging network science, we also develop CV techniques for texture analysis problems, ranging from pure texture images to texture in the wild. We introduce the Spatio-Spectral Network (SSN), a method for image modeling using a graph of pixels, which achieves state-of-the-art (SOTA) results in benchmark datasets. Another new proposal (SSR) couples SSNs with small randomized neural networks, improving performance without a substantial increase in computational costs. The thesis further presents the Randomized encoding of Aggregated Deep Activation Maps (RADAM), a transfer learning method for deep convolutional networks (CNNs) that offers remarkable performance in all CV tasks that were evaluated. We also explore the potential of fully randomized deep CNNs (FR-DCNN) coupled with PARw and RADAM, which prove to be robust texture feature extractors. Lastly, we apply our methods to real-world tasks, including prostate cancer and COVID-19 diagnosis, environmental biosensors using plants, and plant species identification. Our methods, particularly RADAM, consistently achieved SOTA results in these tasks. In conclusion, this thesis introduces six network-based methods for ANNs and CV, and the results show that they consistently improve the SOTA on various image classification problems. (AU)

• FAPESP's process: 19/07811-0 - Artificial neural networks and complex networks: an integrative study of topological properties and pattern recognition
• Grantee: Leonardo Felipe dos Santos Scabini
• Support Opportunities: Scholarships in Brazil - Doctorate