Topics in neural networks: I. interaction between attractor neural networks. II. learning dynamics in deep learning architectures

• Grant number: 21/07951-7
• Support Opportunities: Scholarships in Brazil - Master
• Start date: March 01, 2022
• End date: August 31, 2022
• Field of knowledge: Physical Sciences and Mathematics - Physics
• Principal Investigator: Nestor Felipe Caticha Alfonso
• Grantee: Pietro Zanin
• Host Institution: Instituto de Física (IF). Universidade de São Paulo (USP). São Paulo , SP, Brazil

Abstract

I. We introduce a new type of interaction between agents modelled by attractor neural networks (ANN). We study analytically the influence on the critical capacity of the interaction with another ANN. The two agents, which receive the same amount of information, undergo an different periods of an unlearning (dream) dynamics, which partially eliminates spurious minima of the Hamiltonian. Preliminary results point to a large range of possible changes in the agents' information processing behavior. II. We will study using Entropic Dynamics inference techniques, the design of optimal learning algorithms for Deep architecture Neural Networks, with hidden units with a ReLu transfer function, which has been shown to lead to a less singular behavior than sigmoidal transfer function units. Convexity of the transfer functions, apparently has a bearing on the order of the phase transitions into the specialized regime of the internal branches of the network. (AU)

Scientific publications

(References retrieved automatically from Web of Science and SciELO through information on FAPESP grants and their corresponding numbers as mentioned in the publications by the authors)

ZANIN, PIETRO; CATICHA, NESTOR. Interacting dreaming neural networks. JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, v. 2023, n. 4, p. 23-pg., 2023-04-01. (21/07951-7)

Academic Publications

(References retrieved automatically from State of São Paulo Research Institutions)

ZANIN, Pietro. Analysis of interacting attractor neural networks with a model with analytic solution. 2022. Master's Dissertation - Universidade de São Paulo (USP). Instituto de Física (IF/SBI) São Paulo.