| Full text | |
| Author(s): |
Passos, Leandro A.
;
Papa, Joao Paulo
;
Hussain, Amir
;
Adeel, Ahsan
Total Authors: 4
|
| Document type: | Journal article |
| Source: | Neurocomputing; v. 527, p. 8-pg., 2023-01-20. |
| Abstract | |
Despite the recent success of machine learning algorithms, most models face drawbacks when consider-ing more complex tasks requiring interaction between different sources, such as multimodal input data and logical time sequences. On the other hand, the biological brain is highly sharpened in this sense, empowered to automatically manage and integrate such streams of information. In this context, this work draws inspiration from recent discoveries in brain cortical circuits to propose a more biologically plausible self-supervised machine learning approach. This combines multimodal information using intra-layer modulations together with Canonical Correlation Analysis, and a memory mechanism to keep track of temporal data, the overall approach termed Canonical Cortical Graph Neural networks. This is shown to outperform recent state-of-the-art models in terms of clean audio reconstruction and energy efficiency for a benchmark audio-visual speech dataset. The enhanced performance is demonstrated through a reduced and smother neuron firing rate distribution. suggesting that the proposed model is amenable for speech enhancement in future audio-visual hearing aid devices.(c) 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). (AU) | |
| FAPESP's process: | 14/12236-1 - AnImaLS: Annotation of Images in Large Scale: what can machines and specialists learn from interaction? |
| Grantee: | Alexandre Xavier Falcão |
| Support Opportunities: | Research Projects - Thematic Grants |
| FAPESP's process: | 19/07665-4 - Center for Artificial Intelligence |
| Grantee: | Fabio Gagliardi Cozman |
| Support Opportunities: | Research Grants - Research Program in eScience and Data Science - Research Centers in Engineering Program |
| FAPESP's process: | 13/07375-0 - CeMEAI - Center for Mathematical Sciences Applied to Industry |
| Grantee: | Francisco Louzada Neto |
| Support Opportunities: | Research Grants - Research, Innovation and Dissemination Centers - RIDC |
| FAPESP's process: | 17/02286-0 - Probabilistic models for commercial losses detection |
| Grantee: | André Nunes de Souza |
| Support Opportunities: | Regular Research Grants |
| FAPESP's process: | 19/18287-0 - Real-time Urban Forest Management Using Machine Learning |
| Grantee: | Danilo Samuel Jodas |
| Support Opportunities: | Scholarships in Brazil - Post-Doctoral |
| FAPESP's process: | 18/21934-5 - Network statistics: theory, methods, and applications |
| Grantee: | André Fujita |
| Support Opportunities: | Research Projects - Thematic Grants |