| Full text | |
| Author(s): |
Mizuno Nakamura, Angelica Tiemi
;
Grassi Jr, Valdir
;
Wolf, Denis Fernando
Total Authors: 3
|
| Document type: | Journal article |
| Source: | Neurocomputing; v. 511, p. 11-pg., 2022-09-13. |
| Abstract | |
Multi-Task Learning is a learning paradigm that uses correlated tasks to improve performance generalization. A common way to learn multiple tasks is through the hard parameter sharing approach, in which a single architecture is used to share the same subset of parameters, creating an inductive bias between them during the training process. Due to its simplicity, potential to improve generalization, and reduce computational cost, it has gained the attention of the scientific and industrial communities. However, tasks often conflict with each other, which makes it challenging to define how the gradients of multiple tasks should be combined to allow simultaneous learning. To address this problem, we use the idea of multi-objective optimization to propose a method that takes into account temporal behaviour of the gradients to create a dynamic bias that adjusts the importance of each task during backpropagation. The result of this method is to give more attention to tasks that are diverging or not being benefited during the last iterations, ensuring that the simultaneous learning is heading to the performance maximization of all tasks. As a result, we empirically show that the proposed method outperforms the state-of-the-art approaches on learning conflicting tasks. Unlike the adopted baselines, our method ensures that all tasks reach good generalization performances. (C) 2022 Elsevier B.V. All rights reserved. (AU) | |
| FAPESP's process: | 14/50851-0 - INCT 2014: National Institute of Science and Technology for Cooperative Autonomous Systems Applied in Security and Environment |
| Grantee: | Marco Henrique Terra |
| Support Opportunities: | Research Projects - Thematic Grants |
| FAPESP's process: | 19/03366-2 - Spatial instance segmentation in monocular images through convolutional neural networks |
| Grantee: | Angelica Tiemi Mizuno Nakamura |
| Support Opportunities: | Scholarships in Brazil - Doctorate |