Training Deep Networks from Zero to Hero: avoiding pitfalls and going beyond

Autor(es)

Ponti, Moacir A. ; dos Santos, Fernando P. ; Ribeiro, Leo S. F. ; Cavallari, Gabriel B. ; IEEE Comp Soc

• Tipo de documento: Artigo Científico
• Fonte: 2021 34TH SIBGRAPI CONFERENCE ON GRAPHICS, PATTERNS AND IMAGES (SIBGRAPI 2021); v. N/A, p. 8-pg., 2021-01-01.

Resumo

Training deep neural networks may be challenging in real world data. Using models as black-boxes, even with transfer learning, can result in poor generalization or inconclusive results when it comes to small datasets or specific applications. This tutorial covers the basic steps as well as more recent options to improve models, in particular, but not restricted to, supervised learning. It can be particularly useful in datasets that are not as well-prepared as those in challenges, and also under scarce annotation and/or small data. We describe basic procedures as data preparation, optimization and transfer learning, but also recent architectural choices such as use of transformer modules, alternative convolutional layers, activation functions, wide/depth, as well as training procedures including curriculum, contrastive and self-supervised learning. (AU)

• Processo FAPESP: 19/07316-0 - Teoria de singularidades e aplicações a geometria diferencial, equações diferenciais e visão computacional
• Beneficiário: Farid Tari
• Modalidade de apoio: Auxílio à Pesquisa - Temático
• Processo FAPESP: 17/22366-8 - Redes geradoras e aprendizado de características para busca entre domínios visuais
• Beneficiário: Leo Sampaio Ferraz Ribeiro
• Modalidade de apoio: Bolsas no Brasil - Doutorado Direto
• Processo FAPESP: 19/02033-0 - Estudo de representações de imagens de múltiplos domínios a partir de aprendizado profundo não supervisionado e semi-supervisionado
• Beneficiário: Gabriel Biscaro Cavallari
• Modalidade de apoio: Bolsas no Brasil - Mestrado