| Texto completo | |
| Autor(es): |
Sacilotti, Andre
;
dos Santos, Samuel Felipe
;
Sebe, Nicu
;
Almeida, Jurandy
Número total de Autores: 4
|
| Tipo de documento: | Artigo Científico |
| Fonte: | 2025 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV); v. N/A, p. 11-pg., 2025-01-01. |
| Resumo | |
Unsupervised domain adaptation (UDA) in videos is a challenging task that remains not well explored compared to image-based UDA techniques. Although vision transformers (ViT) achieve state-of-the-art performance in many computer vision tasks, their use in video UDA has been little explored. Our key idea is to use transformer layers as a feature encoder and incorporate spatial and temporal transferability relationships into the attention mechanism. A Transferable-guided Attention (TransferAttn) framework is then developed to exploit the capacity of the transformer to adapt cross-domain knowledge across different backbones. To improve the transferability of ViT, we introduce a novel and effective module, named Domain Transferable-guided Attention Block (DTAB). DTAB compels ViT to focus on the spatio-temporal transferability relationship among video frames by changing the self-attention mechanism to a transferability attention mechanism. Extensive experiments were conducted on UCF-HMDB, Kinetics-Gameplay, and Kinetics-NEC Drone datasets, with different backbones, like ResNet101, I3D, and STAM, to verify the effectiveness of TransferAttn compared with state-of-the-art approaches. Also, we demonstrate that DTAB yields performance gains when applied to other state-of-the-art transformer-based UDA methods from both video and image domains. Our code is available at https:// github.com/ Andre-Sacilotti/transferattn-project-code. (AU) | |
| Processo FAPESP: | 24/04500-2 - Aprendizado profundo para visão computacional: melhorando a generalização com poucos dados |
| Beneficiário: | Samuel Felipe dos Santos |
| Modalidade de apoio: | Bolsas no Brasil - Pós-Doutorado |
| Processo FAPESP: | 23/17577-0 - Compreensão de vídeo por meio de aprendizagem profunda com mínima supervisão humana |
| Beneficiário: | Jurandy Gomes de Almeida Junior |
| Modalidade de apoio: | Auxílio à Pesquisa - Regular |