obust image features for classification and zero-shot tasks by merging visual and semantic attribute

Autor(es)

Oliveira de Resende, Damares Crystina ^[1] ; Ponti, Moacir Antonelli ^[1]

Afiliação do(s) autor(es)

^[1] Univ Sao Paulo, ICMC, Sao Carlos - Brazil

• Tipo de documento: Artigo Científico
• Fonte: NEURAL COMPUTING & APPLICATIONS; v. 34, n. 6 JAN 2022.
• Citações Web of Science: 0

Resumo

We investigate visual-semantic representations by combining visual features and semantic attributes to form a compact subspace containing the most relevant properties of each domain. This subspace can better represent image features for recognition tasks and even allow to better interpret results in the light of the nature of semantic attributes, offering a path for explainable learning. Experiments were performed in four benchmark datasets and compared against state-of-the-art algorithms. The method shows to be robust for up to 20% degradation of semantic attributes and offering possibilities for future work on the deployment of an automatic gathering of semantic data to improve representations for image classification. Additionally, empirical evidence suggests the high-level concepts adds linearity to the feature space, allowing for example PCA and SVM to perform well in the combined visual and semantic features. Also, the representations allow for zero-shot learning which demonstrates the viability of merging semantic and visual data at both training and test time for learning aspects that transcend class boundaries that allow the classification of unseen data. (AU)

• Processo FAPESP: 18/22482-0 - Aprendendo características de conteúdo visual sob condições de supervisão limitada utilizando múltiplos domínios
• Beneficiário: Moacir Antonelli Ponti
• Modalidade de apoio: Auxílio à Pesquisa - Regular
• Processo FAPESP: 19/07316-0 - Teoria de singularidades e aplicações a geometria diferencial, equações diferenciais e visão computacional
• Beneficiário: Farid Tari
• Modalidade de apoio: Auxílio à Pesquisa - Temático