Connoisseur: Provenance Analysis in Paintings

Author(s)

David, Lucas ; Pedrini, Helio ; Dias, Zanoni ; Rocha, Anderson ; IEEE

• Document type: Journal article
• Source: 2021 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI 2021); v. N/A, p. 8-pg., 2021-01-01.

Abstract

Authorship attribution and matching have become paramount activities in current digital art repositories and communities, which seek to efficiently catalog and authenticate the ever-growing number of digitized paintings, uploaded in professional and casual capturing setups, by their own authors or enthusiasts alike. In this work, we employ convolutional network-based strategies to identify and classify art-related digital artifacts over the Painter by Numbers dataset. Firstly, we propose to exploit the authorship, style and genre annotated information in a multi-task setup, in which patches of paintings are encoded through a multiple outputs network and, in a second stage, used in an Siamese discriminating network to solve the authorship matching problem. Secondly, we combine the available annotated information in a more efficient manner, by posing the Painter by Numbers challenge as a multi-label problem. Empirical results show a substantial increase in class-balanced accuracy and ROC AUC score for both multi-task solutions, compared with their simpler counterparts trained using only authorship annotation. Furthermore, a slight increase in ROC AUC score is observed in the multi-label setup, indicating that this simple combination strategy is beneficial to training convergence. (AU)

• FAPESP's process: 17/12646-3 - Déjà vu: feature-space-time coherence from heterogeneous data for media integrity analytics and interpretation of events
• Grantee: Anderson de Rezende Rocha
• Support Opportunities: Research Projects - Thematic Grants
• FAPESP's process: 15/11937-9 - Investigation of hard problems from the algorithmic and structural stand points
• Grantee: Flávio Keidi Miyazawa
• Support Opportunities: Research Projects - Thematic Grants
• FAPESP's process: 17/16246-0 - Sensitive media analysis through deep learning architectures
• Grantee: Sandra Eliza Fontes de Avila
• Support Opportunities: Regular Research Grants