Interpretation and explanation of computer vision classification of carambola ( Averrhoa carambola L. ) according to maturity stage

Autor(es)

de Moraes, Ingrid Alves ; Barbon Junior, Sylvio ; Barbin, Douglas Fernandes

• Tipo de documento: Artigo Científico
• Fonte: Food Research International; v. 192, p. 10-pg., 2024-07-31.

Resumo

The classification of carambola, also known as starfruit, according to quality parameters is usually conducted by trained human evaluators through visual inspections. This is a costly and subjective method that can generate high variability in results. As an alternative, computer vision systems (CVS) combined with deep learning (DCVS) techniques have been introduced in the industry as a powerful and an innovative tool for the rapid and noninvasive classification of fruits. However, validating the learning capability and trustworthiness of a DL model, aka black box, to obtain insights can be challenging. To reduce this gap, we propose an integrated eXplainable Artificial Intelligence (XAI) method for the classification of carambolas at different maturity stages. We compared two Residual Neural Networks (ResNet) and Visual Transformers (ViT) to identify the image regions that are enhanced by a Random Forest (RF) model, with the aim of providing more detailed information at the feature level for classifying the maturity stage. Changes in fruit colour and physicochemical data throughout the maturity stages were analysed, and the influence of these parameters on the maturity stages was evaluated using the Gradient-weighted Class Activation Mapping (Grad-CAM), the Attention Maps using RF importance. The proposed approach provides a visualization and description of the most important regions that led to the model decision, in wide visualization follows the models an importance features from RF. Our approach has promising potential for standardized and rapid carambolas classification, achieving 91 % accuracy with ResNet and 95 % with ViT, with potential application for other fruits. (AU)

• Processo FAPESP: 19/27354-3 - Arquitetura de sistemas de liberação coloidais: qual é o papel da estrutura na digestibilidade?
• Beneficiário: Rosiane Lopes da Cunha
• Modalidade de apoio: Auxílio à Pesquisa - Temático
• Processo FAPESP: 23/07385-7 - Aplicações de visão artificial para análise de novas fontes de alimentos
• Beneficiário: Douglas Fernandes Barbin
• Modalidade de apoio: Auxílio à Pesquisa - Regular
• Processo FAPESP: 15/24351-2 - Análise de imagens e espectroscopia de infravermelho próximo (NIR) na avaliação de qualidade e autenticação de alimentos
• Beneficiário: Douglas Fernandes Barbin
• Modalidade de apoio: Auxílio à Pesquisa - Jovens Pesquisadores