iRisk: A Scalable Microservice for Classifying Issue Risks Based on Crowdsourced App Reviews

Author(s)

Alves de Lima, Vitor Mesaque ; Barbosa, Jacson Rodrigues ; Marcacini, Ricardo Marcodes

• Document type: Journal article
• Source: 2024 IEEE INTERNATIONAL CONFERENCE ON SOFTWARE MAINTENANCE AND EVOLUTION, ICSME 2024; v. N/A, p. 5-pg., 2024-01-01.

Abstract

Analyzing mobile app reviews is essential for identifying trends and issue patterns that affect user experience and app reputation in app stores. A risk matrix provides a straightforward, intuitive method to prioritize software maintenance actions to mitigate negative ratings. However, manually constructing a risk matrix is time-consuming, and stakeholders often struggle to understand the context of risks due to varied descriptions and the sheer volume of reviews. Therefore, machine learning-based methods are needed to extract risks and classify their priority effectively. While existing studies have automated risk matrix generation in software development, they have not explored app reviews or utilized Large Language Models (LLMs) in a scalable architecture. To address this gap, we present iRisk (scalable microservice for classifying issue Risks), a tool for generating a risk matrix based on crowdsourced app reviews using LLM. We present i-LLAMA, a fine-tuned version of LLaMA 3, optimized to detect and prioritize app-related issues using a risk analysis dataset of reviews categorized by severity and likelihood of occurrence. This dataset is also publicly available. Our contributions include the open-source resources to support the software maintenance and evolution industry, fine-tuning of LLaMA 3, and a scalable microservice architecture to handle large volumes of data. The iRisk can manage app issues and risks and provide an automated dashboard and visualizations for decision-making, monitoring, and risk mitigation. The tool is available on GitHub(1), and a presentation about the tool can be found in this video(2). (AU)

• FAPESP's process: 19/25010-5 - Semantically enriched representations for Portuguese textmining: models and applications
• Grantee: Solange Oliveira Rezende
• Support Opportunities: Regular Research Grants
• FAPESP's process: 19/07665-4 - Center for Artificial Intelligence
• Grantee: Fabio Gagliardi Cozman
• Support Opportunities: Research Grants - Research Program in eScience and Data Science - Research Centers in Engineering Program