Practices for Managing Machine Learning Products: A Multivocal Literature Review

Author(s)

Alves, Isaque ; Leite, Leonardo A. F. ; Meirelles, Paulo ; Kon, Fabio ; Aguiar, Carla Silva Rocha

• Document type: Journal article
• Source: IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT; v. N/A, p. 31-pg., 2023-07-06.

Abstract

Machine learning (ML) has grown in popularity in the software industry due to its ability to solve complex problems. Developing ML systems involves more uncertainty and risk because it requires identifying a business opportunity and managing source code, data, and trained models. Our research aims to identify the existing practices used in the industry for building ML applications and comprehending the organizational complexity of adopting ML systems. We conducted a multivocal literature review and then created a taxonomy of the practices applied to the ML system life cycle discussed among practitioners and researchers. The core of the study emerged from 41 selected posts from the grey literature and 37 selected scientific papers. Applying Initial Coding and Focused Coding techniques into these data, we mapped 91 practices into six core categories related to designing, developing, testing, and deploying ML systems. The results, including a taxonomy of practices, provide organizations with valuable insights to identify gaps in their current ML processes and practices and a roadmap for improving, optimizing, and managing ML systems. (AU)

• FAPESP's process: 19/12743-4 - A study about the impact of code annotations in software evolution
• Grantee: Paulo Roberto Miranda Meirelles
• Support Opportunities: Regular Research Grants
• FAPESP's process: 14/50937-1 - INCT 2014: on the Internet of the Future
• Grantee: Fabio Kon
• Support Opportunities: Research Projects - Thematic Grants
• FAPESP's process: 15/24485-9 - Future internet for smart cities
• Grantee: Fabio Kon
• Support Opportunities: Research Projects - Thematic Grants