Development of a methodology for the definition of software appraisal proficiency tests

• Grant number: 20/09560-2
• Support Opportunities: Regular Research Grants
• Start date: July 01, 2021
• End date: December 31, 2023
• Field of knowledge: Physical Sciences and Mathematics - Computer Science - Computing Methodologies and Techniques
• Principal Investigator: Simone Do Rocio Senger de Souza
• Grantee: Simone Do Rocio Senger de Souza
• Host Institution: Instituto de Ciências Matemáticas e de Computação (ICMC). Universidade de São Paulo (USP). São Carlos , SP, Brazil
• Associated researchers: Márcio Eduardo Delamaro ; Paulo Roberto de Mesquita Nascimento ; Paulo Sérgio Lopes de Souza ; Raphael Carlos Santos Machado ; Vânia de Oliveira Neves ; Wilson de Souza Melo Junior ; Wladmir Araujo Chapetta
• Associated scholarship(s): 21/11307-6 - Development of tools and resources to support the application of appraisal proficiency of software testing, BP.TT

Abstract

A Proficiency Test consists of evaluating the performance of a set of participants based on pre-established criteria. It is generally applied in contexts where bodies that carry out the conformity assessment of a given product need to have the competence of the participants demonstrated through an inter-comparison. Although Proficiency Tests have very well-defined methods and procedures in different areas of knowledge, the same does not happen when the responsible agencies have to evaluate software products. This difficulty is mainly due to the inherent subjectivity associated with assessing the conformity of a software product, as well as the difficulty in determining efficient metrics to verify how efficient a given evaluation procedure is. In response to this, the present project proposes the development of its own methodology for specifying Proficiency Tests in Software Evaluation, which includes efficient procedures and metrics to determine the degree of competence with which an organism or laboratory can assess and attest to compliance of software products. This initiative is of great interest to academia, industry and governments, considering that more and more providers of software-based solutions are resorting to third party evaluations to attest the conformity of their products, and consequently need to trust that the organizations that provide these products services are duly competent. (AU)