Systematic Investigation of Error Distribution in Machine Learning Algorithms Applied to the Quantum-Chemistry QM9 Data Set Using the Bias and Variance Decomposition

Autor(es)

de Azevedo, Luis Cesar ^[1] ; Pinheiro, Gabriel A. ^[2] ; Quiles, Marcos G. ^[2] ; Da Silva, Juarez L. F. ^[3] ; Prati, Ronaldo C. ^[1]

Afiliação do(s) autor(es)

^[1] Fed Univ ABC, Ctr Math Computat & Cognit, BR-5001 Santo Andre, SP - Brazil

^[2] Fed Univ Sao Paulo UNIFESP, Inst Sci & Technol, BR-12247014 Sao Jose Dos Campos, SP - Brazil

^[3] Univ Sao Paulo, Sao Carlos Inst Chem, BR-13560970 Sao Carlos, SP - Brazil

• Tipo de documento: Artigo Científico
• Fonte: JOURNAL OF CHEMICAL INFORMATION AND MODELING; v. 61, n. 9, p. 4210-4223, SEP 27 2021.
• Citações Web of Science: 0

Resumo

Most machine learning applications in quantum-chemistry (QC) data sets rely on a single statistical error parameter such as the mean square error (MSE) to evaluate their performance. However, this approach has limitations or can even yield incorrect interpretations. Here, we report a systematic investigation of the two components of the MSE, i.e., the bias and variance, using the QM9 data set. To this end, we experiment with three descriptors, namely (i) symmetry functions (SF, with two-body and three-body functions), (ii) many-body tensor representation (MBTR, with two- and three-body terms), and (iii) smooth overlap of atomic positions (SOAP), to evaluate the prediction process's performance using different numbers of molecules in training samples and the effect of bias and variance on the final MSE. Overall, low sample sizes are related to higher MSE. Moreover, the bias component strongly influences the larger MSEs. Furthermore, there is little agreement among molecules with higher errors (outliers) across different descriptors. However, there is a high prevalence among the outliers intersection set and the convex hull volume of geometric coordinates (VCH). According to the obtained results with the distribution of MSE (and its components bias and variance) and the appearance of outliers, it is suggested to use ensembles of models with a low bias to minimize the MSE, more specifically when using a small number of molecules in the training set. (AU)

• Processo FAPESP: 17/11631-2 - CINE: desenvolvimento computacional de materiais utilizando simulações atomísticas, meso-escala, multi-física e inteligência artificial para aplicações energéticas
• Beneficiário: Juarez Lopes Ferreira da Silva
• Modalidade de apoio: Auxílio à Pesquisa - Programa Centros de Pesquisa em Engenharia