Designing cell signaling pathway dynamic models through the combination of differential equations and transformers pre-trained on biological data

• Grant number: 24/19180-3
• Support Opportunities: Scholarships in Brazil - Doctorate
• Start date: March 01, 2025
• End date: February 28, 2029
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Marcelo da Silva Reis
• Grantee: Carolina Albuquerque Massena Ribeiro
• Host Institution: Instituto de Computação (IC). Universidade Estadual de Campinas (UNICAMP). Campinas , SP, Brazil

Abstract

Cell signaling pathways regulate essential cellular processes, such as specialization, differentiation and apoptosis, through complex networks of biochemical operations. However, traditional models based on chemical kinetics described by ordinary differential equations (ODEs), although practical in describing the dynamics of isolated components, have difficulties in capturing the complex and nonlinear interactions in the broader cellular context, known as the ``lack of isolation problem". To overcome this challenge, our group proposed a modeling of signaling pathways using ODEs, which describe the kinetics of pathway reactions, combined with neural networks, which fulfill the role of communication of the pathway with the remainder of the cell. Results obtained with this strategy, implemented using the formalism of universal differential equations (UDEs), were promising; however, we verified two limitations: i) the model training only makes use of measurements of chemical species present in the reactions; ii) it was used a very simple neural network architecture, the multilayer perceptron (MLP), which might fail to capture relevant contextual cellular information. Therefore, here we propose a hybrid approach for modeling of cellular signaling pathways that combines the usage of UDEs with transformers pre-trained on large sets of real biological data, both proteomic and transcriptomic ones. The transformers would be plugged into UDEs by training MLP layers, to this end using the time series of the pathway species themselves. With this strategy, we expect to achieve significant gains in the quality of the prediction and in the interpretability of the model (due to the transformer's self-attention layers), thus being able to predict the inner workings of molecular mechanisms in different contexts, in particular in two case studies in cancer biology.