Time-aware exploration and state representation for incremental recommender systems

• Grant number: 24/15919-4
• Support Opportunities: Scholarships abroad - Research Internship - Doctorate
• Start date: November 22, 2024
• End date: March 16, 2025
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Tiago Agostinho de Almeida
• Grantee: Pedro Reis Pires
• Supervisor: Joao Manuel Portela da Gama
• Host Institution: Centro de Ciências em Gestão e Tecnologia (CCGT). Universidade Federal de São Carlos (UFSCAR). Campus de Sorocaba. Sorocaba , SP, Brazil
• Institution abroad: Universidade do Porto (UP), Portugal
• Associated to the scholarship: 21/14591-7 - Time-aware top-N recommender systems based on reinforcement learning, BP.DR

Abstract

With the constant popularization of technology, recommender systems have become increasingly important in digital media. Their main objective is to recommend a subset of relevant items to a specific user, helping them discover new interests. Since the beginning of the field, using a static and non-incremental approach is common, in which algorithms are trained with a fixed dataset, captured in the past. However, the practical recommendation scenario operates sequentially: the system generates recommendations for the user, who immediately provides feedback. Recent studies in the field aim for models that take advantage of this property of the problem. Based on reinforcement learning strategies, these methods learn incrementally, thus generating a recommender agent that adapts to the user's interests over time. As this research area is quite recent, it still has many open challenges and space for development. Although reinforcement learning has, by definition, sequential temporal knowledge of interactions, little is known about the explicit use of time during training, as is done in time-aware recommender systems, which have already established themselves as promising techniques within the area. This study aims to propose new techniques for a top-N recommendation based on reinforcement learning, using the consumption of temporal attributes to increase the knowledge of agents as the main strategy, acting in different stages of reinforcement learning, such as exploration and state representation.