Time as an exploration factor for recommender systems based on reinforcement learning

• Grant number: 23/00158-5
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Effective date (Start): April 01, 2023
• Effective date (End): March 31, 2024
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Tiago Agostinho de Almeida
• Grantee: Gregório Fornetti Azevedo
• Host Institution: Centro de Ciências em Gestão e Tecnologia (CCGT). Universidade Federal de São Carlos (UFSCAR). Campus de Sorocaba. Sorocaba , SP, Brazil

Abstract

Currently, with the increasing popularization of technology, it has become easy to provide and access a large amount of information online. This scenario resulted in a problem for digital users in finding what is of their interests among all available options. Recommender systems were then proposed, which a common goal of filtering a subset of relevant items for a specific user. The area experienced an impressive growth in the last decades, being today one of the main strategies for companies to provide a pleasant experience for its customers. Since the first recommender algorithms, it has prevailed a static and non-incremental approach, i.e., algorithms are trained with fixed bacthes of datasets captured in the past. However, this strategy is not faithful to the practical scenario of the recommendation, in which users are constantly receiving recommendations and generating feedback. In order to address this characteristic, recent studies seek to approach the problem employing algorithms based on reinforcement learning. This type of algorithm operates incrementally, by training intelligent agents through a trial-and-error proccess at runtime. One of the main dilemmas within the area is the balance between exploration and exploitation, that is, the practice of discovering new knowledge about the user or in specializing in already existing knowledge. Strategies regadring this problem that are commonly used in other areas can be very detrimental to the recommendation scenario, and little has been studied about the use of temporal information for this purpose. This project seeks to develop improved techniques that balance exploration and exploitation through consumption of temporal data. With this strategy, we expect that the agents will be capable of generating impotant knowledge about the users, thus improving the final recommendation.