Using Reinforcement Learning to optimize a multivariate Market-Making strategy

• Grant number: 23/16028-3
• Support Opportunities: Scholarships in Brazil - Scientific Initiation
• Start date: August 01, 2024
• End date: July 31, 2025
• Field of knowledge: Physical Sciences and Mathematics - Computer Science
• Principal Investigator: Oswaldo Luiz Do Valle Costa
• Grantee: Rafael Zimmer
• Host Institution: Escola Politécnica (EP). Universidade de São Paulo (USP). São Paulo , SP, Brazil

Abstract

We propose to conduct an analysis on existing financial strategies for market-making, as well as the potential use of Reinforcement Learning (RL) techniques for maximizing the expected return of such strategies. Recent literary advances that use the approach of RL methods for the optimization of financial agents focus on the search for policies that maximize daily return and minimize the risk of the orders and inventory managed by such agents. The risk associated with a market-making strategy comes from the lack of guarantee for the execution of created orders, and depends on the respective process of buy and sell offers. It may happen that the agent has only some or none of their orders executed, even ending the day with a negative return. Considering this definition of risk and a gap in the literature aimed at minimizing post-close market risk - called overnight risk - we propose conducting a literature review on RL techniques for optimizing trading policies, as well as the conceptualization and training of a MM agent that maximizes the expected daily return under the constraint of ending the day without a remaining position, that is, zeroing the overnight risk by interacting in multiple markets.