Adaptive discounted control for piecewise deterministic Markov processes

Author(s)

Costa, O. L. V. ; Dufour, F.

• Document type: Journal article
• Source: Journal of Mathematical Analysis and Applications; v. 528, n. 2, p. 23-pg., 2023-06-27.

Abstract

The main goal of this paper is to study the adaptive infinite-horizon discounted continuous-time optimal control problem of piecewise deterministic Markov pro-cesses (PDMPs) with the control acting continuously on the jump intensity A and on the transition measure Q of the process. It is assumed that jump parameters (A and Q), as well the continuous and boundary costs (Cg and Ci respectively), de-pend on an unknown parameter 3*. It is shown that the principle of estimation and control holds, that is, the strategy consisting of choosing, at each stage n, an action according to an optimal stationary policy, where the true but unknown parameter 3 ⠂n, is asymptotically discount optimal, pro-3 ⠂n 3* is replaced by its estimated value vided that the sequence of estimators {3 ⠂n} of 3* is strongly consistent, that is, converge to 3* almost surely. In the framework of PDMPs, the so-called discrepancy function depends on the derivative along the flow of the value function as well as on some boundary conditions, which brings new challenges in the analysis of this problem.& COPY; 2023 Elsevier Inc. All rights reserved. (AU)

• FAPESP's process: 14/50851-0 - INCT 2014: National Institute of Science and Technology for Cooperative Autonomous Systems Applied in Security and Environment
• Grantee: Marco Henrique Terra
• Support Opportunities: Research Projects - Thematic Grants