A linear programming formulation for constrained discounted continuous control for piecewise deterministic Markov processes

Autor(es)

Costa, O. L. V. ^[1] ; Dufour, F. ^[2]

Afiliação do(s) autor(es)

^[1] Escola Politecn Univ Sao Paulo, Dept Engn Telecomun & Controle, BR-05508900 Sao Paulo - Brazil

^[2] Univ Bordeaux, IMB, INRIA Bordeaux Sud Quest, Team CQPD, F-33405 Talence - France

• Tipo de documento: Artigo Científico
• Fonte: Journal of Mathematical Analysis and Applications; v. 424, n. 2, p. 892-914, APR 15 2015.
• Citações Web of Science: 5

Resumo

This paper deals with the constrained discounted control of piecewise deterministic Markov process (PDMPs) in general Borel spaces. The control variable acts on the jump rate and transition measure, and the goal is to minimize the total expected discounted cost, composed of positive running and boundary costs, while satisfying some constraints also in this form. The basic idea is, by using the special features of the PDMPs, to re-write the problem via an embedded discrete-time Markov chain associated to the PDMP and re-formulate the problem as an infinite dimensional linear programming (LP) problem, via the occupation measures associated to the discrete-time process. It is important to stress however that our new discrete-time problem is not in the same framework of a general constrained discrete-time Markov Decision Process and, due to that, some conditions are required to get the equivalence between the continuous-time problem and the LP formulation. We provide in the sequel sufficient conditions for the solvability of the associated LP problem, based on a generalization of Theorem 4.1 in {[}8]. In Appendix A we present the proof of this generalization which, we believe, is of interest on its own. The paper is concluded with some examples to illustrate the obtained results. (C) 2014 Elsevier Inc. All rights reserved. (AU)

• Processo FAPESP: 13/50759-3 - Controle de sistemas dinâmicos sujeitos a saltos estocásticos
• Beneficiário: Eduardo Fontoura Costa
• Modalidade de apoio: Auxílio à Pesquisa - Regular