On the equivalence of the integral and differential Bellman equations in impulse control problems
Autor(a) principal: | |
---|---|
Data de Publicação: | 2020 |
Outros Autores: | , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10773/31153 |
Resumo: | When solving optimal impulse control problems, one can use the dynamic programming approach in two different ways: at each time moment, one can make the decision whether to apply a particular type of impulse, leading to the instantaneous change of the state, or apply no impulses at all; or, otherwise, one can plan an impulse after a certain interval, so that the length of that interval is to be optimized along with the type of that impulse. The first method leads to the differential Bellman equation, while the second method leads to the integral Bellman equation. The target of the current article is to prove the equivalence of those Bellman equations. Firstly, we prove that, for the simple deterministic optimal stopping problem, the equations in the integral and differential form are equivalent under very mild conditions. Here, the impulse means that the uncontrolled process is stopped, i.e., sent to the so called cemetery. After that, the obtained result immediately implies the similar equivalence of the Bellman equations for other models of optimal impulse control. Those include abstract dynamical systems, controlled ordinary differential equations, piece-wise deterministic Markov processes and continuous-time Markov decision processes. |
id |
RCAP_dbabf52eae679e88c7a93a88728f1e62 |
---|---|
oai_identifier_str |
oai:ria.ua.pt:10773/31153 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
On the equivalence of the integral and differential Bellman equations in impulse control problemsDynamical systemOptimal stoppingImpulse controlDynamic programmingTotal costWhen solving optimal impulse control problems, one can use the dynamic programming approach in two different ways: at each time moment, one can make the decision whether to apply a particular type of impulse, leading to the instantaneous change of the state, or apply no impulses at all; or, otherwise, one can plan an impulse after a certain interval, so that the length of that interval is to be optimized along with the type of that impulse. The first method leads to the differential Bellman equation, while the second method leads to the integral Bellman equation. The target of the current article is to prove the equivalence of those Bellman equations. Firstly, we prove that, for the simple deterministic optimal stopping problem, the equations in the integral and differential form are equivalent under very mild conditions. Here, the impulse means that the uncontrolled process is stopped, i.e., sent to the so called cemetery. After that, the obtained result immediately implies the similar equivalence of the Bellman equations for other models of optimal impulse control. Those include abstract dynamical systems, controlled ordinary differential equations, piece-wise deterministic Markov processes and continuous-time Markov decision processes.Taylor & Francis2021-06-30T00:00:00Z2020-06-30T00:00:00Z2020-06-30info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10773/31153eng0020-717910.1080/00207179.2020.1786766Dufour, FrancoisPiunovskiy, AlexeyPlakhov, Alexanderinfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-02-22T12:00:09Zoai:ria.ua.pt:10773/31153Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T03:03:06.636624Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
On the equivalence of the integral and differential Bellman equations in impulse control problems |
title |
On the equivalence of the integral and differential Bellman equations in impulse control problems |
spellingShingle |
On the equivalence of the integral and differential Bellman equations in impulse control problems Dufour, Francois Dynamical system Optimal stopping Impulse control Dynamic programming Total cost |
title_short |
On the equivalence of the integral and differential Bellman equations in impulse control problems |
title_full |
On the equivalence of the integral and differential Bellman equations in impulse control problems |
title_fullStr |
On the equivalence of the integral and differential Bellman equations in impulse control problems |
title_full_unstemmed |
On the equivalence of the integral and differential Bellman equations in impulse control problems |
title_sort |
On the equivalence of the integral and differential Bellman equations in impulse control problems |
author |
Dufour, Francois |
author_facet |
Dufour, Francois Piunovskiy, Alexey Plakhov, Alexander |
author_role |
author |
author2 |
Piunovskiy, Alexey Plakhov, Alexander |
author2_role |
author author |
dc.contributor.author.fl_str_mv |
Dufour, Francois Piunovskiy, Alexey Plakhov, Alexander |
dc.subject.por.fl_str_mv |
Dynamical system Optimal stopping Impulse control Dynamic programming Total cost |
topic |
Dynamical system Optimal stopping Impulse control Dynamic programming Total cost |
description |
When solving optimal impulse control problems, one can use the dynamic programming approach in two different ways: at each time moment, one can make the decision whether to apply a particular type of impulse, leading to the instantaneous change of the state, or apply no impulses at all; or, otherwise, one can plan an impulse after a certain interval, so that the length of that interval is to be optimized along with the type of that impulse. The first method leads to the differential Bellman equation, while the second method leads to the integral Bellman equation. The target of the current article is to prove the equivalence of those Bellman equations. Firstly, we prove that, for the simple deterministic optimal stopping problem, the equations in the integral and differential form are equivalent under very mild conditions. Here, the impulse means that the uncontrolled process is stopped, i.e., sent to the so called cemetery. After that, the obtained result immediately implies the similar equivalence of the Bellman equations for other models of optimal impulse control. Those include abstract dynamical systems, controlled ordinary differential equations, piece-wise deterministic Markov processes and continuous-time Markov decision processes. |
publishDate |
2020 |
dc.date.none.fl_str_mv |
2020-06-30T00:00:00Z 2020-06-30 2021-06-30T00:00:00Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10773/31153 |
url |
http://hdl.handle.net/10773/31153 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
0020-7179 10.1080/00207179.2020.1786766 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.publisher.none.fl_str_mv |
Taylor & Francis |
publisher.none.fl_str_mv |
Taylor & Francis |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799137686393454592 |