Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems
Autor(a) principal: | |
---|---|
Data de Publicação: | 2022 |
Outros Autores: | , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10400.14/38036 |
Resumo: | Problem definition: Is deep reinforcement learning (DRL) effective at solving inventory problems? Academic/practical relevance: Given that DRL has successfully been applied in computer games and robotics, supply chain researchers and companies are interested in its potential in inventory management. We provide a rigorous performance evaluation of DRL in three classic and intractable inventory problems: lost sales, dual sourcing, and multi-echelon inventory management. Methodology: We model each inventory problem as a Markov decision process and apply and tune the Asynchronous Advantage Actor-Critic (A3C) DRL algorithm for a variety of parameter settings. Results: We demonstrate that the A3C algorithm can match the performance of the state-of-the-art heuristics and other approximate dynamic programming methods. Although the initial tuning was computationally demanding and time demanding, only small changes to the tuning parameters were needed for the other studied problems. Managerial implications: Our study provides evidence that DRL can effectively solve stationary inventory problems. This is especially promising when problem-dependent heuristics are lacking. Yet, generating structural policy insight or designing specialized policies that are (ideally provably) near optimal remains desirable. |
id |
RCAP_57d93a9bae5e98e7926d9be41ca66abf |
---|---|
oai_identifier_str |
oai:repositorio.ucp.pt:10400.14/38036 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problemsOM-information technology interfaceInventory theory and controlLogistics and transportationSupply chain managementProblem definition: Is deep reinforcement learning (DRL) effective at solving inventory problems? Academic/practical relevance: Given that DRL has successfully been applied in computer games and robotics, supply chain researchers and companies are interested in its potential in inventory management. We provide a rigorous performance evaluation of DRL in three classic and intractable inventory problems: lost sales, dual sourcing, and multi-echelon inventory management. Methodology: We model each inventory problem as a Markov decision process and apply and tune the Asynchronous Advantage Actor-Critic (A3C) DRL algorithm for a variety of parameter settings. Results: We demonstrate that the A3C algorithm can match the performance of the state-of-the-art heuristics and other approximate dynamic programming methods. Although the initial tuning was computationally demanding and time demanding, only small changes to the tuning parameters were needed for the other studied problems. Managerial implications: Our study provides evidence that DRL can effectively solve stationary inventory problems. This is especially promising when problem-dependent heuristics are lacking. Yet, generating structural policy insight or designing specialized policies that are (ideally provably) near optimal remains desirable.Veritati - Repositório Institucional da Universidade Católica PortuguesaGijsbrechts, JorenBoute, Robert N.Mieghem, Jan A. vanZhang, Dennis J.2022-07-01T08:36:34Z2022-052022-05-01T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10400.14/38036eng1523-461410.1287/msom.2021.106485132222235000803569300001info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-01-16T01:44:10Zoai:repositorio.ucp.pt:10400.14/38036Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T18:30:59.975641Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems |
title |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems |
spellingShingle |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems Gijsbrechts, Joren OM-information technology interface Inventory theory and control Logistics and transportation Supply chain management |
title_short |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems |
title_full |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems |
title_fullStr |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems |
title_full_unstemmed |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems |
title_sort |
Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problems |
author |
Gijsbrechts, Joren |
author_facet |
Gijsbrechts, Joren Boute, Robert N. Mieghem, Jan A. van Zhang, Dennis J. |
author_role |
author |
author2 |
Boute, Robert N. Mieghem, Jan A. van Zhang, Dennis J. |
author2_role |
author author author |
dc.contributor.none.fl_str_mv |
Veritati - Repositório Institucional da Universidade Católica Portuguesa |
dc.contributor.author.fl_str_mv |
Gijsbrechts, Joren Boute, Robert N. Mieghem, Jan A. van Zhang, Dennis J. |
dc.subject.por.fl_str_mv |
OM-information technology interface Inventory theory and control Logistics and transportation Supply chain management |
topic |
OM-information technology interface Inventory theory and control Logistics and transportation Supply chain management |
description |
Problem definition: Is deep reinforcement learning (DRL) effective at solving inventory problems? Academic/practical relevance: Given that DRL has successfully been applied in computer games and robotics, supply chain researchers and companies are interested in its potential in inventory management. We provide a rigorous performance evaluation of DRL in three classic and intractable inventory problems: lost sales, dual sourcing, and multi-echelon inventory management. Methodology: We model each inventory problem as a Markov decision process and apply and tune the Asynchronous Advantage Actor-Critic (A3C) DRL algorithm for a variety of parameter settings. Results: We demonstrate that the A3C algorithm can match the performance of the state-of-the-art heuristics and other approximate dynamic programming methods. Although the initial tuning was computationally demanding and time demanding, only small changes to the tuning parameters were needed for the other studied problems. Managerial implications: Our study provides evidence that DRL can effectively solve stationary inventory problems. This is especially promising when problem-dependent heuristics are lacking. Yet, generating structural policy insight or designing specialized policies that are (ideally provably) near optimal remains desirable. |
publishDate |
2022 |
dc.date.none.fl_str_mv |
2022-07-01T08:36:34Z 2022-05 2022-05-01T00:00:00Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10400.14/38036 |
url |
http://hdl.handle.net/10400.14/38036 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
1523-4614 10.1287/msom.2021.1064 85132222235 000803569300001 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799132033224540160 |