Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning

Detalhes bibliográficos
Autor(a) principal: Li, Changan
Data de Publicação: 2022
Outros Autores: Wu, Sirui, Li, Zhan, Zhang, Yuxiao, Zhang, Lijie, Gomes, Luís
Tipo de documento: Artigo
Idioma: eng
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10362/143923
Resumo: Funding Information: Funding: This research was funded by the National Natural Science Foundation of China under Grant U1964201 and Grant U21B6001, the Major Scientific and Technological Special Project of Hei-longjiang Province under Grant 2021ZX05A01, the Heilongjiang Natural Science Foundation under Grant LH2019F020, and the Major Scientific and Technological Research Project of Ningbo under Grant 2021Z040. Publisher Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland.
id RCAP_10966a2047f202cc10d6d098f4340d4a
oai_identifier_str oai:run.unl.pt:10362/143923
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learningbulk cargo loadingdeep reinforcement learningintelligent schedulingMDP modelControl and Systems EngineeringSignal ProcessingHardware and ArchitectureComputer Networks and CommunicationsElectrical and Electronic EngineeringFunding Information: Funding: This research was funded by the National Natural Science Foundation of China under Grant U1964201 and Grant U21B6001, the Major Scientific and Technological Special Project of Hei-longjiang Province under Grant 2021ZX05A01, the Heilongjiang Natural Science Foundation under Grant LH2019F020, and the Major Scientific and Technological Research Project of Ningbo under Grant 2021Z040. Publisher Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland.Sea freight is one of the most important ways for the transportation and distribution of coal and other bulk cargo. This paper proposes a method for optimizing the scheduling efficiency of the bulk cargo loading process based on deep reinforcement learning. The process includes a large number of states and possible choices that need to be taken into account, which are currently performed by skillful scheduling engineers on site. In terms of modeling, we extracted important information based on actual working data of the terminal to form the state space of the model. The yard information and the demand information of the ship are also considered. The scheduling output of each convey path from the yard to the cabin is the action of the agent. To avoid conflicts of occupying one machine at same time, certain restrictions are placed on whether the action can be executed. Based on Double DQN, an improved deep reinforcement learning method is proposed with a fully connected network structure and selected action sets according to the value of the network and the occupancy status of environment. To make the network converge more quickly, an improved new epsilon-greedy exploration strategy is also proposed, which uses different exploration rates for completely random selection and feasible random selection of actions. After training, an improved scheduling result is obtained when the tasks arrive randomly and the yard state is random. An important contribution of this paper is to integrate the useful features of the working time of the bulk cargo terminal into a state set, divide the scheduling process into discrete actions, and then reduce the scheduling problem into simple inputs and outputs. Another major contribution of this article is the design of a reinforcement learning algorithm for the bulk cargo terminal scheduling problem, and the training efficiency of the proposed algorithm is improved, which provides a practical example for solving bulk cargo terminal scheduling problems using reinforcement learning.CTS - Centro de Tecnologia e SistemasRUNLi, ChanganWu, SiruiLi, ZhanZhang, YuxiaoZhang, LijieGomes, Luís2022-09-21T22:37:40Z2022-04-272022-04-27T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/article18application/pdfhttp://hdl.handle.net/10362/143923eng2079-9292PURE: 46646499https://doi.org/10.3390/electronics11091390info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-03-11T05:22:53Zoai:run.unl.pt:10362/143923Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T03:51:13.145977Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
title Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
spellingShingle Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
Li, Changan
bulk cargo loading
deep reinforcement learning
intelligent scheduling
MDP model
Control and Systems Engineering
Signal Processing
Hardware and Architecture
Computer Networks and Communications
Electrical and Electronic Engineering
title_short Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
title_full Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
title_fullStr Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
title_full_unstemmed Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
title_sort Intelligent Scheduling Method for Bulk Cargo Terminal Loading Process Based on Deep Reinforcement Learning
author Li, Changan
author_facet Li, Changan
Wu, Sirui
Li, Zhan
Zhang, Yuxiao
Zhang, Lijie
Gomes, Luís
author_role author
author2 Wu, Sirui
Li, Zhan
Zhang, Yuxiao
Zhang, Lijie
Gomes, Luís
author2_role author
author
author
author
author
dc.contributor.none.fl_str_mv CTS - Centro de Tecnologia e Sistemas
RUN
dc.contributor.author.fl_str_mv Li, Changan
Wu, Sirui
Li, Zhan
Zhang, Yuxiao
Zhang, Lijie
Gomes, Luís
dc.subject.por.fl_str_mv bulk cargo loading
deep reinforcement learning
intelligent scheduling
MDP model
Control and Systems Engineering
Signal Processing
Hardware and Architecture
Computer Networks and Communications
Electrical and Electronic Engineering
topic bulk cargo loading
deep reinforcement learning
intelligent scheduling
MDP model
Control and Systems Engineering
Signal Processing
Hardware and Architecture
Computer Networks and Communications
Electrical and Electronic Engineering
description Funding Information: Funding: This research was funded by the National Natural Science Foundation of China under Grant U1964201 and Grant U21B6001, the Major Scientific and Technological Special Project of Hei-longjiang Province under Grant 2021ZX05A01, the Heilongjiang Natural Science Foundation under Grant LH2019F020, and the Major Scientific and Technological Research Project of Ningbo under Grant 2021Z040. Publisher Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland.
publishDate 2022
dc.date.none.fl_str_mv 2022-09-21T22:37:40Z
2022-04-27
2022-04-27T00:00:00Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10362/143923
url http://hdl.handle.net/10362/143923
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv 2079-9292
PURE: 46646499
https://doi.org/10.3390/electronics11091390
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 18
application/pdf
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799138106830487552

Registros relacionados