Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach

Detalhes bibliográficos
Autor(a) principal: Emami, Yousef
Data de Publicação: 2021
Outros Autores: Wei, Bo, Li, Kai, Ni, Wei, Tovar, Eduardo
Tipo de documento: Artigo
Idioma: eng
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10400.22/18706
Resumo: Recently, Unmanned Aerial Vehicle (UAV) swarm has been increasingly studied to collect data from ground sensors in remote and hostile areas. A key challenge is the joint design of the velocities and data collection schedules of the UAVs, as inadequate velocities and schedules would lead to failed transmissions and buffer overflows of sensors and, in turn, significant packet losses. In this paper, we optimize jointly the velocity controls and data collection schedules of multiple UAVs to minimize data losses, adapting to the battery levels, queue lengths and channel conditions of the ground sensors, and the trajectories of the UAVs. In the absence of the upto-date knowledge of the ground sensors' states, a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MADRL-SA) is proposed to allow the UAVs to asymptotically minimize the data loss of the system under the outdated knowledge of the network states at individual UAVs. Numerical results demonstrate that the proposed MADRL-SA reduces the packet loss by up to 54\% and 46\% in the considered simulation setting, as compared to an existing DRL solution with single-UAV and non-learning greedy heuristic, respectively.
id RCAP_ab762d82c611f4252c3cd912e31b7023
oai_identifier_str oai:recipp.ipp.pt:10400.22/18706
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach210903Unmanned aerial vehiclesCommunication schedulingVelocity controlMulti-UAV Deep Reinforcement LearningDeep Q-NetworkRecently, Unmanned Aerial Vehicle (UAV) swarm has been increasingly studied to collect data from ground sensors in remote and hostile areas. A key challenge is the joint design of the velocities and data collection schedules of the UAVs, as inadequate velocities and schedules would lead to failed transmissions and buffer overflows of sensors and, in turn, significant packet losses. In this paper, we optimize jointly the velocity controls and data collection schedules of multiple UAVs to minimize data losses, adapting to the battery levels, queue lengths and channel conditions of the ground sensors, and the trajectories of the UAVs. In the absence of the upto-date knowledge of the ground sensors' states, a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MADRL-SA) is proposed to allow the UAVs to asymptotically minimize the data loss of the system under the outdated knowledge of the network states at individual UAVs. Numerical results demonstrate that the proposed MADRL-SA reduces the packet loss by up to 54\% and 46\% in the considered simulation setting, as compared to an existing DRL solution with single-UAV and non-learning greedy heuristic, respectively.IEEERepositório Científico do Instituto Politécnico do PortoEmami, YousefWei, BoLi, KaiNi, WeiTovar, Eduardo2021-09-282031-01-01T00:00:00Z2021-09-28T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10400.22/18706eng10.1109/TVT.2021.3110801info:eu-repo/semantics/embargoedAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-03-13T13:10:49Zoai:recipp.ipp.pt:10400.22/18706Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T17:38:12.657808Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
210903
title Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
spellingShingle Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
Emami, Yousef
Unmanned aerial vehicles
Communication scheduling
Velocity control
Multi-UAV Deep Reinforcement Learning
Deep Q-Network
title_short Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
title_full Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
title_fullStr Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
title_full_unstemmed Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
title_sort Joint Communication Scheduling and Velocity Control in Multi-UAV-Assisted Sensor Networks: A Deep Reinforcement Learning Approach
author Emami, Yousef
author_facet Emami, Yousef
Wei, Bo
Li, Kai
Ni, Wei
Tovar, Eduardo
author_role author
author2 Wei, Bo
Li, Kai
Ni, Wei
Tovar, Eduardo
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Repositório Científico do Instituto Politécnico do Porto
dc.contributor.author.fl_str_mv Emami, Yousef
Wei, Bo
Li, Kai
Ni, Wei
Tovar, Eduardo
dc.subject.por.fl_str_mv Unmanned aerial vehicles
Communication scheduling
Velocity control
Multi-UAV Deep Reinforcement Learning
Deep Q-Network
topic Unmanned aerial vehicles
Communication scheduling
Velocity control
Multi-UAV Deep Reinforcement Learning
Deep Q-Network
description Recently, Unmanned Aerial Vehicle (UAV) swarm has been increasingly studied to collect data from ground sensors in remote and hostile areas. A key challenge is the joint design of the velocities and data collection schedules of the UAVs, as inadequate velocities and schedules would lead to failed transmissions and buffer overflows of sensors and, in turn, significant packet losses. In this paper, we optimize jointly the velocity controls and data collection schedules of multiple UAVs to minimize data losses, adapting to the battery levels, queue lengths and channel conditions of the ground sensors, and the trajectories of the UAVs. In the absence of the upto-date knowledge of the ground sensors' states, a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MADRL-SA) is proposed to allow the UAVs to asymptotically minimize the data loss of the system under the outdated knowledge of the network states at individual UAVs. Numerical results demonstrate that the proposed MADRL-SA reduces the packet loss by up to 54\% and 46\% in the considered simulation setting, as compared to an existing DRL solution with single-UAV and non-learning greedy heuristic, respectively.
publishDate 2021
dc.date.none.fl_str_mv 2021-09-28
2021-09-28T00:00:00Z
2031-01-01T00:00:00Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10400.22/18706
url http://hdl.handle.net/10400.22/18706
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv 10.1109/TVT.2021.3110801
dc.rights.driver.fl_str_mv info:eu-repo/semantics/embargoedAccess
eu_rights_str_mv embargoedAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv IEEE
publisher.none.fl_str_mv IEEE
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799131471803318272