Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks
Autor(a) principal: | |
---|---|
Data de Publicação: | 2021 |
Outros Autores: | , , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10400.22/18260 |
Resumo: | Unmanned Aerial Vehicles (UAVs) can collaborate to collect and relay data for ground sensors in remote and hostile areas. In multi-UAV-assisted wireless sensor networks (MA-WSN), the UAVs' movements impact on channel condition and can fail data transmission, this situation along with newly arrived data give rise to buffer overflows at the ground sensors. Thus, scheduling data transmission is of utmost importance in MA-WSN to reduce data packet losses resulting from buffer overflows and channel fading. In this paper, we investigate the optimal ground sensor selection at the UAVs to minimize data packet losses . The optimization problem is formulated as a multiagent Markov decision process, where network states consist of battery levels and data buffer lengths of the ground sensor, channel conditions, and waypoints of the UAV along the trajectory. In practice, an MA-WSN contains a large number of network states, while the up-to-date knowledge of the network states and other UAVs' sensor selection decisions is not available at each agent. We propose a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MUAIS) to minimize the data packet loss, where the UAVs learn the underlying patterns of the data and energy arrivals at all the ground sensors. Numerical results show that the proposed MUAIS achieves at least 46\% and 35\% lower packet loss than an optimal solution with single-UAV and an existing non-learning greedy algorithm, respectively. |
id |
RCAP_02783d71ea396a12ec0fd657b48dca24 |
---|---|
oai_identifier_str |
oai:recipp.ipp.pt:10400.22/18260 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks210304Unmanned aerial vehiclesCommunication schedulingMulti-UAV Deep Reinforcement LearningDeep QNetworkUnmanned Aerial Vehicles (UAVs) can collaborate to collect and relay data for ground sensors in remote and hostile areas. In multi-UAV-assisted wireless sensor networks (MA-WSN), the UAVs' movements impact on channel condition and can fail data transmission, this situation along with newly arrived data give rise to buffer overflows at the ground sensors. Thus, scheduling data transmission is of utmost importance in MA-WSN to reduce data packet losses resulting from buffer overflows and channel fading. In this paper, we investigate the optimal ground sensor selection at the UAVs to minimize data packet losses . The optimization problem is formulated as a multiagent Markov decision process, where network states consist of battery levels and data buffer lengths of the ground sensor, channel conditions, and waypoints of the UAV along the trajectory. In practice, an MA-WSN contains a large number of network states, while the up-to-date knowledge of the network states and other UAVs' sensor selection decisions is not available at each agent. We propose a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MUAIS) to minimize the data packet loss, where the UAVs learn the underlying patterns of the data and energy arrivals at all the ground sensors. Numerical results show that the proposed MUAIS achieves at least 46\% and 35\% lower packet loss than an optimal solution with single-UAV and an existing non-learning greedy algorithm, respectively.This work was partially supported by National Funds through FCT/MCTES (Portuguese Foundation for Science and Technology), within the CISTER Research Unit (UIDP/UIDB/04234/2020); also by national funds through the FCT, under CMU Portugal partnership, within project CMU/TIC/0022/2019 (CRUAV).Repositório Científico do Instituto Politécnico do PortoEmami, YousefWei, BoLi, KaiNi, WeiTovar, Eduardo2021-08-30T10:54:52Z2021-07-022021-07-02T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10400.22/18260eng10.1109/IWCMC51323.2021.9498726info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-03-13T13:09:43Zoai:recipp.ipp.pt:10400.22/18260Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T17:37:53.080507Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks 210304 |
title |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks |
spellingShingle |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks Emami, Yousef Unmanned aerial vehicles Communication scheduling Multi-UAV Deep Reinforcement Learning Deep QNetwork |
title_short |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks |
title_full |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks |
title_fullStr |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks |
title_full_unstemmed |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks |
title_sort |
Deep Q-Networks for Aerial Data Collection in Multi-UAV-Assisted Wireless Sensor Networks |
author |
Emami, Yousef |
author_facet |
Emami, Yousef Wei, Bo Li, Kai Ni, Wei Tovar, Eduardo |
author_role |
author |
author2 |
Wei, Bo Li, Kai Ni, Wei Tovar, Eduardo |
author2_role |
author author author author |
dc.contributor.none.fl_str_mv |
Repositório Científico do Instituto Politécnico do Porto |
dc.contributor.author.fl_str_mv |
Emami, Yousef Wei, Bo Li, Kai Ni, Wei Tovar, Eduardo |
dc.subject.por.fl_str_mv |
Unmanned aerial vehicles Communication scheduling Multi-UAV Deep Reinforcement Learning Deep QNetwork |
topic |
Unmanned aerial vehicles Communication scheduling Multi-UAV Deep Reinforcement Learning Deep QNetwork |
description |
Unmanned Aerial Vehicles (UAVs) can collaborate to collect and relay data for ground sensors in remote and hostile areas. In multi-UAV-assisted wireless sensor networks (MA-WSN), the UAVs' movements impact on channel condition and can fail data transmission, this situation along with newly arrived data give rise to buffer overflows at the ground sensors. Thus, scheduling data transmission is of utmost importance in MA-WSN to reduce data packet losses resulting from buffer overflows and channel fading. In this paper, we investigate the optimal ground sensor selection at the UAVs to minimize data packet losses . The optimization problem is formulated as a multiagent Markov decision process, where network states consist of battery levels and data buffer lengths of the ground sensor, channel conditions, and waypoints of the UAV along the trajectory. In practice, an MA-WSN contains a large number of network states, while the up-to-date knowledge of the network states and other UAVs' sensor selection decisions is not available at each agent. We propose a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MUAIS) to minimize the data packet loss, where the UAVs learn the underlying patterns of the data and energy arrivals at all the ground sensors. Numerical results show that the proposed MUAIS achieves at least 46\% and 35\% lower packet loss than an optimal solution with single-UAV and an existing non-learning greedy algorithm, respectively. |
publishDate |
2021 |
dc.date.none.fl_str_mv |
2021-08-30T10:54:52Z 2021-07-02 2021-07-02T00:00:00Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10400.22/18260 |
url |
http://hdl.handle.net/10400.22/18260 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
10.1109/IWCMC51323.2021.9498726 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799131468279054336 |