Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
Autor(a) principal: | |
---|---|
Data de Publicação: | 2023 |
Outros Autores: | , , |
Tipo de documento: | Artigo |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10362/147038 |
Resumo: | project UIDB/EMS/00667/2020 (UNIDEMI) |
id |
RCAP_1d34e036235336a6c1183d5952783975 |
---|---|
oai_identifier_str |
oai:run.unl.pt:10362/147038 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processesvisual assembly task recognitionhuman–robot collaborative assemblyonline class detectiondeep learningAnalytical ChemistryInformation SystemsAtomic and Molecular Physics, and OpticsBiochemistryInstrumentationElectrical and Electronic Engineeringproject UIDB/EMS/00667/2020 (UNIDEMI)The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applicationsDEMI - Departamento de Engenharia Mecânica e IndustrialUNIDEMI - Unidade de Investigação e Desenvolvimento em Engenharia Mecânica e IndustrialRUNGarcia, Pedro P.Santos, Telmo G.Machado, Miguel A.Mendes, Nuno2023-01-05T22:17:48Z2023-01-032023-01-03T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/article18application/pdfhttp://hdl.handle.net/10362/147038eng1424-8220PURE: 49789001https://doi.org/10.3390/s23010553info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-03-11T05:28:02Zoai:run.unl.pt:10362/147038Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T03:52:45.693757Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
spellingShingle |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes Garcia, Pedro P. visual assembly task recognition human–robot collaborative assembly online class detection deep learning Analytical Chemistry Information Systems Atomic and Molecular Physics, and Optics Biochemistry Instrumentation Electrical and Electronic Engineering |
title_short |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_full |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_fullStr |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_full_unstemmed |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_sort |
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
author |
Garcia, Pedro P. |
author_facet |
Garcia, Pedro P. Santos, Telmo G. Machado, Miguel A. Mendes, Nuno |
author_role |
author |
author2 |
Santos, Telmo G. Machado, Miguel A. Mendes, Nuno |
author2_role |
author author author |
dc.contributor.none.fl_str_mv |
DEMI - Departamento de Engenharia Mecânica e Industrial UNIDEMI - Unidade de Investigação e Desenvolvimento em Engenharia Mecânica e Industrial RUN |
dc.contributor.author.fl_str_mv |
Garcia, Pedro P. Santos, Telmo G. Machado, Miguel A. Mendes, Nuno |
dc.subject.por.fl_str_mv |
visual assembly task recognition human–robot collaborative assembly online class detection deep learning Analytical Chemistry Information Systems Atomic and Molecular Physics, and Optics Biochemistry Instrumentation Electrical and Electronic Engineering |
topic |
visual assembly task recognition human–robot collaborative assembly online class detection deep learning Analytical Chemistry Information Systems Atomic and Molecular Physics, and Optics Biochemistry Instrumentation Electrical and Electronic Engineering |
description |
project UIDB/EMS/00667/2020 (UNIDEMI) |
publishDate |
2023 |
dc.date.none.fl_str_mv |
2023-01-05T22:17:48Z 2023-01-03 2023-01-03T00:00:00Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/article |
format |
article |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10362/147038 |
url |
http://hdl.handle.net/10362/147038 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
1424-8220 PURE: 49789001 https://doi.org/10.3390/s23010553 |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
18 application/pdf |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799138119201587200 |