Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes

Detalhes bibliográficos
Autor(a) principal: Garcia, Pedro P.
Data de Publicação: 2023
Outros Autores: Santos, Telmo G., Machado, Miguel A., Mendes, Nuno
Tipo de documento: Artigo
Idioma: eng
Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
Texto Completo: http://hdl.handle.net/10362/147038
Resumo: project UIDB/EMS/00667/2020 (UNIDEMI)
id RCAP_1d34e036235336a6c1183d5952783975
oai_identifier_str oai:run.unl.pt:10362/147038
network_acronym_str RCAP
network_name_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository_id_str 7160
spelling Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processesvisual assembly task recognitionhuman–robot collaborative assemblyonline class detectiondeep learningAnalytical ChemistryInformation SystemsAtomic and Molecular Physics, and OpticsBiochemistryInstrumentationElectrical and Electronic Engineeringproject UIDB/EMS/00667/2020 (UNIDEMI)The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applicationsDEMI - Departamento de Engenharia Mecânica e IndustrialUNIDEMI - Unidade de Investigação e Desenvolvimento em Engenharia Mecânica e IndustrialRUNGarcia, Pedro P.Santos, Telmo G.Machado, Miguel A.Mendes, Nuno2023-01-05T22:17:48Z2023-01-032023-01-03T00:00:00Zinfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/article18application/pdfhttp://hdl.handle.net/10362/147038eng1424-8220PURE: 49789001https://doi.org/10.3390/s23010553info:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2024-03-11T05:28:02Zoai:run.unl.pt:10362/147038Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T03:52:45.693757Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse
dc.title.none.fl_str_mv Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
spellingShingle Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
Garcia, Pedro P.
visual assembly task recognition
human–robot collaborative assembly
online class detection
deep learning
Analytical Chemistry
Information Systems
Atomic and Molecular Physics, and Optics
Biochemistry
Instrumentation
Electrical and Electronic Engineering
title_short Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_full Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_fullStr Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_full_unstemmed Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_sort Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
author Garcia, Pedro P.
author_facet Garcia, Pedro P.
Santos, Telmo G.
Machado, Miguel A.
Mendes, Nuno
author_role author
author2 Santos, Telmo G.
Machado, Miguel A.
Mendes, Nuno
author2_role author
author
author
dc.contributor.none.fl_str_mv DEMI - Departamento de Engenharia Mecânica e Industrial
UNIDEMI - Unidade de Investigação e Desenvolvimento em Engenharia Mecânica e Industrial
RUN
dc.contributor.author.fl_str_mv Garcia, Pedro P.
Santos, Telmo G.
Machado, Miguel A.
Mendes, Nuno
dc.subject.por.fl_str_mv visual assembly task recognition
human–robot collaborative assembly
online class detection
deep learning
Analytical Chemistry
Information Systems
Atomic and Molecular Physics, and Optics
Biochemistry
Instrumentation
Electrical and Electronic Engineering
topic visual assembly task recognition
human–robot collaborative assembly
online class detection
deep learning
Analytical Chemistry
Information Systems
Atomic and Molecular Physics, and Optics
Biochemistry
Instrumentation
Electrical and Electronic Engineering
description project UIDB/EMS/00667/2020 (UNIDEMI)
publishDate 2023
dc.date.none.fl_str_mv 2023-01-05T22:17:48Z
2023-01-03
2023-01-03T00:00:00Z
dc.type.status.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.driver.fl_str_mv info:eu-repo/semantics/article
format article
status_str publishedVersion
dc.identifier.uri.fl_str_mv http://hdl.handle.net/10362/147038
url http://hdl.handle.net/10362/147038
dc.language.iso.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv 1424-8220
PURE: 49789001
https://doi.org/10.3390/s23010553
dc.rights.driver.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 18
application/pdf
dc.source.none.fl_str_mv reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron:RCAAP
instname_str Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
instacron_str RCAAP
institution RCAAP
reponame_str Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
collection Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
repository.name.fl_str_mv Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação
repository.mail.fl_str_mv
_version_ 1799138119201587200

Registros relacionados