Autonomous trigger-task-sequencing system for Human- Robot Collaborative assembly: A Deep Learning approach on Visual Task Recognition

• Autor(a) principal: Garcia, Pedro Miguel Rodrigues Pinto
• Data de Publicação: 2021
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10362/138621
• topic: Visual assembly task recognition; Human-Robot Collaborative assembly; Online Object Detection; YOLO; Faster R-CNN; Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Mecânica

Human-robot collaborative (HRC) assembly environments are becoming increasingly important as the paradigm of manufacturing is shifting from mass production towards mass customization, and the introduction of a HRC system could significantly improve the flexibility and intelligence of automation. To efficiently accomplish tasks in HRC assembly environments, a robot needs to be able to understand its surroundings by detecting, recognizing, and locating the presence of humans and targeted objects, as well as to have a higher level of understanding beyond what is being currently seen, i.e., to fully understand a sequence of operations and tasks related to each step of the assembly process. Within the scope of the Internet of Things (IoT) and Artificial Intelligence (AI), the most widely used framework to enable robot vision (or vision in other type of cyber-physical systems) is deep learning, precisely by means of deep learning algorithms based on Convolutional Neural Networks (CNNs). Hence, in this study a system capable of managing a HRC assembly process of a mechanical component was developed, where computer vision was enabled by CNN-based methods and the assembly task recognition made from solely visual RGB data of the components in working space. A model for components’ detection was firstly developed by comparing two main CNN-based framework branches – the Faster Region-based Convolutional Neural Network (Faster R-CNN) and the You Only Look Once (YOLO). Then, a system that correlated the current state of the working space – i.e., whether certain components were already correctly assembled – with the progression of the assembly sequence was developed with the best-performing 11 class object detector, the YOLOv3. This framework was the only capable of detecting a small object classes in comparison with the other benchmarked frameworks and presented a mean average precision (mAP) of 72.26% over the test dataset. Using YOLOv3 as the computer vision-enabling framework, a successful and efficient HRC industrial demonstrator was created.