Fusion object detection and action recognition to predict violent action

• Autor(a) principal: Rodrigues, Nelson Ricardo Pereira
• Data de Publicação: 2023
• Outros Autores: Costa, Nuno Miguel Cerqueira, Melo, César Gonçalo Macedo, Abbasi, Ali, Fonseca, Jaime C., Cardoso, Paulo, Borges, João
• Tipo de documento: Artigo
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: https://hdl.handle.net/1822/85724
• topic: Machine learning; Visual intelligence; Object detection; Image processing; Action recognition; Autonomous vehicles

In the context of Shared Autonomous Vehicles, the need to monitor the environment inside the car will be crucial. This article focuses on the application of deep learning algorithms to present a fusion monitoring solution which was three different algorithms: a violent action detection system, which recognizes violent behaviors between passengers, a violent object detection system, and a lost items detection system. Public datasets were used for object detection algorithms (COCO and TAO) to train state-of-the-art algorithms such as YOLOv5. For violent action detection, the MoLa InCar dataset was used to train on state-of-the-art algorithms such as I3D, R(2+1)D, SlowFast, TSN, and TSM. Finally, an embedded automotive solution was used to demonstrate that both methods are running in real-time.