Real-time Human Action Localization in the Wild

• Autor(a) principal: Pereira, João Alexandre Cardeira
• Data de Publicação: 2023
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10362/163514
• topic: Spatiotemporal Action Localization; Online Action Localization; Frame Selection; Autoencoders; Background Subtraction; Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática

Action Localization has been emerging as an important field of research, aiming to temporally and spatially distinguish human actions from video data. Due to the large amounts of video content available, deep convolutional networks have become the most commonly used models in Action Localization research. Usual applications for Action Localization algorithms (e.g. camera surveillance) tend to offer sub-optimal conditions for deploying these complex models, often having limited computer resources and requiring fast inference speed for online use. Therefore, research has been evolving towards developing faster and more efficient solutions. Parallel to this, the field of Frame Selection has been showing promising results, improving the perfor- mance and efficiency of action recognition models. Therefore, this thesis hypothesises that these techniques could be applied to Action Localization models to achieve the goal of efficient Spatiotemporal Action Localization. We conduct a study on the impact of reducing the size of the input for the YOWO [38] model on the UCF101-24 dataset [62]. This study leads to the definition of a function that selects one of two input variants based on the background complexity of the input clip through an extremely lightweight and fast background subtraction method. We improve the model’s inference speed by 73% while maintaining good performance, surpassing the baselines defined and approximating the reference YOWO model. Influenced by the work in Wang et al. [73], we also introduce a method that calculates the individual scores of each clip frame using a lightweight convolutional network and selects the top-scoring frames for the input. This method has also reduced the inference time of the model, achieving an improvement of 58%. Finally, we propose an Autoencoder-based solution that outputs a representation of the original clip using fewer frames without disregarding any frame of the clip.