Extracting relevant information regarding customer behaviour from surveillance videos
Autor(a) principal: | |
---|---|
Data de Publicação: | 2023 |
Tipo de documento: | Dissertação |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10071/29911 |
Resumo: | In the modern retail environment, leveraging technologies such as high-resolution video surveillance and artificial intelligence to study in-store customer behaviour is a crucial factor in improving valuable business aspects including marketing, customer service, and security. In this context, this dissertation focuses on the development of a framework for extracting information regarding customer behaviour from high-resolution surveillance videos. This framework incorporates a series of steps, which include detecting and tracking each person, extracting trajectory points, estimating walking speeds, detecting groups, and recognising actions using pose (skeleton) data. Along with the framework, we propose two contributions designed to enhance its performance: occlusion-aware mechanism and trajectory smoothing method. The occlusion-aware mechanism was created as a means of mitigating the impact of partial occlusions on location data. The experimental results indicated statistically significant improvements resulting from its application. Furthermore, the smoothing method was introduced to attenuate oscillations in the trajectory points, considering both past and future path information. Combined with the occlusion-aware mechanism, it proved to be a valuable tool for improving trajectory mapping. Regarding action recognition, we compared three skeleton-based models (i.e. ST-GCN, AGCN, and PoseC3D) on subsets of the People in Public dataset featuring 12 shopping-related action classes. The results obtained from training and testing the models demonstrated their effectiveness in recognising customer behaviour (reaching accuracy values of around 90%) whilst ensuring privacy, and allowed us to deduce appropriate use cases for each of them. |
id |
RCAP_a207534e9e60be0819bb70a897168451 |
---|---|
oai_identifier_str |
oai:repositorio.iscte-iul.pt:10071/29911 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Extracting relevant information regarding customer behaviour from surveillance videosCustomer behaviourObject detectionMulti-object trackingTrajectory extractionPose estimationAction recognitionComportamento do clienteDeteção de objectosSeguimento de objectosExtração de trajectóriasEstimativa de poseReconhecimento de açõesIn the modern retail environment, leveraging technologies such as high-resolution video surveillance and artificial intelligence to study in-store customer behaviour is a crucial factor in improving valuable business aspects including marketing, customer service, and security. In this context, this dissertation focuses on the development of a framework for extracting information regarding customer behaviour from high-resolution surveillance videos. This framework incorporates a series of steps, which include detecting and tracking each person, extracting trajectory points, estimating walking speeds, detecting groups, and recognising actions using pose (skeleton) data. Along with the framework, we propose two contributions designed to enhance its performance: occlusion-aware mechanism and trajectory smoothing method. The occlusion-aware mechanism was created as a means of mitigating the impact of partial occlusions on location data. The experimental results indicated statistically significant improvements resulting from its application. Furthermore, the smoothing method was introduced to attenuate oscillations in the trajectory points, considering both past and future path information. Combined with the occlusion-aware mechanism, it proved to be a valuable tool for improving trajectory mapping. Regarding action recognition, we compared three skeleton-based models (i.e. ST-GCN, AGCN, and PoseC3D) on subsets of the People in Public dataset featuring 12 shopping-related action classes. The results obtained from training and testing the models demonstrated their effectiveness in recognising customer behaviour (reaching accuracy values of around 90%) whilst ensuring privacy, and allowed us to deduce appropriate use cases for each of them.No ambiente retalhista atual, o uso de tecnologias como videovigilância e inteligência artificial para o estudo do comportamento dos clientes é um fator essencial para melhorar aspectos como marketing, apoio ao cliente, e segurança. Neste contexto, a presente dissertação foca-se no desenvolvimento de um sistema para a extração de informações sobre o comportamento dos clientes a partir de vídeos de vigilância, o que envolve detetar e seguir pessoas, extrair pontos de trajetória, estimar velocidades de caminhada, detetar grupos, e reconhecer ações usando dados de pose (esqueleto). Em complemento ao sistema, propomos duas contribuições para melhorar o seu desempenho: o mecanismo de compensação de oclusões e um método de suavização de trajetórias. O mecanismo de compensação de oclusões foi criado para mitigar o impacto das oclusões nos dados de localização. Os resultados indicaram melhorias estatisticamente significativas derivadas da sua utilização. Além disso, o método de suavização foi introduzido para atenuar as oscilações nos pontos de trajetória, tendo em conta informação anterior e posterior. Aliado ao mecanismo de compensação de oclusões, provou ser uma ferramenta valiosa para melhorar o mapeamento de trajectórias. Relativamente ao reconhecimento de acções, comparámos três modelos baseados em esqueleto (ST-GCN, AGCN e PoseC3D) em subconjuntos do conjunto de dados People in Public com 12 classes de ação. Os resultados obtidos com o treino e teste dos modelos revelaram a sua eficácia em reconhecer ações típicas de clientes (atingindo taxas de acerto na ordem dos 90%), e permitiram-nos inferir casos de utilização adequados para cada um deles.2023-12-05T14:44:47Z2023-11-24T00:00:00Z2023-11-242023-10info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisapplication/pdfhttp://hdl.handle.net/10071/29911TID:203406460engCorreia, Simão de São José Gregório de Oliveira Frazãoinfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-12-10T01:18:08Zoai:repositorio.iscte-iul.pt:10071/29911Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-20T00:41:50.696375Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Extracting relevant information regarding customer behaviour from surveillance videos |
title |
Extracting relevant information regarding customer behaviour from surveillance videos |
spellingShingle |
Extracting relevant information regarding customer behaviour from surveillance videos Correia, Simão de São José Gregório de Oliveira Frazão Customer behaviour Object detection Multi-object tracking Trajectory extraction Pose estimation Action recognition Comportamento do cliente Deteção de objectos Seguimento de objectos Extração de trajectórias Estimativa de pose Reconhecimento de ações |
title_short |
Extracting relevant information regarding customer behaviour from surveillance videos |
title_full |
Extracting relevant information regarding customer behaviour from surveillance videos |
title_fullStr |
Extracting relevant information regarding customer behaviour from surveillance videos |
title_full_unstemmed |
Extracting relevant information regarding customer behaviour from surveillance videos |
title_sort |
Extracting relevant information regarding customer behaviour from surveillance videos |
author |
Correia, Simão de São José Gregório de Oliveira Frazão |
author_facet |
Correia, Simão de São José Gregório de Oliveira Frazão |
author_role |
author |
dc.contributor.author.fl_str_mv |
Correia, Simão de São José Gregório de Oliveira Frazão |
dc.subject.por.fl_str_mv |
Customer behaviour Object detection Multi-object tracking Trajectory extraction Pose estimation Action recognition Comportamento do cliente Deteção de objectos Seguimento de objectos Extração de trajectórias Estimativa de pose Reconhecimento de ações |
topic |
Customer behaviour Object detection Multi-object tracking Trajectory extraction Pose estimation Action recognition Comportamento do cliente Deteção de objectos Seguimento de objectos Extração de trajectórias Estimativa de pose Reconhecimento de ações |
description |
In the modern retail environment, leveraging technologies such as high-resolution video surveillance and artificial intelligence to study in-store customer behaviour is a crucial factor in improving valuable business aspects including marketing, customer service, and security. In this context, this dissertation focuses on the development of a framework for extracting information regarding customer behaviour from high-resolution surveillance videos. This framework incorporates a series of steps, which include detecting and tracking each person, extracting trajectory points, estimating walking speeds, detecting groups, and recognising actions using pose (skeleton) data. Along with the framework, we propose two contributions designed to enhance its performance: occlusion-aware mechanism and trajectory smoothing method. The occlusion-aware mechanism was created as a means of mitigating the impact of partial occlusions on location data. The experimental results indicated statistically significant improvements resulting from its application. Furthermore, the smoothing method was introduced to attenuate oscillations in the trajectory points, considering both past and future path information. Combined with the occlusion-aware mechanism, it proved to be a valuable tool for improving trajectory mapping. Regarding action recognition, we compared three skeleton-based models (i.e. ST-GCN, AGCN, and PoseC3D) on subsets of the People in Public dataset featuring 12 shopping-related action classes. The results obtained from training and testing the models demonstrated their effectiveness in recognising customer behaviour (reaching accuracy values of around 90%) whilst ensuring privacy, and allowed us to deduce appropriate use cases for each of them. |
publishDate |
2023 |
dc.date.none.fl_str_mv |
2023-12-05T14:44:47Z 2023-11-24T00:00:00Z 2023-11-24 2023-10 |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10071/29911 TID:203406460 |
url |
http://hdl.handle.net/10071/29911 |
identifier_str_mv |
TID:203406460 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799136320804618240 |