Assessing the emotional impact of video using machine learning techniques
Autor(a) principal: | |
---|---|
Data de Publicação: | 2020 |
Tipo de documento: | Dissertação |
Idioma: | eng |
Título da fonte: | Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
Texto Completo: | http://hdl.handle.net/10071/21975 |
Resumo: | Typically, when a human being watches a video, different sensations and mind states can be stimulated. Among these, the sensation of fear can be triggered by watching segments of movies containing themes such as violence, horror and suspense. Both the audio and visual stimuli may contribute to induce fear onto the viewer. This dissertation studies the use of machine learning for forecasting the emotional effects triggered by video, more precisely, the automatic identification of fear inducing video segments. Using the LIRIS-ACCEDE dataset, several experiments have been performed in order to identify feature sets that are most relevant to the problem and to assess the performance of different machine learning classifiers. Both classical and deep learning techniques have been implemented and evaluated, using the Scikit-learn and TensorFlow machine learning libraries. Two different approaches for training and testing have been followed: film-level dataset splitting, where different films were used for training and testing; and sample-level dataset splitting, which allowed that different samples coming from the same films were used for training and testing. The prediction of movie segments that trigger fear sensations achieved a F1-score of 18.5% in the first approach, a value suggesting that the dataset does not adequately represent the universe of movies. The second approach achieved a F1-score of about 84.0%, a substantially higher value that shows promising outcomes when performing the proposed task. |
id |
RCAP_883e5ed69651df0a3e1a2972fdf311d0 |
---|---|
oai_identifier_str |
oai:repositorio.iscte-iul.pt:10071/21975 |
network_acronym_str |
RCAP |
network_name_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository_id_str |
7160 |
spelling |
Assessing the emotional impact of video using machine learning techniquesMachine learningEmotional predictionFear predictionVideo classificationAprendizagem automáticaPredição emocionalPredição de medoClassificação de vídeoTypically, when a human being watches a video, different sensations and mind states can be stimulated. Among these, the sensation of fear can be triggered by watching segments of movies containing themes such as violence, horror and suspense. Both the audio and visual stimuli may contribute to induce fear onto the viewer. This dissertation studies the use of machine learning for forecasting the emotional effects triggered by video, more precisely, the automatic identification of fear inducing video segments. Using the LIRIS-ACCEDE dataset, several experiments have been performed in order to identify feature sets that are most relevant to the problem and to assess the performance of different machine learning classifiers. Both classical and deep learning techniques have been implemented and evaluated, using the Scikit-learn and TensorFlow machine learning libraries. Two different approaches for training and testing have been followed: film-level dataset splitting, where different films were used for training and testing; and sample-level dataset splitting, which allowed that different samples coming from the same films were used for training and testing. The prediction of movie segments that trigger fear sensations achieved a F1-score of 18.5% in the first approach, a value suggesting that the dataset does not adequately represent the universe of movies. The second approach achieved a F1-score of about 84.0%, a substantially higher value that shows promising outcomes when performing the proposed task.Quando o ser humano assiste a filmes, diferentes sensações e estados de espírito são despoletados. Entre estes encontra-se o medo, que pode ser despoletado através da visualização de excertos de filmes contendo, por exemplo, violência gráfica, horror ou suspense. Tanto a componente visual como a auditiva contribuem para o despoletar desta sensação. Nesta dissertação é analisada a utilização de aprendizagem automática para prever o impacto emocional que a visualização de vídeos possa causar nas pessoas, mais concretamente os segmentos de um filme que despoletam a sensação de medo. Foram realizadas diversas experiências usando o conjunto de dados LIRIS-ACCEDE com os objetivos de encontrar conjuntos de atributos de imagem e áudio com maior relevância para o problema e de avaliar o desempenho de diversos modelos de aprendizagem automática usados para classificação. Foram usados diversos algoritmos clássicos e de aprendizagem profunda, recorrendo-se às bibliotecas Scikit-learn e TensorFlow. No que se refere à separação dos dados usados para treino e teste foram seguidas duas abordagens: divisão dos dados ao nível do filme, sendo usados filmes distintos para treino e teste; e divisão dos dados ao nível da amostra, possibilitando que os conjuntos de treino e teste contenham amostras distintas, mas pertencentes aos mesmos filmes. Para previsão dos segmentos que despoletam medo, na primeira abordagem chegou-se a um resultado de F1-score de 18,5%, concluindo-se que o conjunto de dados usado não é representativo, e na segunda abordagem a um F1-score de 84,0%, um valor substancialmente mais alto e promissor no desempenho da tarefa proposta.2021-02-09T16:07:07Z2020-12-22T00:00:00Z2020-12-222020-12info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/masterThesisapplication/pdfhttp://hdl.handle.net/10071/21975TID:202626415engMaia, André Filipe Lopesinfo:eu-repo/semantics/openAccessreponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãoinstacron:RCAAP2023-11-09T17:59:41Zoai:repositorio.iscte-iul.pt:10071/21975Portal AgregadorONGhttps://www.rcaap.pt/oai/openaireopendoar:71602024-03-19T22:31:22.562071Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informaçãofalse |
dc.title.none.fl_str_mv |
Assessing the emotional impact of video using machine learning techniques |
title |
Assessing the emotional impact of video using machine learning techniques |
spellingShingle |
Assessing the emotional impact of video using machine learning techniques Maia, André Filipe Lopes Machine learning Emotional prediction Fear prediction Video classification Aprendizagem automática Predição emocional Predição de medo Classificação de vídeo |
title_short |
Assessing the emotional impact of video using machine learning techniques |
title_full |
Assessing the emotional impact of video using machine learning techniques |
title_fullStr |
Assessing the emotional impact of video using machine learning techniques |
title_full_unstemmed |
Assessing the emotional impact of video using machine learning techniques |
title_sort |
Assessing the emotional impact of video using machine learning techniques |
author |
Maia, André Filipe Lopes |
author_facet |
Maia, André Filipe Lopes |
author_role |
author |
dc.contributor.author.fl_str_mv |
Maia, André Filipe Lopes |
dc.subject.por.fl_str_mv |
Machine learning Emotional prediction Fear prediction Video classification Aprendizagem automática Predição emocional Predição de medo Classificação de vídeo |
topic |
Machine learning Emotional prediction Fear prediction Video classification Aprendizagem automática Predição emocional Predição de medo Classificação de vídeo |
description |
Typically, when a human being watches a video, different sensations and mind states can be stimulated. Among these, the sensation of fear can be triggered by watching segments of movies containing themes such as violence, horror and suspense. Both the audio and visual stimuli may contribute to induce fear onto the viewer. This dissertation studies the use of machine learning for forecasting the emotional effects triggered by video, more precisely, the automatic identification of fear inducing video segments. Using the LIRIS-ACCEDE dataset, several experiments have been performed in order to identify feature sets that are most relevant to the problem and to assess the performance of different machine learning classifiers. Both classical and deep learning techniques have been implemented and evaluated, using the Scikit-learn and TensorFlow machine learning libraries. Two different approaches for training and testing have been followed: film-level dataset splitting, where different films were used for training and testing; and sample-level dataset splitting, which allowed that different samples coming from the same films were used for training and testing. The prediction of movie segments that trigger fear sensations achieved a F1-score of 18.5% in the first approach, a value suggesting that the dataset does not adequately represent the universe of movies. The second approach achieved a F1-score of about 84.0%, a substantially higher value that shows promising outcomes when performing the proposed task. |
publishDate |
2020 |
dc.date.none.fl_str_mv |
2020-12-22T00:00:00Z 2020-12-22 2020-12 2021-02-09T16:07:07Z |
dc.type.status.fl_str_mv |
info:eu-repo/semantics/publishedVersion |
dc.type.driver.fl_str_mv |
info:eu-repo/semantics/masterThesis |
format |
masterThesis |
status_str |
publishedVersion |
dc.identifier.uri.fl_str_mv |
http://hdl.handle.net/10071/21975 TID:202626415 |
url |
http://hdl.handle.net/10071/21975 |
identifier_str_mv |
TID:202626415 |
dc.language.iso.fl_str_mv |
eng |
language |
eng |
dc.rights.driver.fl_str_mv |
info:eu-repo/semantics/openAccess |
eu_rights_str_mv |
openAccess |
dc.format.none.fl_str_mv |
application/pdf |
dc.source.none.fl_str_mv |
reponame:Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) instname:Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação instacron:RCAAP |
instname_str |
Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
instacron_str |
RCAAP |
institution |
RCAAP |
reponame_str |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
collection |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) |
repository.name.fl_str_mv |
Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos) - Agência para a Sociedade do Conhecimento (UMIC) - FCT - Sociedade da Informação |
repository.mail.fl_str_mv |
|
_version_ |
1799134875529248768 |