A multi-modal approach for affective data gathering

• Autor(a) principal: Oliveira, Daniel Barbosa de
• Data de Publicação: 2019
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10773/29400
• topic: Affective computing; Multimodal system; Stress

Recognizing, interpreting and processing emotions (Affective Computing) is an emerging field of computer science. Multiple methods of data acquisition and emotion classification exist with different accuracy performances. Despite this, multimodal systems, generally have a higher accuracy than unimodal ones. This dissertation’s goal is to research the current methods of both affective data gathering and emotion classification while developing a multi-modal system, that focuses primarily on the utilisation of non-intrusive methods with potential application in cccupational stress. The system has the purpose of collecting affective data including multiple data gathering methods such as mouse and keyboard utilisation data, ECG data, face and upper body video recordings and computer screen video recordings (for activity detection). For the emotion classification, the Clustering and Random Forest algorithms were utilised. In the exploratory study with the already existent SWELL investigation dataset, we tested the algorithm of Random Forest and an overall accuracy of 89.97% was achieved, which we considered acceptable. In order to validate the final system, a study with eleven participants was conducted. An overall error rate of approximately 65% was achieved with the Random Forest algorithm. For the majority of the participants, the Clustering algorithm did not recognize most of the data above 3% in class 2. The participants also reported in the questionnaires an overall decrease in the stress felt. Therefore, it is possible that the proposed protocol did not induce the desired emotional state (stress) in the participants. The developed multimodal system is functional and can be utilised in other studies with emotional markings gathering.