What a nice gesture: supporting the communication of people with aphasia for the in-bed scenario

• Autor(a) principal: Nunes, Fábio Ribeiro
• Data de Publicação: 2022
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10773/36521
• topic: Remote communication; Aphasia; Accessibility; Gestures; In-bed scenario; Smart environments

Communication is an essential part of life and, when ffected, as it happens to people with Aphasia, it severely impacts their quality of life. Augmentative and Alternative Communication (AAC) approaches aim to aid people with their communication disabilities. However, not all parts of the day are easily covered by these solutions, e.g., when lying in bed. In this dissertation, we aim at developing a solution that allows for two-way remote communication between a person with Aphasia (PWA) and other people (e.g., caregiver), while the former is lying in bed, alone. This work was carried out in the scope of the APH-ALARM project and builds on previous works within the same project. To address this goal, we collaborated with domain experts to obtain relevant information about the needs and motivations of a PWA and to evaluate our initial concept proposal. These results translated into a refined proposal of the system, and can also be considered an asset in other works with aiding people with aphasia as the main focus. As part of the system, we implemented an assistant that mediates communication relying on Yes/No questions with the purpose of gathering information before sending a message to the caregiver. These questions are presented audiovisually to the PWA and answered through gestures. The gestures are recognized using a model that classifies gestures that can be executed in different postures and using either arm. Considering a dataset corresponding to 8 subjects, this model achieved an F1-score of 99% and 93% for the subject dependent and independent cases, respectively. The bedroom scenario can be seen as the first step of an ambitious vision of providing support in every division of a home, while also not being limited to only aphasics, showing potential to aid anyone with speech difficulties.