Portuguese sign language recognition via computer vision and depth sensor

• Autor(a) principal: Almeida, Rui Nuno de
• Data de Publicação: 2011
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10071/8231
• topic: Sign language recognition; Computer vision; Depth sensor; Machine learning; Template matching; Automatic gesture recognition; Hidden Markov models; Reconhecimento de língua gestual; Visão por computador; Sensores de profundidade; Aprendizagem automática; Correspondência de padrões; Reconhecimento automático de gestos; Modelos ocultos de Markov; Reconocimiento del lenguaje de signos; Visión por computadora; Sensores de profundidad; Aprendizaje automático; Correspondencia de plantillas; Reconocimiento automático de gestos; Modelos ocultos de Markov

Sign languages are used worldwide by a multitude of individuals. They are mostly used by the deaf communities and their teachers, or people associated with them by ties of friendship or family. Speakers are a minority of citizens, often segregated, and over the years not much attention has been given to this form of communication, even by the scientific community. In fact, in Computer Science there is some, but limited, research and development in this area. In the particular case of sign Portuguese Sign Language-PSL that fact is more evident and, to our knowledge there isn’t yet an efficient system to perform the automatic recognition of PSL signs. With the advent and wide spreading of devices such as depth sensors, there are new possibilities to address this problem. In this thesis, we have specified, developed, tested and preliminary evaluated, solutions that we think will bring valuable contributions to the problem of Automatic Gesture Recognition, applied to Sign Languages, such as the case of Portuguese Sign Language. In the context of this work, Computer Vision techniques were adapted to the case of Depth Sensors. A proper gesture taxonomy for this problem was proposed, and techniques for feature extraction, representation, storing and classification were presented. Two novel algorithms to solve the problem of real-time recognition of isolated static poses were specified, developed, tested and evaluated. Two other algorithms for isolated dynamic movements for gesture recognition (one of them novel), have been also specified, developed, tested and evaluated. Analyzed results compare well with the literature.