Rastreamento e reconhecimento de movimentos de punho na execução de excertos musicais ao piano: uma abordagem com MD-DTW (Multi-Dimensional Dynamic Time Warping)

• Autor(a) principal: Carvalho, Thyago Peres
• Data de Publicação: 2015
• Tipo de documento: Dissertação
• Idioma: por
• Título da fonte: Repositório Institucional da UFG
• dARK ID: ark:/38995/001300000f441
• Texto Completo: http://repositorio.bc.ufg.br/tede/handle/tede/5234
• topic: Interpretação musical; Movimentação do punho; Ensino de piano; Tutor; Aprendiz; Rastreamento; DTW; MD-DTW; Musical performance; Fist movement; Piano teaching; Tutor; Learner; Tracking; DTW; MD-DTW; CIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO

This paper proposes a method to support the teaching-learning fist gestures in the piano music performance, using tracking and recognition of fist movements in executions of musical piano excerpts. For this, a system was built by means of computer vision techniques, aiming to present to the student videos produced to verify the execution of the exercise by the learner and aims to provide data related to performance. The system also uses the same computer vision techniques for the generation of the proposed exercises to class by the tutor in order to support the production of educational material as well. To recognize gestures, the system uses a regular low cost webcam, and from a colored marker on the back of the musician’s hand, the wrist movements are detected and tracked. A multidimensional dynamic time warping algorithm (MD-DTW) was used in order to develop this tool, which is an n-dimensional version of Dynamic Time Warping (DTW). In the work sequence, three rounds of experiments were performed, being the first of which to adjust the system parameters from video excerpts performed by an expert trainer. The second and third step assessed, respectively, the learning gain of piano students to the proposed method and system usability. The experiments were performed on volunteers with musical reading skills, however, without requiring minimum technical domain while playing the piano. The results of these tests showed that in addition to the method being able to detect and recognize successful gestures, the volunteers presented learning gain within middle range, which shows that this is a very promising method. In addition, usability testing revealed that the implemented interface, is well suited and has reached good satisfaction results among the volunteers. As a result, it can be said that the method and the proposed prototype demonstrate the potential of these tools in transferring techniques, such as musical performance gestures in a piano teachinglearning environment.