Classification of table tennis strokes using a wearable device and deep learning

• Autor(a) principal: Ferreira, Nuno Micael
• Data de Publicação: 2022
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10284/10740
• topic: Domínio/Área Científica::Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática

The analysis of sports using everyday mobile devices is an area that has been increasingly explored aiming to help the user to improve in all aspects of the sport. The objective of the work proposed for this dissertation is to developed application capable of detecting strokes in table tennis using the iPhone and the Apple Watch, in which a recorded table tennis strokes data set performed by several table tennis athletes was created to help develop the application. Since the Artificial Intelegence area is increasingly present in our daily lives, the motivation in this work is to have a first contact with the current state of AI, the technologies available and most used in today’s present, and as within the company, it was intended to begin research in this area, mainly using Apple devices, it was decided to try and create a mobile application capable of detecting strokes performed in table tennis that would work with devices capable of AI processing, in order to provide statistical data to help table tennis athletes and coaches, which can later be sell for. After a study of devices available on the apple market with the necessary capabilities for the purpose of the work, it was concluded that for this work, the devices to be used would be the iPhone (above the X model) and the Apple Watch (above the model 5). Also because there were no public table tennis data set available, a methodology was developed with the objective of capturing table tennis strokes trough motion data. The recording of motion data was done by using an application capable of recording sensors data using the apple watch who was used by each athlete on the wrist. The sensors used to record motion data were accelerometer and gyroscope, and the capture methodology was planned and overseen by coaches and athletes. From the methodology created, 2 base data sets were created. One consisting of a short interval between strokes and the second and last with a bigger interval between strokes. From these 2 data sets, 3 more were created with different pre processing configurations applied followed by a filtering and reformatting of data to the necessary format for the creation of a Deep Learning model. To generate a DL classifier model, two approaches were tested, one by using Create ML, and the other by using Convolution Neural Network-Long Short Term Memory and Convolution Neural Network-Long Short Term Memory architecture. To evaluate the models, statistics generated from training were saved during model testing and creation. Create ML data set classifier models showed average performance except in one data set, with the generated classifier model having a maximum performance of 89.66% F1 score while CNN-LSTM and ConvLSTM approach generated good performance from all data set generated classifier models with the best classifier being the ConvLSTM with a 97.33% F1 score. After the creation of this same model, development of the application was performed consisting of two parts, one on the iPhone where it is possible to see the statistics and another on the Apple Watch where the ML model is executed and the stroke performed is detected being then sent to the application on the iPhone. The final step consisted on evaluation of the application during a live game scenario followed by an user rating application feedback questionnaire on athletes and coaches. Final application feedback was positive across all subjects with recommendations to the application interface and improvements to the classifier model. The live game application scenario with the generated classifier model obtained a 80% correct labelled strokes.