Robotic soccer: behaviors for simulated humanoid robots

• Autor(a) principal: Resende, Francisco Manuel Gonçalves de
• Data de Publicação: 2020
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10773/30084
• topic: Artificial intelligence; Machine learning; Reinforcement learning; Deep reinforcement learning; Robotic soccer; RoboCup; Humanoid robots

This dissertation was focused on the improvement of the goalkeeper of the FC Portugal 3D team, participant in the RoboCup 3D Soccer Simulation league. All the experiments during this work were undertaken within the league environment. This competition consists on a virtual soccer match between simulated humanoid robots. A soccer goalkeeper has as its primary mission keeping his goal clear. To accomplish that goal, he needs to save every shot targeted to its goal and that can be a very difficult challenge. The challenge of saving a shot can be divided into two different tasks: predicting the ball movement and intersecting the ball trajectory. During this work, the primary objective is the application of machine learning and reinforcement learning techniques to solve these problems. Predicting the ball movement is a matter of replicating the environment physics. So, a ball position estimator was developed following a supervised machine learning approach. Although this estimator was initially developed thinking mainly on the goalkeeper, it can also be very useful for the field players. A precise ball estimator opens the door for behaviors capable of interacting with the ball while it is moving, which is rare at the current state of the league. Intersecting the ball trajectory and saving the shots was accomplished by a generic behavior developed with a Deep Reinforcement Learning approach. This generic behavior is capable of saving different types of shots: shots coming from different positions of the field and different 3D initial velocities. The new behavior was tested and evaluated in comparison with the previous version of the goalkeeper and achieved indisputable better results. This was a huge achievement because the previous version of the goalkeeper was composed by thirteen different behaviors and a selection algorithm and it was outperformed and replaced by only one behavior.