Reinforcement learning applied to vessel navigation in fast-time simulations.

• Autor(a) principal: Andrade, José Amendola Netto
• Data de Publicação: 2020
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Biblioteca Digital de Teses e Dissertações da USP
• Texto Completo: https://www.teses.usp.br/teses/disponiveis/3/3152/tde-04052021-085708/
• topic: Aprendizado computacional; Fast-time simulations; Navegação em águas restritas; Navigation in restricted waters; Navios; Portos; Reinforcement learning; Simulação

Fast-time simulations have been proven to be an essential tool for maritime engineering, not only in ship design but also by detecting critical situations and bottlenecks in projects of ports. However, such simulations are not performed by professional pilots and might become a complex task with results not so close to reality. Such issues can present an opportunity for introducing Reinforcement Learning methods in the maritime domain. This work proposes a Reinforcement Learning based solution which is able to automatically generate vessel trajectories in restricted waters under the effect of environment forces. The agent learns by interacting with the simulator and receiving reward signals. It also gives discrete commands in spaced time steps in order to emulate limitations of human piloting. The method evaluates the distributed version of two state-of-art Reinforcement Learning algorithms. It handles channel segments as separate episodes and includes curvature information for anticipating actions. Experiments were run considering realistic scenarios with narrow curved channels where wind and current incidence varies along the trajectory. The novelty of the work is the fact that the solution proposed requires no prior knowledge on dynamic models or predefined line paths to be followed by the ship. It may impact in fast-time simulations by requiring less human effort in trajectories generation. The method adopted keeps a simple representation and can be applied to any port channel configuration that respects local technical regulations.