Development and generalization of a reinforcement learning model for the pump scheduling problem

• Autor(a) principal: Borges, Guilherme Simões
• Data de Publicação: 2023
• Tipo de documento: Dissertação
• Idioma: eng
• Título da fonte: Repositório Científico de Acesso Aberto de Portugal (Repositórios Cientìficos)
• Texto Completo: http://hdl.handle.net/10773/40879
• topic: WSS; DRL; DQN; Pump scheduling problem; EPANET; Optimization

Optimization techniques are used as efficient strategies for the operation of Water Supply Systems (WSS). However, with the increase in complexity of the WSS’s network, the computational resources required by optimization methods also increases, potentially compromising optimal operation in real-time. This work presents a Reinforcement Learning (RL) model as an alternative to traditional optimization techniques for solving the Pump Scheduling Problem (PSP). The model is made of a Deep Q-Learning Network (DQN) agent and an environment where EPANET 2.0 simulates the hydraulic behaviour of water networks. This study aims at analysing the RL agent’s capability in arriving to a near optimal solution in the PSP, testing several energy tariffs and consumption demand patterns. While the computational cost of training the agent is significantly higher than that of finding optimal strategies using traditional optimisation algorithms, once trained the model is capable of providing near-optimal solutions almost immediately. A state features analysis is done where different configurations of both state features and rewards are tested. The results of the RL model are satisfactory by giving information of both demand and tariff patterns, while limiting the number of pump starts for the time window. A generalization analysis is made as well. The RL model achieved an average accuracy of 94% when trained with multiple tariff and consumption patterns for 60k seconds, compared to the results obtained using a non-linear programming optimizer. Additionally, the RL model used only 1% of the CPU time after training.