The Barcelona Supercomputing Center (BSC) and SUEZ in Spain have signed a collaboration agreement, whereby the BSC will provide knowledge in computing technologies, high-performance algorithms, artificial intelligence, visualization and big data, and SUEZ will contribute its expertise in the operation of drinking water distribution networks for the testing, development, validation and application of new management tools.
Within the framework of this agreement, the feasibility of reinforced learning techniques will be explored. Deep reinforcement learning (or Deep RL) is a novel artificial intelligence technique that uses neural networks and reinforced learning techniques to train software agents for autonomous decision making.
From this premise, the project has defined a series of challenges that will allow the evaluation and identification of value in the use of Deep RL techniques for the optimization of hydraulic networks. The application of these technologies in the field of water management still has a high research component, so the objective of the project is to initially validate that agents are capable of assigning management instructions for a network that equal or exceed performance achieved by operators.
In addition, the feasibility of reinforced learning techniques with deep learning will be explored to apply it in the mitigation of emergencies and the intelligent control in real time of the water transport network.
For the current project with SUEZ in Spain, classic Deep Q-Learning techniques will also be evaluated, in which a neural network is used to predict the impact of an action on a certain network state, as well as Actor-Critic techniques. These techniques are aimed at predicting the best stock policy and not the specific value of a specific stock.
"The BSC is interested in developing artificial intelligence algorithms that are capable of automatic piloting, so to speak, of industrial machinery using strategies similar to the lines that Google's DeepMind uses to beat Go or chess or video games", says Josep Martorell, associate director of Barcelona Supercomputing Center.
"At SUEZ Spain we are committed to innovation and digital transformation, collaborating with centers of excellence such as BSC, for the development of new tools that allow us to continue increasing our operating efficiency by developing new knowledge and capabilities", says Manuel Cermerón (CEO of SUEZ Spain).
In this project, the technical team will be made up of artificial intelligence researchers from the Barcelona Supercomputing Center and researchers from Cetaqua, the water technology center attached to SUEZ Spain, and concept tests will be carried out in the metropolitan area of Barcelona with Aigües de Barcelona.
The ultimate objective of the project is not only to develop the solutions but also to validate them once they have been applied to considerable network sizes (city scale, for example), which allows their operation in production and that the reinforced learning algorithms can be retrained and replicated in networks with different topology and under different hydraulic operating conditions.