Towards the long term implementation of real time control of combined sewer systems: a review of performance and influencing factors

Enhanced resilience in urban stormwater management through model predictive control and optimal layout schemes under extreme rainfall events

Optimizing the layout of urban stormwater management systems is an effective method for mitigating the risk of urban flooding under extreme storms. However, traditional approaches that consider only economic costs or annual runoff control rates cannot dynamically respond to the uncertainties of extreme weather, making it difficult to completely avoid large accumulations of water and flooding in a short period. This study proposes an integrated method combining system layout optimization and Model Predictive Control(MPC)to enhance the system's resilience and effectiveness in flood control. An optimization framework was initially built to identify optimal system layouts, balancing annual average life cycle cost (AALCC) and resilience index. The MPC was then applied to the optimal layout selected using the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method, aiming to alleviate inundation cost-effectively. The adaptability of MPC to varying sets of control horizons and its efficacy in managing the hydrograph and flood dynamics of urban drainage system were examined. Conducted in Yubei, Chongqing, this study revealed patterns in optimal layout fronts among various extreme design rainfalls, showing that peak position rate and return period significantly influence system resilience. The contribution of MPC to the optimal system layout was particularly notable, resulting in improved instantaneous and overall flood mitigation. The application of MPC increased the resilience index by an average of 0.0485 and offered cost savings of 0.0514 million yuan in AALCC. Besides, our findings highlighted the importance of selecting an optimal set of control horizons for MPC, which could reduce maximum flood depth from 0.43m to 0.19m and decrease conduit peak flow by up to 14% at a flood-prone downstream location.

A review of pollution-based real-time modelling and control for sewage systems

Conventional solutions for wastewater collection focus on reducing overflow events in the sewage network, which can be achieved by adapting sewer infrastructure or, a more cost-effective alternative, by implementing a non-engineering management solution. The state-of-the-art solution is centered on Real-Time Control (RTC), which is already resulting in a positive impact on the environment by decreasing the volume of wastewater being discharged into receiving waters. Researchers have been continuing efforts towards upgrading RTC solutions for sewage systems and a new approach, although rudimentary, was introduced in 1997, known as Pollution-based RTC (P-RTC), which added water quality (concentration or load) information explicitly within the RTC algorithm. Formally, P-RTC is encompassed of several control methodologies using a measurement or estimation of the concentration (i.e. COD or ammonia) of the sewage throughout the network. The use of P-RTC can result in a better control performance with a reduction in concentration of overflowing wastewater observed associated with an increase of concentration of sewage arriving at the Wastewater Treatment Plant (WWTP). The literature revealed that P-RTC can be differentiated by: (1) implementation method; (2) how water quality is incorporated, and (3) overall control objectives. Additionally, this paper evaluates the hydrological models used for P-RTC. The objective of this paper is to compile relevant research in pollution-based modelling and real-time control of sewage systems, explaining the general concepts within each P-RTC category and their differences.

Tunable linear feedback control of urban drainage systems using models defined purely from data

Real-time and model-predictive control promises to make urban drainage systems (UDS) adaptive, coordinated, and dynamically optimal. Though early implementations are promising, existing control algorithms have drawbacks in computational expense, trust, system-level coordination, and labor cost. Linear feedback control has distinct advantages in computational expense, interpretation, and coordination. However, current methods for building linear feedback controllers require calibrated software models. Here we present an automated method for generating tunable linear feedback controllers that require only system response data. The controller design consists of three main steps: (1) estimating the network connectivity using tools for causal inference, (2) identifying a linear, time-invariant (LTI) dynamical system which approximates the network, and (3) designing and tuning a feedback controller based on the LTI urban drainage system approximation. The flooding safety, erosion prevention, and water treatment performance of the method are evaluated across 190 design storms on a separated sewer model. Strong results suggest that the system knowledge required for generating effective, safe, and tunable controllers for UDS is surprisingly basic. This method allows near-turnkey synthesis of controllers solely from sensor data or reduction of process-based models.

Automated screening of control potential with spatially explicit results to support dialogue about sewer overflow reduction and beyond

For real-time control to become a standard measure for upgrading urban drainage systems, control potential screenings need to be easily integrated into the early planning processes that already take place. However, current screening methods are either not aligned with the present planning process, unrelatable for water managers or too time-consuming. We therefore developed an automated screening methodology through a co-design process with six Danish utilities. The process started out from a literature review, included interviews and workshops, and resulted in the control potential screening tool COPOTO. In the co-design process, utilities generally responded that indicators based solely on an assessment of static system attributes are insufficient. Thus, COPOTO instead post-processes the results of urban drainage simulation models that are commonly available. The decision context considered in initial planning phases was found to include environmental, economic, social and technical objectives that were highly case-dependent. When presenting CSO reduction potentials, the utilities therefore generally preferred interactive, spatially explicit visualisations that link the CSO reduction at a particular location to the storages and actuators that need to be activated. This enables water managers to discuss, for example, operational constraints of a considered control location. COPOTO provides such assessments with very limited manual and computational effort and thus facilitates the integration of real-time control into standard planning workflows of utilities.

Deciphering carbon emissions in urban sewer networks: Bridging urban sewer networks with city-wide environmental dynamics

As urbanization accelerates, understanding and managing carbon emissions from urban sewer networks have become crucial for sustainable urban water cycles. This review examines the factors influencing greenhouse gas (GHG) emissions within urban sewage systems, analyzing the complex effects between water quality, hydrodynamics, and sewer infrastructure on GHG production and emission processes. It reveals significant spatiotemporal heterogeneity in GHG emissions, particularly under long-term scenarios where flow rates and temperatures exhibit strong impacts and correlations. Given the presence of fugitive and dissolved potential GHGs, standardized monitoring and accounting methods are deemed essential. Advanced modeling techniques emerge as crucial tools for large-scale carbon emission prediction and management. The review identifies that traditional definitions and computational frameworks for carbon emission boundaries fail to fully consider the inherent heterogeneity of sewers and the dynamic changes and impacts of multi-source pollution within the sewer system during the urban water cycle. This includes irregular fugitive emissions, the influence of stormwater systems, climate change, geographical features, sewer design, and the impacts of food waste and antibiotics. Key strategies for emission management are discussed, focusing on the need for careful consideration of approaches that might inadvertently increase global emissions, such as ventilation, chemical treatments, and water management practices. The review advocates for an overarching strategy that encompasses a holistic view of carbon emissions, stressing the importance of refined emission boundary definitions, novel accounting practices, and comprehensive management schemes in line with the water treatment sector's move towards carbon neutrality. It champions the adoption of interdisciplinary, technologically advanced solutions to mitigate pollution and reduce carbon emissions, emphasizing the importance of integrating cross-scale issues and other environmentally friendly measures in future research directions.

Addressing the challenges of combined sewer overflows

Europe's ageing wastewater system often combines domestic sewage with surface runoff and industrial wastewaters. To reduce the associated risk of overloading wastewater treatment works during storms, and to prevent wastewater backing-up into properties, Combined Sewer Overflows (CSOs) are designed into wastewater networks to release excess discharge into rivers or coastal waters without treatment. In view of growing regulatory scrutiny and increasing public concern about their excessive discharge frequencies and potential impacts on environments and people, there is a need to better understand these impacts to allow prioritisation of cost-effective solutions.We review: i) the chemical, physical and biological composition of CSOs discharges; ii) spatio-temporal variations in the quantity, quality and load of overflows spilling into receiving waters; iii) the potential impacts on people, ecosystems and economies. Despite investigations illustrating the discharge frequency of CSOs, data on spill composition and loading of pollutants are too few to reach representative conclusions, particularly for emerging contaminants. Studies appraising impacts are also scarce, especially in contexts where there are multiple stressors affecting receiving waters. Given the costs of addressing CSOs problems, but also the likely long-term gains (e.g. economic stimulation as well as improvements to biodiversity, ecosystem services, public health and wellbeing), we highlight here the need to bolster these evidence gaps. We also advocate no-regrets options to alleviate CSO problems taking into consideration economic costs, carbon neutrality, ecosystem benefit and community well-being. Besides pragmatic, risk-based investment by utilities and local authorities to modernise wastewater systems, these include i) more systemic thinking, linking policy makers, consumers, utilities and regulators, to shift from local CSO issues to integrated catchment solutions with the aim of reducing contributions to wastewater from surface drainage and water consumption; ii) broader societal responsibilities for CSOs, for example through improved regulation, behavioural changes in water consumption and disposal of waste into wastewater networks, and iii) greater cost-sharing of wastewater use.

Improving the interpretability of deep reinforcement learning in urban drainage system operation

Deep reinforcement learning (DRL) has been increasingly used as an adaptive and efficient solution for real-time control (RTC) of the urban drainage system (UDS). Despite the promising potential of DRL, it is a black-box model whose control logic and control consequences are difficult to be understood and evaluated. This leads to issues of interpretability and poses risks in practical applications. This study develops an evaluation framework to analyze and improve the interpretability of DRL-based UDS operation. The framework includes three analysis methods: Sobol sensitivity analysis, tree-based surrogate modelling, and conditional probability analysis. It is validated using two different DRL approaches, i.e., deep Q-learning network (DQN) and proximal policy optimization (PPO), which are trained to reduce combined sewer overflow (CSO) discharges and flooding in a real-world UDS. According to the results, the two DRLs have been shown to perform better than a rule-based control system that is currently being used. Sobol sensitivity analysis indicates that DQN is particularly sensitive to the flow of links and rainfall, while PPO is sensitive to all the states. Tree-based surrogate models effectively reveal the control logic behind the DRLs and indicate that PPO is more comprehensible but DQN is more forward-looking. Conditional probability analysis demonstrates the potential control consequences of the DRLs and identifies three situations where the DRLs are ineffective: a) the storage of UDS is fully utilized; b) peak flows have already passed through actuators; c) a substantial amount of water enters one location simultaneously. The proposed evaluation framework enhances the interpretability of DRL in UDS operations, fostering trust and confidence from operators, stakeholders, and regulators.

Towards coordinated and robust real-time control: a decentralized approach for combined sewer overflow and urban flooding reduction based on multi-agent reinforcement learning

The real-time control (RTC) of urban drainage systems can make full use of the capabilities of existing infrastructures to mitigate combined sewer overflow (CSO) and urban flooding. Despite the benefits of RTC, it may encounter potential risks and failures, which need further consideration to enhance its robustness. Besides failures of hardware components such as sensors and actuators, the RTC performance is also sensitive to communication failures between the devices that are spatially distributed in a catchment-scale system. This paper proposes a decentralized control strategy based on multi-agent reinforcement learning to enhance communication robustness and coordinate the decentralized control agents through centralized training. To investigate different control structures, a centralized and a fully decentralized strategy are also developed based on reinforcement learning (RL) for comparison. A benchmark drainage model and a real-world drainage model are formulated as two cases, and the control agents are trained to control the orifices or pumps for CSO or flooding mitigation in each case. The three RL strategies reduce the CSO volume by 5.62-9.30% compared with a static baseline in historical rainfalls of the benchmark case and reduce the CSO and flooding volume by 14.39-21.36% compared with currently-used rule-based control in synthetic rainfalls of the real-world case. Benefitting from centralized training, the decentralized agents can achieve similar performance to the centralized agent. The decentralized control also enhances the communication robustness with smaller performance loss than the centralized control when observation communication fails, and provides a robust backup at the local level to limit the uncertainties when action commands from the centralized agent are lost. The results and findings indicate that multi-agent RL contributes to a coordinated and robust solution for RTC of urban drainage systems.

Predictive heuristic control: Inferring risks from heterogeneous nowcast accuracy

Urban Drainage Systems can cause ecological and public health issues by releasing untreated contaminated water into the environment. Real-time control (RTC), augmented with rainfall nowcast, can effectively reduce these pollution loads. This research aims to identify key dynamics in the nowcast accuracies and relate those to the performance of nowcast-informed rule-based (RB)-RTC procedures. The developed procedures are tested in the case study of Rotterdam, the Netherlands. Using perfect nowcast data, all developed procedures showed a reduction in combined sewer overflow volumes of up to 14.6%. Considering real nowcast data, it showed a strong ability to predict if no more rain was expected, whilst performing poorly in quantifying rainfall depths. No relation was found in the nowcast accuracy and the consistency of the predicted rainfall using a moving horizon. Using the real nowcast data, all procedures, with the exception of the one predicting the end of the rainfall event, showed a significant risk of operative deterioration (performing worse than the baseline RB-RTC), linked to the relative performance of the nowcast algorithm. Understanding the strengths of a nowcast algorithm can ensure the reliability of the RB-RTC procedure and can negate the need for detailed modelling studies by inferring risks from nowcast data.

Costs and benefits of combined sewer overflow management strategies at the European scale

Combined sewer overflows (CSOs) may represent a significant source of pollution, but they are difficult to quantify at a large scale (e.g. regional or national), due to a lack of accessible data. In the present study, we use a large scale, 6-parameter, lumped hydrological model to perform a screening level assessment of different CSO management scenarios for the European Union and United Kingdom, considering prevention and treatment strategies. For each scenario we quantify the potential reduction of CSO volumes and duration, and estimate costs and benefits. A comparison of scenarios shows that treating CSOs before discharge in the receiving water body (e.g. by constructed wetlands) is more cost-effective than preventing CSOs. Among prevention strategies, urban greening has a benefit/cost ratio one order of magnitude higher than grey solutions, due to the several additional benefits it entails. We also estimate that real time control may bring on average a CSO volume reduction of just above 20%. In general, the design of appropriate CSO management strategies requires consideration of context-specific conditions, and is best made in the context of an integrated urban water management plan taking into account factors such as other ongoing initiatives in urban greening, the possibility to disconnect impervious surfaces from combined drainage systems, and the availability of space for grey or nature-based solutions.