Optimizing the Management of Multireservoir Systems Under Shifting Flow Regimes

Abstract

Over the past few decades, significant research efforts have been devoted to the development of tools and techniques to improve the operational effectiveness of multireservoir systems. One of those efforts focuses on the incorporation of relevant hydrologic information into reservoir operation models. This effort is particularly relevant in regions characterized by low-frequency climate signals, where time series of river discharges exhibit regime-like behavior. Failure to properly capture such regime-like behavior yields suboptimal operating policies, especially in systems characterized by large storage capacity such as large multireservoir systems. Hidden Markov Modeling is a class of hydrological models that can accommodate both overdispersion and serial dependence in time series, two essential hydrological properties that must be captured when modeling a system where the climate is switching between different states (e.g., dry, normal, and wet). In terms of reservoir operation, Stochastic Dual Dynamic Programming (SDDP) is one of the few optimization techniques that can accommodate both system and hydrologic complexity, that is, a large number of reservoirs and diverse hydrologic information. However, current SDDP formulations are unable to capture the long-term persistence of the streamflow process found in some regions. In this paper, we present an extension of the SDDP algorithm that can handle the long-term persistence and provide reservoir operating policies that explicitly capture regime shifts. Using the Senegal River Basin as a case study, we illustrate the potential gain associated with reservoir operating policies tailored to climate states.

Key Points

• Incorporating climate variables into the state vector of Stochastic Dual Dynamic Programming yields more efficient reservoir operating policies

• Reservoir operating policies based on climate states naturally hedge against adverse hydrologic conditions

• Climate-derived streamflow forecasts are more reliable and accurate than those obtained from an autoregressive model

1 Introduction

Anthropogenic disturbances of the terrestrial water cycle, climate change, and low frequency climate signals all challenge the stationarity assumption, which has been the backbone of water resources planning techniques during much of the 20th century (Loucks & Van Beek, 2017b). Time series of hydrological observations can exhibit nonstationarity in different forms such as increasing or decreasing trends (Bayazit, 2015; Olsen et al., 1999), shifts in the mean (Fortin et al., 2004; Potter, 1991), and shifts in variance (Coulibaly & Burn, 2004; Whitcher et al., 2002).

Detecting and attributing the source of nonstationarity in hydrological records is essential for the efficient management of water resources systems. This paper focuses on nonstationarity due to climate variability, which must not be confused with climate change. Climate variability is random variation from a long-run average distribution, whereas climate change is a trend or a shift in the long-run distribution (Stedinger & Griffis, 2011). More specifically, we investigate the value of water allocation policies that explicitly capture climate variability through shifting flow regimes. Climate variability is often linked to low frequency climate signals such as El Niño-Southern Oscillation (ENSO), Pacific Decadal Oscillations, and North Atlantic Oscillation, which impact the hydrology in many regions worldwide (Akintug & Rasmussen, 2005; Bracken et al., 2014; Jha et al., 2021).

Observations of long-term fluctuations in time series of hydrological variables such as annual flows have triggered the development of methods to quantify this long-term persistence. Among such investigations, the work of Hurst (1951) stands out. While analyzing the time series of river discharges in the Nile River basin, Hurst found that there is a tendency of wet years to cluster into multi-year wet periods or of dry years to cluster into multi-year drought periods (Koutsoyiannis, 2002). In the late 60th, Baum and his colleagues presented the basic concept of a regime-shifting time series model named Hidden Markov Model (HMM) (Rabiner, 1989). HMMs, also known as Markov mixture models or Markov switching models, is a class of probabilistic models for “labeling” the observations. Rather than focusing on shifts in the mean of a process, HMMs estimate shifts in the state of a process (Whiting et al., 2004). HMMs, which allow the probability distribution of each observation to depend on the unobserved (hidden) state of a Markov chain, can accommodate both overdispersion (when there is a greater variability in the data set than would be expected based on a given statistical model) and serial dependence, which measures the correlation between two successive variables in time series (Loucks & Van Beek, 2017a). The motivation for adopting this type of model in hydrology lies in that the climate regime can be represented by a state variable that can take only a limited number of values. Consequently, along with a time series of historical river discharges, a HMM considers another time series with discrete climate states. These properties make this method a suitable option for modeling the governing hydrology of a system where the climate is switching between different states, say dry, normal, and wet. HMMs have been applied, for example, to analyze time series of daily precipitation data (Zucchini & Guttorp, 1991) or streamflow observations (Akintug & Rasmussen, 2005; Tan et al., 2017; Turner & Galelli, 2016; Yu et al., 2018).

Capturing those low-frequency shifts is particularly important in water resources systems characterized by large storage capacity because of their ability to move water over extended periods of time means that they could potentially take advantage of climate information in order to hedge against prolonged dry or wet periods (You & Cai, 2008; Zhao et al., 2012). However, when it comes to modeling techniques to optimize reservoir operation, only a few solution approaches can address the complexities associated with the presence of: (a) multiple hydrologic information/processes, (b) multiple, often conflicting, operating objectives, (c) stochastic variables, (d) nonlinear relationships, and (e) trade-offs between immediate and future consequences associated with the operation of the system (Pina et al., 2017).

Dynamic Programming (DP) is a well-established optimization method to solve the reservoir operation problem (Labadie, 2004). To achieve this, DP decomposes the original problem into sub-problems that are solved recursively. Stochastic DP (SDP) is an extension of DP that can accommodate stochastic state variables like reservoir inflows, snowpack, and flow forecasts (Stedinger et al., 1984). The incorporation of more hydrological information into the state vector has the potential to enhance SDP-derived policies and thus improve system efficiency. However, due to the curse of dimensionality, that is, the exponential increase in computational effort and volume associated with adding extra dimensions to the state-space, the number of state variables is often limited to three to four. To handle the uncertainties associated with various hydrologic information, variants of SDP have been proposed such as Bayesian SDP Karamouz et al. (2003) or Sampling SDP (Kelman et al., 1990; Kim & Palmer, 1997; Liu et al., 2020; Mujumdar & Nirmala, 2007; Tang et al., 2010).

Although useful, those variants fall short of capturing the long-term persistence that characterizes time series of river discharges exhibiting a regime-like behavior. Recently, Turner and Galelli (2016) proposed a SDP formulation that can handle long-term persistence through an additional state variable representing climate states. This new formulation, called SDP-Φ, uses a Hidden Markov Model (HMM) to partition the streamflow time series into a small number of discrete regime states such as dry, normal or wet. That study confirms that regime-like behavior is a major cause of suboptimal hedging decisions for reservoirs that are vulnerable to prolonged dry spells. While this SDP-Φ has been successfully implemented on single reservoir operation problems, it cannot be applied to larger systems due to the curse of dimensionality.

Our literature review reveals that only few solutions are available to handle both hydrologic and system complexities. One of them is Direct Policy Search, a methodology that has the capability of handling multiple objectives and sources of uncertainties (Libisch-Lehner et al., 2019). However, as pointed out by Castelletti et al. (2013), those simulation-based optimization methods are computationally demanding when the number of objectives increases, and difficult to parametrize when the water resources system is large and the network complex. Another alternative solutions is Stochastic Dual Dynamic Programming (SDDP) (M. V. Pereira & Pinto, 1991) which belongs to the field of Approximate DP (Powell, 2007). The SDDP algorithm relies on an iterative procedure (i.e., backward optimization and forward simulation) to constructs a locally-accurate approximation of the benefit-to-go function through sampling and Benders' decomposition (M. Pereira & Pinto, 1983). The basic principle is to approximate the benefit-to-go function using a piecewise linear function which is progressively refined by iteratively adding new hyperplanes (cuts) until the approximation is found to be statistically acceptable. As we will see later in Section 2.5, these piecewise linear approximations are constructed from the primal and dual solutions of the one-stage optimization problems as the algorithm progresses backward.

Generally speaking, SDDP uses a multisite periodic autoregressive model (MPAR) to capture the hydrologic uncertainty. This model can represent serial and spatial correlations within a river basin and between different basins as well as seasonality. A number of recent researches have examined ways of improving the built-in hydrological model. Lohmann et al. (2016) suggests a Spatial PAR model for SDDP that can take spatial information (such as distance between hydro plant) into account. The idea is that inflows into “neighboring” plants are correlated and this correlation can be accounted for by using neighboring inflow series to explain the inflow at a given location. Pritchard (2015) introduces an inflow modeling approach that can overcome probabilistic sampling errors introduced when the fitted continuous inflow model is replaced by a discrete approximation obtained through sampling. Poorsepahy-Samian et al. (2016) use the Box-Cox transformation in the inflow modeling of SDDP while meeting the convexity requirement of the algorithm. Raso et al. (2017) present a stochastic streamflow model with a multiplicative component and a nonuniform time step. Pina et al. (2017) propose an approach to incorporate exogenous hydrological variables (e.g., snow water equivalent, ENSO variables) into the SDDP algorithm through a MPARX model. In Treistman et al. (2020), ENSO variables are rather treated as additional state variables modeled by a Markov chain along with the PAR(p) model for the streamflows. More recently, Mbeutcha et al. (2021) incorporate a moving average component in the built-in hydrological model of SDDP.

However, none of the reported extensions of the SDDP algorithm can handle low-frequency signals. The main focus of this study is to extend the SDDP algorithm so as to determine reservoir operating policies that explicitly capture shifting flow regimes due to climate variability. To achieve this, the state-space vector now includes a climate variable whose transition is governed by a HMM and the cuts approximating the benefit-to-go functions are also classified according to their climate state. The proposed SDDP-HMM model is then implemented on the multipurpose, multireservoir system in the Senegal River basin in West Africa, a region characterized by shifting flow regimes (Faye et al., 2015).

This paper is organized as follows: Section 2 presents the methodology to efficiently capture historical streamflow regime changes through HMM followed by a description of the modified SDDP algorithm to solve the multireservoir operation problem. Section 3 then provides a description of the Senegal River Basin. Section 4 discusses the results. Finally concluding remarks, limitations, and potential extensions are discussed in Section 5.

2 Material and Methods

2.1 Unfolding the Hidden Climate States

In numerous river basins around the world, hydrologic time series display characteristic responses to large scale climate patterns (Akintug & Rasmussen, 2005; Bracken et al., 2014). For example, let us consider the time series of annual streamflows at Bakel in the Senegal River basin (West Africa) from 1904 to 2011 (Figure 1). This time series displays a regime-like behavior with wet periods during [1904–1910], [1915–1937], and [1943–1970], and dry periods during [1910–1915], [1937–1943], and [1970–2011].

Figure 1

Annual streamflow at Bakel in SRB (1904–2011) (a) Mean annual streamflow, (b) Normalized streamflow (the dashed line represents the 3-year running mean).

For multi-year reservoir operation planning, characterizing these lengthy departures from the mean are essential as they stress the system far more than single wet or dry years (Bracken et al., 2014). Capturing such a long-term persistence therefore can be achieved by considering the climate states associated with flow regimes, something that can be achieved using, for example, Hidden Markov Models (HMM) through labeling the observations (Rabiner, 1989). In other words, when dealing with a time series of river discharges, it means that there exists another time series with discrete climate states that can take only a limited number of values (i.e., dry/wet for 2 states; dry/normal/wet for 3 states) (Figure 2). Denote {q₁, q₂,…,q_T} the time series of monthly flows and let {Φ₁, Φ₂,…,Φ_T} be the time series of climate states which can only take Φ possible climate states.

Figure 2

Time series of river discharges and hidden climate states.

The state variable is unobserved and is accordingly referred to as a hidden variable. The key information that we need to extract from the analysis of time series of river discharges are the transition probabilities from one climate state to another in a single time step. The probabilities of moving from climate state i at time t − 1 to climate state j at the next time step (t):

(1)

are the elements of the (Φ × Φ) transition probability matrix P between time t−1 and t.

The observed variable q_t is assumed to have been drawn from a probability distribution whose parameters are conditional upon the distinct state at time t such that, when Φ_t is known, the distribution of q_t only depends on the current state Φ_t (Figure 3). A HMM is described by (a) the parameters of the Gaussian distributions, that is, the mean μ = (μ₁, μ₂, …, μ_Φ) and standard deviation σ = (σ₁, σ₂, …, σ_Φ) associated with Φ states, (b) the (Φ × Φ) transition probability matrix P, and (c) the initial distribution of the Markov chain δ. Consequently, the set of parameters to be estimated is θ = {μ, σ, P, δ}.

Figure 3

Structure of Hidden Markov Model with 3 hidden climate states Dry (D), Normal (N), and Wet (W).

Prior to fitting a HMM to the observed sequence (here the time series of monthly inflows), outliers and the seasonal component must be removed. Then, the likelihood of observing that sequence, as calculated under a Φ-state HMM, must be evaluated. In this study, we use the Expectation-Maximization (EM) algorithm, an iterative method for finding the maximum-likelihood estimate of the parameters of an underlying distribution when some of the data are missing. In the context of HMM, the EM algorithm is known as the Baum-Welch algorithm (Welch, 2003) and the hidden climate states are treated as missing data (Bilmes, 1998; Zucchini et al., 2017). The EM algorithm consists of two main phases: an expectation phase called “E step,” followed by a maximization phase called “M step.” Given the current estimate of the HMM parameters θ, the following steps are repeated until acceptable convergence is achieved: The “E step” phase of the algorithm computes the expected value of unobserved data (i.e., hidden climate states) using the current estimate of the parameters and the observed data. The “M step” phase of the algorithm then provides a new estimate of the parameters by using the data from the “E step” phase as if they were actually measured data. These parameters are then used to calculate the distribution of unobserved data in the next “E step” phase of the algorithm. The resulting value of θ is then the stationary point of the likelihood of the observed data.

Next, we want to determine the sequence of hidden climate states {Φ₁, Φ₂, …, Φ_T} that has most likely (under the fitted HMM) given rise to the sequence of observations (here the time series of monthly river discharges). In the literature, this is well-known as the decoding problem. In this study, we use the Viterbi algorithm (Viterbi, 1967) to unfold the sequence of hidden climate states (called the Viterbi path). This, in turn, enables us to breaking the observations into Φ distinct regimes and to calculating the probabilities of transitioning between inflow regime states. As we shall see in the following sections, these transition probabilities can then be used in the SDDP algorithm to derive water allocation policies tailored to discrete climate states such a dry, normal, or wet.

2.2 The Multireservoir Operation Problem

The operation of a multireservoir system is a multistage decision-making problem. The problem is to determine a sequence of allocation decisions x_t that maximize the expected net benefit from system operation over a given planning period while meeting operational and/or institutional constraints (Labadie, 2004).

(2)

where Z is the expected sum of one-stage net benefits from system operation, T is the planning horizon, x_t is the vector of allocation (decision) variables, ν(.) is the terminal value function, α_t is the discount factor at stage t, and is the expectation operator. In general, the vector of the state variables S_t includes the volume of water in storage s_t as well as hydrologic state variables h_t. The allocation decisions can be reservoir storages, turbined outflows, spillage losses, and water withdrawals for off-stream uses, b_t(.) represents the aggregated net benefits at stage t, which includes here the net benefit from irrigated agriculture and hydropower generation (the two dominant uses). Note that other operating objectives (e.g., navigation, ecological flows) are handled using penalty coefficients: b_t(.) is penalized when the desired objective is not met.

The most common constraints are:

1. Water balance equations:

   (3)
   where s_t is the vector of storage at the beginning of the period t, r_t is the vector of releases, l_t is the vector of spills, i_t is the vector of water withdrawals, q_t is inflow during time t, C^R is the reservoir system connectivity matrix,  = 1 (−1) when reservoir n₁ receives (release) water from (to) reservoir n₂, C^I is the connectivity matrix for return flows, and e_t is the vector of evaporation losses.

2. Lower and upper bounds on storage levels:

   (4)
   where , and are the lower and upper bound on storage level respectively.

3. Limits on reservoir releases:

   (5)

   Maximum release is introduced to account for the maximum carrying capacity downstream of the reservoir, depending on the capacity of the hydraulic structures (e.g., penstock or spillway). Minimum allowable release is given to maintain a desired downstream minimum flow for water quality, navigation, and etc.

4. Water withdrawals can be limited by the pumping station or channel capacity:

(6)

As indicated earlier, several features make the reservoir operation problem Equations 2-6 computationally challenging to solve with optimization techniques: the presence of stochastic variables, multiple, often conflicting, objectives, and nonlinear functions such as the hydropower production function, evaporation, and other losses (Rani & Moreira, 2010). Karamouz et al. (2003) suggests a time decomposition approach to deal with the complexity of the reservoir optimization problem. A time decomposition approach breaks down the original problem into long, mid, and short-term planning periods, each having a specific model. Long-term planning models (e.g., monthly time step and over-year planning horizons) are run first and provide strategic and tactical policy information. This information (e.g., future marginal water value and storage levels) then constitutes the boundary conditions for the mid- and short-term optimization tools (e.g., weekly to daily time horizons), whose outputs finally feed real-time operation models. This study focuses on the long-term reservoir operation problem and seeks to identify steady-state, monthly, reservoir operating policies that explicitly capture regime shifts.

2.3 One-Stage SDDP Problem

SDDP solves the optimization problem Equations 2-6 by decomposing it into a sequence of one-stage problems that are solved recursively. Let F_t be the benefit-to-go function (BTF) from stage t to the end of the planning period T associated with decision x_t, h_t be the vector of hydrological state variables, the one-stage SDDP optimization problem (OSOP) is a linear program:

(7)

Subject to constraints Equations 3-6, as well as:

1. Approximation of the hydropower production functions:

   (8)
   where H is the number of planes approximating the true non-linear hydropower functions, ω, δ, and ψ are the parameters derived from the corresponding convex hulls (Goor et al., 2010).

2. Benefit-to-go F_t+1 is now a scalar bounded from above by the following inequalities (cuts):

   (9)
   where L is the number of cuts (hyperplanes), φ_t+1, γ_t+1, and β_t+1 are 1 × n vectors of cut parameters calculated at stage t + 1 (Tilmant et al., 2008), and h_t typically includes the natural inflows observed during the last p periods (q_t−1, q_t−2, …, q_t−p).

2.4 Incorporating the Hidden Climate States Into SDDP

A common approach to capture the inflow uncertainty in SDDP is through a built-in multi-site periodic autoregressive model MPAR. The parameters of the MPAR model are estimated from historical flow records (Pina et al., 2017). Suppose the periodic process is modeled by an MPAR of order p - MPAR(p) - then at each site n, the incremental inflows can be derived from:

(10)

where q_t is the incremental flow at time t, is periodic mean of q_t, is standard deviation of q_t, ϕ_t is autoregressive parameter of the periodic model, and ϵ_t is a time dependent-spatially correlated stochastic noise.

Incorporating the (hidden) climate state variables into the state space vector of SDDP offers the potential to improve the operating policies derived by SDDP since the long-term persistence will be captured. As mentioned earlier in Section 2.1, the historical flow record can be broken into Φ distinct classes by a fitted HMM, providing Φ classes of inflows for each month (e.g., a HMM with 3 states, say dry, normal, and wet, yields 3 classes of inflow for each month). Instead of having a single time series of historical observations, the HMM now provides another time series with discrete climate states. At stage t − 1, assume (a) that is sampled from ith climate state and (b) that, for notational simplicity, an autoregressive model of order one is adjusted:

(11)

where , and are respectively the vectors of periodic means and standard deviations of jth climate state (Figure 4). In the backward phase of SDDP, at the beginning of stage t, we can use the modified hydrological model to (a) generate the K inflow scenarios q_t,j from the sampled previous flow q_t−1,i and (b) analytically calculate the cuts' parameters for stage t − 1 (see Appendix A). Note that the K inflow scenarios correspond to an ensemble forecast with K members and with a forecast horizon of only one period (here 1 month). These inflow scenarios are also called backward openings in SDDP (M. V. Pereira & Pinto, 1991). In the forward phase, we can either use the synthetically generated scenarios or historical records to simulate the system over the planning period. The proposed approach can therefore handle both the short-term and the long-term persistence, the former through autoregressive models, and the latter via the fitted HMM.

Figure 4

Schematization of inflow modeling considering 3 climate states: dry (d), normal (n), and wet (w).

2.5 SDDP Algorithm With Hidden Climate States

When hidden climate states are included in the state-space vector of SDDP, the objective function of the corresponding OSOP becomes:

(12)

where are the transition probabilities.

Compared to the traditional SDDP formulation, the cuts are now categorized based on their climate state j:

(13)

where , , and are 1 × n vectors of cuts' parameters when the system is in climate state j. These parameters are analytically derived from the primal and dual information that becomes available as the algorithm progresses backward (see Appendix A for more details on the mathematical procedure to derive the cuts' parameters). Note that the other constraints Equations 3-8 of the standard SDDP model remain the same.

Since the cuts Equation 13 only offer an approximation overestimating the true benefit-to-go and thus Z, a forward simulation of the system with those cuts will automatically provide a lower bound to Z. In each forward simulation phase, the system is simulated using a set of M hydrologic sequences, which can be historical or synthetically-generated using, for example, Equation 10 or Equation 11. At the end of the simulation phase, the average total return over the M hydrologic sequences is therefore a lower bound to Z, which must be compared to the upper bound calculated at the end of the backward optimization phase. When the difference between the upper bound falls within the confidence interval around the lower bound, the algorithm stops (Tilmant et al., 2008).

The pseudo-codes of the backward optimization and the forward simulation phases with hidden climate states are given next:

Algorithm 1 Backward Optimization

Algorithm 2 Forward Simulation

Compared to the traditional SDDP formulation, the backward optimization phase comprises an additional loop on the hidden climate states (Φ) and the cuts are stored according to their climate state. In the forward simulation phase, at stage t, the benefit-to-go F_t+1 is then the weighted sum of F_t+1,j where the weights are the transition probabilities , that is, the probabilities of moving from, say, the climate state i at time t to the different climate states j at time t + 1 (j = 1, 2, …, Φ). Since this additional loop in the backward optimization phase automatically increases the computation time, it is one factor to be considered when selecting the number of discrete climate states alongside with the length and inherent variability of streamflow records.

The next section describes the case study that was used to illustrate the gain associated with SDDP-derived allocation policies tailored to specific climates/regimes.

3 Case Study

The Senegal River has a flow regime characterized by multiyear dry, normal, and wet periods (Bader et al., 2014). The basin covers an area of about 337 000 km² in West Africa and is shared by four countries: Guinea, Mali, Mauritania, and Senegal (Figure 5). Three tributaries, the Bafing, Faleme, and Bakoye all have their source in the Fouta Djallon, a high plateau located mainly in the Guinean portion of the SRB. Together, these three tributaries contribute to about 90% of the water flowing in the Senegal River. The basin has three distinct stretches: the mountainous upper basin, the valley, and the delta, which is rich in biological diversity and wetlands.

Figure 5

The Senegal river basin.

In the 1940s, the first major attempt to control the Senegal River discharges were made in order to cultivate rice in the delta (United Nations Report, 2003). Today, irrigation is still the motor of development in the basin, particularly in the valley and in the delta. Fishing, is also an important activity in the basin, specially in the valley and the delta. More recently, Senegal River flows have been used to spin the turbines of two hydropower plants: Manantali, which is a 200-MW power station, and Félou, a 62-MW run-of-river power plant. Both supply energy to the West African Power Pool.

The climatic regime of the basin is defined by three seasons, a rainy season from June to September, a cold, dry off-season from October to February, and a hot, dry off-season during the rest of the year. The river flow regime depends, for the most part, on the rain that falls in the upper basin in Guinea (about 2000 mm/year). In the valley and the delta, however, the rainfall is rarely more than 500 mm/year. The year-to-year variability of the river discharges during the high flow season is significant and exposes water users to a high hydrological risk (Tilmant et al., 2020). As indicated above, the flow regime is also characterized by periods of low, normal, and high flows, which can persist for several years. The multiyear drought of the 80ies, for example, is well documented and has had devastating consequences on local communities whose livelihood depends on the flow and the banks of the river for fishing and/or subsistence agriculture (Bruckmann & Beltrando, 2016).

The Senegal is one of the few transboundary river basins managed by a genuine active joint-management organization (OMVS, the Senegal River Basin Authority) with personnel coming from the four riparian countries (Kliot et al., 2001). OMVS’ mandates involve not only the planning but also the operation of water resources in the basin. In this study, the intermediate development of the SRB is considered (Figure 6). It involves 5 reservoirs, 6 power plants, and about 200.000 irrigated hectares. In the next section, optimal allocation policies for that particular future development scenario will be determined and then simulated using historical streamflow data over the 1904–2011 period.

Figure 6

Schematization of the SRB for an intermediate development scenario (Tilmant et al., 2020).

Table 1 lists the main characteristics of the system while agricultural water demands are presented in Table 2. The future configuration of the system has a total storage capacity of 22.3 km³, which corresponds to a 88% increase compared to the current storage capacity. The power plants listed in Table 1, will add 626 MW (+310%) to the installed capacity of the current system. The reservoir inflow-to-storage ratios computed on the mean of the cumulative inflow are 2.09, 3.86, 1.65, and 2.47 for Koukoutamba, Boureya, Manantali, and Gourbassi, respectively, indicating that the system has a seasonal carryover capacity.

Table 1. SRB System Characteristics (Intermediate Development)

Name	Node ID	Storage (km^3)	Installed capacity (MW)
Koukoutamba	2	3.60	294
Boureya	4	4.75	114
Manantali	6	11.30	200
Gouina	12	ROR^a	140
Felou	13	ROR^a	60
Gourbassi	16	2.10	18
Diama	23	0.58	–

• ^a ROR: run of river power plant.

Table 2. Crop Water Demands (Intermediate Development Scenario)

Node ID	Withdrawals (hm^3/y)	Area (ha)
3	369	19926
7	30	1562
14	208	10948
17	17	862
18	140	7147
19	1174	59936
20	553	28219
21	1097	44402
22	1537	78498
24	75	3827
Total	5198	255327

The monthly incremental inflows at the different nodes of the network are based on naturalized flows extracted from Bader et al. (2014) for the 1904–2011 period.

4 Results and Discussion

4.1 Climate Sequences and HMM Classifications

To unfold the hidden climate states attached to the inflow time series in the SRB, we first fit a HMM to the time series of monthly streamflow data measured at Bakel (Senegal River). Bakel marks the transition between the upper (humid) part of the river basin and the lower valley where the contribution of tributaries to the river discharges is much smaller; as mentioned earlier, about 90% of the flows in the SRB goes through Bakel. Prior to fitting a HMM, outliers and the seasonal component are removed (Shoelson, 2021). Then, model parameters are fitted using the EM algorithm. The optimum order of HMM (Φ) is not known a priori and must be selected based on objective criteria. Both the Akaike Information Criteria (AIC) (Akaike, 1974) and the Bayesian information Criteria (BIC) (Schwarz, 1978) can be used to determine the optimum order for a HMM. Note that a trade-off must be found between the model complexity (i.e., the number of HMM parameters to be estimated) and the number of climate state that can properly represent historical climate shifts. In this study the added value for both AIC and BIC when considering more than 3 climate states was constant, indicating that there is no added-value to more than three states. The analysis is therefore carried out with Φ = 3, representative of dry, normal, and wet climate regimes as illustrated in Figure 7.

Figure 7

Persistent structure of the underlying regime process at Bakel in SRB considering three possible climate states (i.e., dry, normal, and wet).

Results show that the hidden climate state with the lowest mean, denoted by “dry,” dominates the period between 1971 and 1996, which corresponds to the extended drought of the late 20th century, which has been analyzed, for example, by Faye et al. (2015) and Sagna et al. (2021). The monthly river discharges are then classified based on the fitted HMM. Figure 8 compares the statistics of the climate states for each month. The red, black, and blue boxes are associated with dry, normal, and wet climates/regimes respectively. We can see that the climate states' statistics are distinct, which validate the choice of the underlying structure of the HMM (Figure 8).

Figure 8

Classified monthly inflows at Bakel derived from Hidden Markov Model when Φ = 3.

Table 3 presents the transition probability matrix associated with the HMM. As we can see, the transition probabilities are close to zero or one, suggesting again that the states are clearly distinct. The values close to one on the diagonal indicate that when the climate is in a particular state, it will likely remain in that state at the next time period (month). In other words, shifting from one climate state to another is far less likely. For the dry climate state, for example, there is more than 90% probability of remaining in that state and less than 1% probability of transitioning to a wet state.

Table 3. Transition Probability Matrix From ith Climate State to the jth Derived From Fitted Hidden Markov Model at Bakel in SRB When Φ = 3

	Dry	Normal	Wet
Dry	0.921	0.067	0.012
Normal	0.078	0.858	0.064
Wet	0.010	0.110	0.880

4.2 Analysis of Simulation Results

To assess the gains associated with climate-derived allocation policies, three SDDP formulations, each having a specific hydrologic model, are implemented and their performances compared. The first formulation attempts at capturing the temporal persistence of the hydrologic processes through a MPAR(p) model (de Matos & Finardi, 2012; Jasson et al., 2017) whose order p varies in space (site) and time (month). This version is denoted SDDP-MPAR. The second formulation is the proposed extension in which the state-space vector now includes as hydrologic variables the previous flow q_t−1 and the climate state Φ_t with the latter capturing the long term persistence through three different states (dry, normal, and wet). This second version is denoted SDDP-HMM. The third formulation is actually a deterministic dual DP (DDDP) in which one assumes perfect foresight of future inflows, therefore overestimating the performance of the system. The comparison between SDDP-HMM and SDDP-MPAR will reveal the gains associated with allocation policies that explicitly capture the long-term persistence and that are therefore tailored to shifting flow regimes. The comparison between SDDP-HMM and DDDP will indicate the extent of the losses associated with an imperfect knowledge of future hydrologic conditions. In other words, the difference in terms of performances between SDDP-HMM and DDDP is the opportunity cost associated with imperfect flow forecasts. Table 4 lists the main characteristics of the three dual DP formulations.

Table 4. Characteristics of Two Different Built-In Hydro Model of Stochastic Dual Dynamic Programming

Formulation	p	Φ	Time series of inflow	Time series of climate states
SDDP-MPAR(p)	Up to 6	–	✓	×
SDDP-HMM(Φ)	1	3	✓	✓
DDDP	–	–	✓	×

Both SDDP formulations (with and without climate states) are implemented with the following hydrologic configuration: the number of backward openings is set to forty (i.e., K = 40), and forward simulations are carried out on 40 historical sequences of 5 years (i.e., M = 40). This setup is in line with previous studies (e.g., Pina et al. (2017); Tilmant et al. (2009) and was found to be a good compromise between computation time and the accurate representation of the hydrologic uncertainty. Both SDDP built-in hydro models are calibrated using 107 years of monthly historical record extracted from Bader et al. (2014) for the 1904–2011 period.

For both SDDP formulations, the planning period is 60 months. Since the boundary conditions imposed on the state variables influence the allocation decisions during the first (by initial storages) and last stages (by zero terminal value function) (Rougé & Tilmant, 2016), a buffer of 2 years is considered in this study. This assumption is motivated by the physical characteristics of the system and the fairly detailed data set, which prevent the presence of multiple near-optimal solutions (Rougé & Tilmant, 2016). It means that once the SDDP algorithm has converged, the cuts from January to December of the third year are kept and the others are discarded. The results are analyzed after reoptimizing the policies along the 107 years of historical river discharges. Reoptimizing the policies is similar to the procedure proposed by Tejada-Guibert et al. (1993) with SDP except that the benefit-to-go function F_t+1 is now piecewise linear (Tilmant et al., 2020). The reoptimization procedure was implemented here because we wanted to see how the proposed extension would handle the multiyear drought of the 80ies, which had catastrophic consequences on riverine communities and led to mass migration (Dia, 2007).

Table 5 lists the average and standard deviation of annual hydropower generation for the three dual DP formulations. As we can see, with SDDP-MPAR, the mean annual hydropower generation is roughly 5.7% less than the deterministic case. When the hidden climate states are included in the algorithm, this difference is reduced to 4%. In other words, incorporating the climate states can fill 30% (1.7/5.7 × 100) of the gap in terms of energy. The standard deviation reflects the variability of hydropower generation in the basin. For example, with optimum operating policies provided by SDDP-HMM, the variability of power production during dry, normal, and wet years is 2.3% less than the traditional SDDP-MPAR formulation. The impact of these decisions on the amount of water spilled is also presented in Table 5 where we can see with SDDP-HMM the average annual spillage is reduced by 3.7% compared to the traditional SDDP formulation.

Table 5. Average Annual Results in Terms of Hydropower Generation and Spillage: Mean and Standard Deviation

Energy	SDDP-MPAR	SDDP-HMM	DDDP
Mean (GWh)	3493	3554	3704
Std (GWh)	920	898	881

Spillage	SDDP-MPAR	SDDP-HMM	DDDP
Mean (hm^3)	10923	10512	8966
Std (hm^3)	12582	12077	9976

The stationary probabilities associated with dry, normal, and wet years are 28%, 53%, and 19% respectively. These stationary probabilities can be used to analyze the differences between model formulations within the dry, normal, and wet periods. For example, Figure 9 traces out the cumulative distribution of total annual hydropower generation for the three dual DP formulations. As we can see, the DDDP clearly provides an upper bound on the performance in terms of energy. During dry years, with a non-exceedance probability between zero and 28%, considering the long-term persistence is beneficial because the results with SDDP-HMM are almost as good as DDDP. During normal years, however, this gain is less pronounced and the difference between the cumulative distribution associated with SDDP-MPAR and SDDP-HMM becomes marginal; both SDDP formulations (with and without climate states) generate similar policies meaning that from a decision-making point-of-view, the climate states become informationless during “normal” years. The added-value comes back again during wet years, which are associated with non-exceedance probabilities beyond 81%. These results clearly indicate that the incorporation of hidden climate states yields better policies during extreme hydrological conditions (dry and wet years) compared to the traditional SDDP formulation.

Figure 9

Cumulative distribution of total annual hydropower generation for three Stochastic Dual Dynamic Programming formulation in the SRB.

Taking the DDDP model as a benchmark, Table 6 lists, for both SDDP formulations, the mean annual reduction in energy output and the mean annual increase in spillage losses that one can expect during dry, normal, and wet years. During dry years, compared to the deterministic case, the energy output is reduced by 10.7% when allocation policies only consider the short-term persistence. This difference reaches 6.51% when allocation policies are tailored to climate states. During normal years, as indicated above, SDDP-MPAR and SDDP-HMM propose similar allocation policies since the differences with respect to the deterministic case are roughly the same: 4.41% versus 4.11%. Allocation policies derived from SDDP-MPAR and SDDP-HMM are again distinct during wet years where the difference, in terms of energy output, with respect to the deterministic case are now: 4.22% versus 1.82%.

Table 6. Energy - Average Annual Differences With Respect to the Deterministic Dual Dynamic Programming Model

Hydro condition	Annual loss in terms of energy (%)		Annual increase in spillage losses (%)
	SDDP-MPAR(p)	SDDP-HMM(3)	SDDP-MPAR(p)	SDDP-HMM
Dry years	10.77	6.51	35.73	8.35
Normal years	4.41	4.11	21.66	21.32
Wet years	4.22	1.82	22.34	13.89

The impact of hidden climate states on storage trajectories is analyzed in Figure 10 where we can see the time series of aggregated storages over the simulation period for the three dual DP formulations. At the onset of the multiyear drought centered around the 80ies, SDDP-MPAR is almost depleting the reservoirs too quickly. This can be explained by the fact that the autoregressive model is overestimating the inflows during dry years. However, when the hidden climate states are included in the algorithm, allocation policies better hedge against such adverse conditions, and storage levels are kept higher and closer to that of DDDP.

Figure 10

Aggregated reservoir trajectories for the three Stochastic Dual Dynamic Programming formulations (1904–2011).

4.3 Evaluating the Performance of the SDDP Built-In Hydro Models

As mentioned earlier in Section 2.4, the SDDP algorithm requires that, at stage t, K vectors of inflows q_t be generated by the built-in hydrological model so that K OSOP can be solved to capture the stochasticity of the hydrologic conditions and to derive the expected values of the parameters of the cuts approximating the benefit-to-go function. In other words, for each node n, an ensemble of K streamflow forecasts is available at the beginning of each stage. Generally speaking, the better the forecasts, the better the approximation of the benefit-to-go function F_t since the cuts' parameters directly depend on the primal and dual information associated with the K OSOP solved at stage t, one for each forecast. See Appendix A for more details on the procedure to derive the cuts' parameters.

In hydrology, the overall quality of ensemble streamflow forecasts is assessed with the continuous ranked probability score (CRPS) (Matheson & Winkler, 1976). This common verification metric measures the quadratic distance between the cumulative distribution of the forecasts and the cumulative distribution of the observations, with a value of 0 indicating a perfect forecast. As the CRPS assesses the forecast for a single time step, the MCRPS is defined as the average CRPS over the entire planning period T (Thiboult et al., 2016). To allow for the comparison among different nodes, we normalized the MCRPS with the standard deviation of the observations (SD_o) (Cassagnole et al., 2021). The normalized mean continuous ranked probability score is estimated using the following equation:

(14)

where o_t is the observed inflow, is the cumulative probability distribution of the forecasts at time t, q_t is the predicted inflow, and is the Heaviside function defined as:

(15)

Table 7 essentially compares the overall quality of the forecasts of the incremental flows generated by the two built-in hydrological models: MPAR or MPAR-HMM. The values highlighted in bold show the best model. We can see that HMM-based forecasts are more accurate than those derived from a model that only captures the short-term persistence (MPAR). The difference is larger for the nodes draining portions of the Fouta Djallon where precipitations are abundant but where the contrast between dry and wet years is also more pronounced. Further downstream, as the river runs north-west and the climate gets drier, the relative contribution of tributaries draining the northern part of the basin becomes not only less important but also less variable year-to-year. Consequently, forecasting the incremental flows at the two run-of-river power plants, Félou and Gouina, is achieved with a similar accuracy regardless of the model.

Table 7. NMCRPS Scores for the Two Hydrological Models: Multisite Periodic Autoregressive Model (MPAR) and MPAR-HMM

Name	Node ID	MPAR	MPAR-HMM
Koukoutamba	2	0.103	0.094
Boureya	4	0.103	0.097
Manantali	6	0.104	0.097
Gouina	12	0.108	0.107
Felou	13	0.108	0.109
Gourbassi	16	0.126	0.109

As mentioned above when analyzing both the time series of aggregated storages and the results for the energy sector, the difference between both SDDP formulations is more obvious during dry and wet years. Such a difference comes from the fact that the corresponding built-in hydrological models perform differently as soon as the hydrologic conditions are departing from normal. Figure 11 shows the cumulative distribution of the accumulated annual inflows at each power plant compared to the historical observations for both hydrological models: MPAR and MPAR-HMM. As we can see, none of the models is able to fully capture the variability of the observations. In other words, the spread of the annual streamflows is underestimated. However, as soon as the hidden climate states are incorporated, the situation improves and the cumulative distribution of annual streamflows derived from the MPAR-HMM model moves closer to the CDF of the observations, especially during dry and wet years.

Figure 11

Cumulative distribution of Annual inflows for each hydropower plant in the SRB.

Regarding the agricultural sector, the performances associated with the three dual DP formulations are identical: all irrigation schemes are supplied with a reliability above 95%. Even though the analyzed system corresponds to a future development scenario with almost three times as many irrigated hectares as today, the agricultural water demand is still lower than the available resources, especially when the five reservoirs are operational.

5 Conclusions

As low frequency signals are affecting the hydrology worldwide, several river basins are experiencing cycles of prolonged dry and wet periods. Properly capturing such a regime-like behavior is essential to enhance the performance of our water resources systems. This is especially the case for the multireservoir systems with large storage capacity because they can move water over extended periods of time. However, determining regime-tailored allocation policies for large-scale water resources systems is challenging as it often requires trading-off hydrological complexity for system complexity.

In this study, we address this trade-off by combining Hidden Markov Modeling with SDDP. The former can be used to capture the long-term persistence that characterizes the flow regime of many rivers, while the later can be used to determine optimal allocation policies in large multireservoir systems.

The proposed extension of SDDP, called SDDP-HMM, is tested and compared to two other dual DP formulations using a future development scenario of the Senegal River basin as an example. The analysis of simulation results shows: (a) Considering the hidden climate states is particularly relevant during hydrological extremes; reservoir operating policies provided by SDDP-HMM better hedge against adverse hydrologic conditions; (b) The hydrologic forecasts computed by a model that captures both the short-term and the long-term persistences are more accurate than those obtained by the traditional MPAR model found in many SDDP formulations.

Our results are consistent with previous studies such as Turner and Galelli (2016) and show that the incorporation of relevant hydrologic information can improve the performance of a water resources system. The CPU time required for SDDP-MPAR, and SDDP-HMM, was about 8 and 25 min, respectively, on MacBook Pro with the eight-core Apple M1 chip with 8 GB of RAM. Despite the fact that the new SDDP formulation is three times slower, the computational time remains fairly low, making it possible to embed SDDP-HMM in decision support systems.

The proposed approach to address the trade-off between system complexity and hydrological complexity relies on an extension of SDDP, an optimization algorithm which requires that the one-stage optimization problems be linear programs. This condition has several implications. The most important ones are: (a) that non-linear relationships must be made piecewise linear such as Equation 8 for the hydropower production function, (b) the built-in hydrological model Equation 10 must be linear. As shown by Rougé and Tilmant (2016), a side-effect associated with these approximations is that the algorithm is prone to the presence of multiple near-optimal solutions when inputs data are limited and/or when the system has significant carryover storage capacity. When these two conditions are met, we recommend implementing the year-periodic reoptimization procedure proposed by Rougé and Tilmant (2016) instead of directly using the benefit-to-go functions extracted after the buffer period. It must also be stressed that the proposed optimization model assumes that the river basin is managed by a single entity, which is the case in the Senegal. In transboundary river basins where countries do not cooperate, the model could be implemented sequentially, starting with the upstream country and imposing the resulting outflow to the downstream country and so on (Jeuland & Whittington, 2014). In this study, we also assume that the climate state is the same for the whole basin. Extending the model to river basins with asynchronous climate states is straightforward: sub-basin specific climate states must be included in the state vector. Note that this will automatically increase the computation time as all combinations of climate states must be investigated. It must also be stressed that the proposed optimization model assumes that the operator is able to assess the climate state at the beginning of each month. This 1-month perfect foresight is considered here as a reasonable assumption since it can be adjusted (together with the corresponding allocation decisions) as time goes by and the operators know more about the actual climate state during that month. Moreover, even though the climate state is hidden, in some cases, it can be partially revealed through the use of other hydroclimatic information such as sea surface temperature (SST) or soil moisture (Gelati et al., 2010). In the Senegal River basin, for example, there is a relationship between seasonal flows and SST in the Gulf of Guinea (Gu & Adler, 2004).

Acknowledgments

This project was supported by the Natural Sciences and Engineering Research Council of Canada Discovery Grant of the second author and the GoNEXUS project, which is funded by the European Union Horizon Programme call H2020-LC-CLA-2018-2019-2020 - Grant Agreement Number 101003722. The participation of Canadian researchers to GoNEXUS is made possible by the New Frontiers Research Fund program (Canada) (grant NFRFG-2020-00430) and by the Fonds Nouvelles Frontieres en Recherche (Quebec) (grant 2022-FNFR-310131). Figure 6 is reprinted from Journal of Hydrology, 587, Tilmant et al., Probabilistic trade-off assessment between competing and vulnerable water users - The case of the Senegal River basin, Copyright (2020), with permission from Elsevier.

Appendix A: Estimating Cuts' Parameters in SDDP-HMM

A key step of the Stochastic Dual Dynamic Programming (SDDP) algorithm is the derivation of the cuts' parameters. In the SDDP version with hidden climate states, the lth cut approximating the benefit-to-go F_t,j when the climate is in state j has the following form:

(A1)

where , , and are cuts' parameters when system is in climate state j. These parameters can be analytically derived from the primal and dual information that becomes available as the algorithm progresses backward. The parameters of the cuts can be derived from the Karush-Kuhn-Tucker (KKT) conditions (Kemp & Kimura, 1978):

(A2)

where λ is the dual information of the optimization problem, x denotes a primal variable, and g_i represents the ith linear constraint. Rewriting KKT for our reservoir optimization problem yields:

(A3)

where λ_w is dual variable associated with the mass balance Equation 3, λ_c is dual variable associated with the cuts Equation 9, and λ_hp is dual variable associated with convex hull approximation of the hydropower production function Equation 8. Cut parameters can be calculated by taking the partial derivative with respect to the corresponding state variable.

The vector of slopes with respect to the storage variable s_t is given by:

(A4)

Since the one-stage SDDP optimization problem is evaluated for K inflows scenarios (branches, backward openings) to capture the hydrologic uncertainty, the individually-calculated cut parameters must be aggregated so that the expected benefit-to-go is approximated (Figure A1). For the jth climate state, the vector of expected slopes is given by

(A5)

where K is the number of backward openings, that is, inflow branches generated by the built-in hydrologic model as the algorithm progresses backward.

Figure A1

Illustration of the backwards openings and the aggregation of the cuts when the climate is switching between 3 different states (e.g., dry (d), normal (n), and wet (w)).

Similarly, the vector of slopes can be obtained from the dual and primal information associated with mass balance equalities and cuts inequalities:

(A6)

Again, the aggregation is carried out by taking the expectation over the K artificially generated flows:

(A7)

Finally, the constant term of the cut can be calculated with:

(A8)

where and are the sampled storages and inflows respectively.

Open Research

Data Availability Statement

All the data used in this study are from the Senegal River Basin Authority (OMVS) and were collected during a project funded by UN-FAO (TCP/INT/3602). Because the model contains sensitive information on existing and planned hydropower plants that is protected by a nondisclosure agreement with UN-FAO, it cannot be made public. These data can be requested by contacting OMVS (www.omvs.org, T: [phone], E: [email protected]).