Precise Temporal Disaggregation Preserving Marginals and Correlations (DiPMaC) for Stationary and Nonstationary Processes

Abstract

Hydroclimatic variables such as precipitation and temperature are often measured or simulated by climate models at coarser spatiotemporal scales than those needed for operational purposes. This has motivated more than half a century of research in developing disaggregation methods that break down coarse‐scale time series into finer scales, with two primary objectives: (a) reproducing the statistical properties of the fine‐scale process and (b) preserving the original coarse‐scale data. Existing methods either preserve a limited number of statistical moments at the fine scale, which is often insufficient and can lead to an unrepresentative approximation of the actual marginal distribution, or are based on a limited number of a priori distributional assumptions, for example, lognormal. Additionally, they are not able to account for potential nonstationarity in the underlying fine‐scale process. Here we introduce a novel disaggregation method, named Disaggregation Preserving Marginals and Correlations (DiPMaC), that is able to disaggregate a coarse‐scale time series to any finer scale, while reproducing the probability distribution and the linear correlation structure of the fine‐scale process. DiPMaC is also generalized for arbitrary nonstationary scenarios to reproduce time varying marginals. Additionally, we introduce a computationally efficient algorithm, based on Bernoulli trials, to optimize the disaggregation procedure and guarantee preservation of the coarse‐scale values. We focus on temporal disaggregation and demonstrate the method by disaggregating monthly precipitation to hourly, and time series with trends (e.g., climate model projections), while we show its potential to disaggregate based on general nonstationary scenarios. The example applications demonstrate the performance and robustness of DiPMaC.

1 Half Century of Disaggregation

Half a century has passed since the first disaggregation attempt by Harms and Campbell (1967) for simulation of the river Nile flow. Their early approach was soon followed by the milestone papers of Valencia and Schaake (1972, 1973), which presented a general case for the model proposed by Matalas (1967) as a complete methodological scheme, the so‐called disaggregation model. The whole framework was based upon breaking down a sequence of coarser time scale (e.g., monthly) values into a sequence of finer time scale (e.g., daily) values, generating synthetic time series that reproduce the statistical characteristics of the observed records (or model outputs), both at the time scale of simulation and across coarser scales. Several modifications were proposed in the following years; for instance, Mejia and Rousselle (1976) included linkages with consecutive periods at the different levels of aggregation, while the scheme was also used in a multivariate application that preserved cross‐correlation between different time series (Tao & Delleur, 1976). Loucks et al. (1981) suggested implementing disaggregation in stages by first disaggregating the variables into individual sites, and then individual series were further disaggregated into their seasonal values. Stedinger and Vogel (1984) used a first‐order autoregressive model to introduce short‐term linear dependence and reproduce more successfully the variance and covariance properties of the disaggregated time series, but they also showed that the structure of these disaggregation models places severe constraints on the reproduction of the exact historical serial correlations at the scale of simulation.

Furthermore, these methods did not preserve higher order moments in time series with non‐Gaussian marginal distributions or the actual marginal itself. Therefore, Valencia and Schaake (1973) underlined the necessity to use some suitable transformation of the time series into Gaussian, before applying a disaggregation scheme. Typical transformations are the Box‐Cox transform (Box & Cox, 1964; Tao & Delleur, 1976) and the logarithmic transform (e.g., Salas, 1980). At the same time efforts were undertaken to explicitly preserve the skewness of the seasonal distributions of streamflow (Hoshi & Burges, 1979; Srikanthan, 1978; Todini, 1980), which, however, did not maintain the additive property of the original aggregation scheme (Salas et al., 1985).

Multisite, multiseason disaggregation models can have a large number of parameters because of the many cross correlations that they attempt to reproduce (Grygier & Stedinger, 1988). This led to developing staged disaggregation procedures that disaggregate annual flows at one or more sites to seasonal flows in two or more steps, and to the condensed disaggregation models, which reduce the number of required parameters by explicitly modeling only a selected set of correlations among the seasonal flows (Grygier & Stedinger, 1988; Lane, 1979; Lane & Frevert, 1988; Oliveira et al., 1988; Pereira et al., 1984; Salas, 1980; Stedinger et al., 1985; Stedinger & Vogel, 1984).

In the same period, global circulation models (GCMs) started to offer the first simulations of the global climate and soon the question was risen (see Maraun & Widmann, 2018, for a historical review) whether they can be used to describe climatic features at finer scales and how this could be achieved (Kim et al., 1984). To answer this question many different directions were followed, namely, classified as empirical, semiempirical, and model‐based (Giorgi & Mearns, 1991). The most popular semiempirical schemes were based on regression from the coarser to the finer scales (Hay et al., 1991; Karl et al., 1990; Kim et al., 1984; von Storch et al., 1993; Wigley et al., 1990). These schemes use other atmospheric circulation predictor variables at the regional scale, based on the assumptions that relationships can be established between atmospheric processes occurring at different spatiotemporal scales and that these relationships remain constant in time (Perica & Foufoula‐Georgiou, 1996; Wilby & Wigley, 1997). Disaggregation methods are another important manifestation of wider efforts to scale down GCM outputs for impact studies (e.g., downscaling of climate change scenarios for hydrological applications).

We note that although both disaggregation and downscaling models refer to transferring information from coarser to finer scales, the term “downscaling” has been used mainly in relation to climatic models, for example, spatial downscaling of GCMs, while it is more general as it includes not only statistical approaches like analog methods (e.g., Zorita & von Storch, 1999) but also dynamical methods which are based on physical meteorological processes (e.g., Wood et al., 2004; Xue et al., 2014). On the other hand, the term “disaggregation” has mainly been used in the hydrological literature and refers to statistical methods that most commonly have been applied in the temporal disaggregation of rainfall (e.g., Glasbey et al., 1995). Additionally, most disaggregation schemes require to produce a finer scale time series that adds up to the given coarse‐scale total (Lombardo et al., 2012). In these parallel paths between hydrological and climatological studies, it soon became apparent that precipitation is probably the most challenging process to efficiently disaggregate/downscale (e.g., Wilby & Wigley, 1997).

One of the most promising approaches to disaggregate precipitation has been based on the Bartlett‐Lewis rectangular pulse model (Rodriguez‐Iturbe et al., 1987), as applied by Glasbey et al. (1995), which had led to many subsequent models (e.g., Connolly et al., 1998; Cowpertwait et al., 1996; Kossieris et al., 2018; Koutsoyiannis & Onof, 2001). Other similar methods disaggregate daily precipitation into individual storms (Hershenhorn & Woolhiser, 1987), or used a geometric distribution to model rainfall occurrence and a beta distribution to simulate dimensionless accumulated hourly amounts aiming to preserve the first three moments (Kottegoda et al., 2003). Note also that using linear models to preserve moments results in nonrealistic correlation structures (Tsoukalas et al., 2018). Multifractal simulation schemes based on multiplicative cascade processes (e.g., Gupta & Waymire, 1990; Lovejoy & Schertzer, 1990) are another common methodological approach to perform disaggregation of precipitation records (Deidda, 2000; Molnar & Burlando, 2005; Olsson, 1998; Perica & Foufoula‐Georgiou, 1996; Serinaldi, 2010; Venugopal et al., 1999), while a theoretically consistent stochastic cascade focusing on temporal disaggregation of intermittent rainfall reproducing also long‐term persistence with Lognormal marginals was presented by Lombardo et al. (2017). Interestingly, the study of Perica and Foufoula‐Georgiou (1996) on spatial disaggregation of rainfall using the convective available potential energy as a predictor was one of the first cases where hydrological and climatological methodologies converged, mainly because climate studies were also searching for the most suitable predictors for precipitation downscaling, as highlighted by Wilby and Wigley (2000) and references therein.

During the last decade the strength of the application of statistical methods to climate model simulations has been acknowledged (see, e.g., Maraun et al., 2010, and references therein). Notable references include the generalized linear and additive models (e.g., Dobson & Barnett, 2008), chaotic disaggregation models (Sivakumar et al., 2001), support vector machines (Tripathi et al., 2006), vector generalized linear/additive models (Yee & Stephenson, 2007), quantile mapping (Maraun, 2013), and nonparametric techniques (Prairie et al., 2007) based on bootstrap k‐nearest neighbor resampling algorithms (Lall & Sharma, 1996). Data‐driven modeling techniques based on bootstrap have been widely applied (Lu & Qin, 2014; Nowak et al., 2010; Portela & Silva, 2016; Pui et al., 2012; Westra, Mehrotra et al., 2012), although they exhibit potential limitations, such as the lack of extrapolation beyond the observed records (Beersma & Buishand, 2003) or a poor exploration (and reproduction) of the state space due to insufficient sample sizes (especially in the case of stratified bootstrap). However, while the former shortcoming can be corrected by using for instance information from nearby locations (e.g., Westra, Mehrotra et al., 2012), the latter can be overcome only if longer sequences of records are made available. Pui et al. (2012) provided a comparison of some of the above disaggregation methods (i.e., random multiplicative cascades, point processes, and resampling).

Fifty years after the first disaggregation methodological attempts, both hydrological and climatic communities have a broad range of different methodologies to transform time series into finer scales. Current disaggregation methods in hydrology typically provide insufficient approximations of the fine‐scale process or ad hoc solutions. On the other hand, GCMs seldom capture climatic variability at all temporal and/or spatial scales (see, e.g., Gehne et al., 2016; Karl et al., 1990; Kundzewicz & Stakhiv, 2010; Svoboda et al., 2017). The biggest discrepancies among GCM model outputs seem to appear at finer spatiotemporal scales relevant to hydrologic applications (Hostetler, 1994). Our methodology aims to provide a potential solution to this issue by presenting a precise disaggregation at any timescale under stationary and nonstationary conditions. The method goes beyond attempting to preserve a limited number of statistical moments and correlations, which leads to unreliable approximations of the actual probability distribution and of the correlation structure. In contrast, it enables, using a unified stochastic modeling scheme (Papalexiou, 2018), to disaggregate coarse‐scale time series at any finer time scale by preserving the actual marginal distribution and the actual correlation structure of the fine‐scale process.

2 Making Disaggregation Precise

2.1 Definition of Disaggregation

Hydroclimatic processes at all subannual timescales are commonly modeled by cyclostationary stochastic processes (if other forms of nonstationarity are not assumed) given that their statistical and stochastic properties (partly or all of them) vary cyclically over the year. Thus, a cyclostationary process can be decomposed into a set of stationary processes each one of which is valid for a specific subperiod of the year, for example, a month, a day, or even an hour. Let {X_s(t)| t ∈ T, s = 1, …, N} be a cyclostationary process with T denoting an indexed set and s a cyclically varying index over t indicating the sth stationary process (hereafter {X_s(t)} denotes the stationary process at subperiod s). For example, a process at daily scale, presumed stationary at each month, will have N = 12 stationary processes with {X₁(t)} describing the first 31 days of the year, {X₂(t)} the next 28 or 29 days, and so on. Also, each {X_s(t)} process is described by a marginal distribution function F_Xs(x) (continuous or mixed type in the case of intermittent processes) and an autocorrelation structure (ACS) .

Let us assume a cyclostationary process at timescale k₀ (basic or target timescale) decomposed into N stationary processes, with period n_p, that is, X_s(t) = X_s(t + n_p) and with the sth process valid for a subperiod of n_s values, thus, . Clearly, n_p depends on the k₀ units and the period, while n_s on the subperiod where the sth process is valid, for example, in an annual cycle with presumed monthly stationarity and with k₀ = 1 hour, n_p = 24 × 365 (or 366 for leap years) and n_s = 24 × D_s where D_s is the number of days of the sth month. We denote the nonoverlapping aggregated (or averaged) process at timescale k = m × k₀ (m is any positive integer larger than 1) as { } and we define its jth aggregated random variable (r.v.) as

(1)

where ; for notational simplicity we omit the k₀ indicator hereafter. Let

(2)

be an observed time series of the process { } at the coarse time scale k, and let

(3)

be a random block of k values (k‐block hereafter) at a target fine time scale k₀. Disaggregation aims to find a time series

(4)

which, ideally, reproduces precisely the stochastic properties of the fine‐scale process {X_s(t)} by preserving the actual marginal distributions and the ACSs in all subperiods s, and additionally, the k‐blocks are constrained by .

The disaggregation scheme we propose comprises two parts: (a) the fit and use of a precise stochastic model (the disaggregation kernel hereafter) capable of reproducing desired marginal distributions and linear correlation structures at the target timescale k₀ (Papalexiou, 2018) and (b) an optimization part, based on Bernoulli trials, which uses generated random k‐blocks from the kernels in order to find those that match the imposed constraints, that is, make the disaggregated time series consistent with the coarse‐scale totals (or mean values alternatively). Consequently, this framework is one‐step method disaggregating the coarse‐scale values directly to the target time scale values avoiding thus cascading techniques. We term this method DiPMaC (Disaggregation Preserving Marginals and Correlations).

We stress that for notational clarity, in the subsequent sections, whenever possible and without risking ambiguity, we avoid using the subperiod index s or the time index j or t, for example, a single value from the coarse time series could be denoted as x^(k) instead of .

2.2 Disaggregation Kernels

Precise stochastic models, used as disaggregation kernels, can be constructed based on the premise that an arbitrary process {X(t)} with any prescribed marginal distribution F_X(x) having any valid (positive definite) linear ACS ρ_X(τ) can emerge by transforming a parent‐Gaussian process {Z(t)} having a standard Gaussian marginal Φ_Z(z) and an ACS ρ_Z(τ) that needs to be specified. In general, any nonlinear transformation g(Z(t)) : R → R applied to a Gaussian process modifies not only the marginal distribution but also the ACS ρ_Z(τ) by causing a decrease in its strength, which depends on the mathematical from of the transformation; that is, for an arbitrary bivariate normally distributed vector (Z(t), Z(t − τ)) it is proven that Cor(g(Z(t)), g(Z(t − τ))) ≤ Cor(Z(t), Z(t − τ)) (Kendall & Stuart, 1979, p. 600). It is well known that the marginal‐back transformation g(Z) ≔ Q_X(Φ_Z(Z)), where Q_X is quantile function of F_X(x), transforms the Gaussian r.v. Z into the r.v. X. Thus, simulating the process {X(t)} only requires the assessment of an autocorrelation transformation function such ; takes as argument the target ACS and returns the inflated ACS of the Gaussian process.

For a unified account of this framework that enables fast and easy estimation of the parent‐Gaussian process based on simple analytical correlation transformation functions and for applications in hydroclimatic processes see Papalexiou (2018) and references therein. However, to make this paper standalone, we provide a brief summary and an explanatory application of the procedure for fitting a stochastic model that generates time series with any marginal distribution (including mixed‐type marginals to account for intermittency) and any valid (positive definite) linear correlation structure. Particularly:

1. We assess the marginal distributions and the ACSs of each of the N stationary target processes {X_s(t)} at the time scale k₀. This can be done using observed data at the target fine scale k₀. If data are not available at the location of interest, the distribution and the ACS can be assessed based on regional information of near stations that show similar statistical behavior at the coarse time scale, or, by using scaling laws to infer the statistical properties of the process at the target scale k₀.
2. For each target {X_s(t)} process we estimate the autocorrelation transformation function

(5)

which creates a link between autocorrelation values of the target and the parent Gaussian process; for technical details on how this function is fitted see Papalexiou (2018).

3. The ACS of the parent‐Gaussian processes {Z_s(t)} is now simply estimated by and the process is simulated using autoregressive models AR(p) of sufficiently large order p to reproduce the .
4. The generated Gaussian time series are transformed into the target time series by applying the corresponding marginal‐back transformations, that is, , where denote the quantile functions estimated from step (a).

We illustrate the previous steps with an example. Let us target to generate synthetic time series from an intermittent process (e.g., it could be precipitation) with probability zero p₀ = 0.90 and nonzero values following a Pareto II distribution with (β, γ) = (8,0.2) (Figure 1a), and ACS described by the Weibull parametric form ρ_X(τ) = ρ_W(τ) =  exp (−(τ/b)^c) with (b, c) = (5,0.7) (Figure 1c, solid grey line). Estimation of the autocorrelation transformation function (Figure 1b, red line) allows the assessment of the parent Gaussian ACS, that is, (Figure 1c, dashed grey line). Then, an AR(20) model is used to reproduce inadequately the ρ_Z(τ) (for τ > 20 the ρ_Z ≈ 0) and generate Gaussian time series (Figure 1d). Finally, the Gaussian time series are transformed to the target‐variable times series (Figure 1e) by applying the marginal‐back transformation using of course the mixed‐type quantile in this case, that is, for (Q_Z is the quantile function of the standard Gaussian variable) x = 0, while for , . For verification we show the empirical distribution of the synthetic time series (Figure 1a, green dots), as well as the empirical ACSs of the Gaussian (Figure 1c, blue dots) and synthetic time series (Figure 1c, red dots).

Figure 1

(a) The target marginal distribution for nonzero values compared with the empirical distribution of the target simulation, (b) the fitted autocorrelation transformation function, (c) the target ACS ρ_X(τ) and the inflated parent Gaussian ρ_Z(τ) compared with the corresponding empirical ones from the simulations, (d) simulation of Gaussian time series, and (e) simulation of the target variable.

2.3 A Bernoulli‐Trial Framework for Efficient Optimization

Previously, we described how to generate random sequences (time series) with any given linear ACS and marginal distribution. Now if we target to disaggregate a value x^(k) (target value hereafter; subperiod and time indicators s and j are omitted for notational simplicity) into a k‐block then by definition we desire for totals, or for the average process. Clearly, a generated random k‐block cannot sum (or average) exactly to the target value x^(k) by chance. Particularly, let denote a “candidate” random k‐block with having a distance error from the target value x^(k) equal to |. Now we aim to find a k‐block with ϵ ≤ p_ϵ ∣ x^(k)∣, where p_ϵ is a small value (essentially p_ϵ is the relative error as ). Essentially, the optimization part aims to find a specific k‐block, which has a small relative error, that is, sums close enough to the target value. Although an analytical solution does not exist to specify the k‐block, here, we show that within a Bernoulli‐trial framework we can form a precise procedure that enables to find a k‐block having any prescribed error ϵ, that is, to sum (or average) as close as we wish to the target value. Here we use two common optimization methods:

• Method 1:

  A sufficiently large number ν of random k‐blocks is generated, and the one with the smallest ϵ is selected. Particularly, we generate ν random k‐blocks, estimate their totals (or average values), calculate their error from the target value, and end up with a list of errors {ϵ₁, …, ϵ_ν}. The k‐block corresponding to the smallest error among the {ϵ₁, …, ϵ_ν} is selected.

• Method 2:

  Iterate until the error goal is fulfilled by allowing a maximum number of ν iterations. Particularly, a first random k‐block is generated and its error ϵ₁ from the target value estimated. If the error goal is fulfilled, that is, ϵ₁ ≤ p_ϵ ∣ x^(k)∣, then this random k‐block is selected; if not, a second k‐block is generated and its error ϵ₂ is compared with the error goal, and so on; that is, the procedure is repeated until a k‐block is found to fulfill the error goal. To avoid infinite iterations, a “global” maximum number of iterations ν must be set. Thus, this procedure will create a list of errors {ϵ₁, …, ϵ_i} with i ≤ ν as the error goal is fulfilled in general before the maximum number of iterations is reached; however, if the maximum number of iterations is reached, then the k‐block corresponding to the smallest error among the {ϵ₁, …, ϵ_ν} is selected.

A major difference between these two approaches is that in Method 1 exactly ν k‐blocks are generated and among them it is likely to observe more than one block fulfilling the error goal; in contrast, in Method 2 only one k‐block (or zero if ν iterations are reached) fulfills the error goal (that is by the definition of the method). This results the selected k‐block from Method 1 to have typically smaller error ϵ compared to the one from Method 2, as in the former case the best k‐block is chosen (typically) among more than one block that fulfill the error goal, or else, in Method 1 the selected k‐block is chosen from a larger pool of blocks than in Method 2, and this increases the chances to find one with even smaller error (this is further shown in section 2.4 and in Figures 3e and 3f). Note that a crucial aspect of the operational use of these methods is their implementation speed. The current technology of parallel processing or graphics processing unit programming assures that Method 1 can also be fast as it can be fully “parallelized,” that is, in an n‐core system the algorithm is n times faster compared to a single‐core system. Method 2 has a sequential structure (iterations until the error goal is achieved), and thus, it is not certain that it can be implemented exploiting fully the benefits of parallel computing.

Clearly, the goal in both methods is to specify the number ν. If a target value x^(k) is feasible to emerge by summing a random k‐block, then the larger the number of generated blocks implies a larger probability to find a k‐block with a smaller error ϵ, that is, lim_ν → ∞ min {ϵ₁, …, ϵ_ν} = 0, or else, the minimum error ϵ will converge to 0 (see Figure 2a). For operational use, however, it is impossible to generate an infinite number of random k‐blocks (or for Method 2 we need to set a maximum number of iterations ν to avoid infinite or time‐consuming loops), and thus, it is mandatory to assess the number of k‐blocks or iterations needed to achieve the prescribed error with a given certainty. Clearly, a smaller p_ϵ demands (on average) a larger ν, which however, is affected by the target value x ^(k) itself, for example, if x ^(k) belongs in an interval around the mode of the r.v. X ^(k) then the min{ϵ₁, …, ϵ_ν} converges faster to 0 as ν increases (Figure 2a, blue line), while if x ^(k) is an extreme value, then the convergence is much slower (Figure 2a, red line). Let as assume that if , which implies that any value has error fulfilling thus the error goal; now, given the distribution function of X^(k) the probability to observe a value is

(6)

Figure 2

(a) The minimum relative error value p_ϵ estimated from sets of ν generated k‐blocks showing its convergence to zero as ν increases; also, the p_ϵ converges faster to zero (blue line) for a target value with exceedance probably 0.5 (more probable) and much slower (red line) for one with 0.001 (extreme value); (b) a benchmark hypothetical hourly precipitation (PRCP) time series, and (c) the corresponding aggregated time series at the monthly time scale; (d) probabilities of the r.v. X^(k) to belong in the interval [(1 − p_ϵ)x^(k), (1 + p_ϵ)x^(k)] with p_ϵ = 5% for the the monthly values x^(k) shown in panel c, (e) minimum number ν of k‐blocks or iterations needed to assure with probability 99% that ϵ ≤ p_ϵ ·  ∣ x^(k)∣for p_ϵ = 5%.

In terms of Bernoulli trials, we consider “success” X^(k) ∈ A with probability u, and “failure” X^(k) ∉ A with probability 1 − u, and therefore, the probability to have at least one success in ν independent trials is P = 1 − (1 − u)^ν. In the disaggregation scheme a candidate random k‐block is generated, and then, summed (or averaged) to in order to be compared with the target value x^(k). Thus, if we aim for an error ϵ ≤ p_ϵ ∣ x^(k)| with 100P% confidence (certainty), we have to generate at least a number of k‐blocks equal to

(7)

with u given from equation 6.

Accurate estimation of u needs the distribution function of the r.v. . We can simply estimate by sampling a large number of k‐blocks from the disaggregation kernel and by summing (or averaging) them to obtain values; then it is straightforward to fit an appropriate parametric distribution to the sample of values. For example, assuming a hypothetical hourly precipitation time series (Figure 2b) and its monthly (k = 30 · 24 hr) aggregated counterpart (Figure 2c), we fit the with the aforementioned method and we estimate the corresponding probabilities u for each monthly value and for p_ϵ = 5% (Figure 2d); finally, we estimate the minimum number of ν blocks or iterations needed to achieve the error goal with 99% confidence (Figure 2e). Of course, an extremely large ν would cause a significant delay; thus, a global maximum ν_max should always be set; for example, using the estimated ν values we can a set a global maximum so 99% of ν values are less than it (in this example ν_max = 361).

2.4 Effects of the Correction Factor

Previously, we showed that if we target to disaggregate a value x^(k) into a k‐block, we can find a candidate k‐block (emerging from a process with desired ACS and marginal distribution), which sums to a value close to x^(k), that is, . The Bernoulli framework we introduced enables to calculate the number ν of k‐blocks we need to generate (or the number of iterations) in order to find one (with given confidence) that has any prescribed error . Clearly, it is not optimal to generate an extremely large number of k‐blocks in order to find the one that sums extremely close to x^(k), yet we can find one with an acceptable error, and if we wish, we can eliminate this error ϵ simply by multiplying the values of the selected k‐block by the correction factor . Now an important question is if this correction is meaningful; we deem that essentially it is not if the prescribed error is small. For example, disaggregating with a small relative error p_ϵ = 5%, it is hard to argue on what practical difference it makes, for example, if a generated hourly precipitation block, aiming to disaggregate a monthly value of 10 mm, totals somewhere in the range 10 ± 0.5 mm (measurement errors could be even larger than this). However, since it has been a common practice to correct the disaggregated values to match exactly the target value, probably because the computing power in the past did not allow to find blocks with small error, we provide here a framework to assess the effects of this correction in the distributional properties of the disaggregated time series.

Particularly, rescaling an r.v. X by multiplying it by the numerical value c_ϵ affects its properties, for example, the r.v. c_ϵ X has mean and standard deviation c_ϵ μ_X and c_ϵσ_X, respectively (where μ_X and σ_X denote the mean and standard deviation of the r.v. X). In Figures 3a and 3b we show time series generated by disaggregating the monthly totals in Figure 2c using the two optimization methods, while Figures 3c and 3d shows the correction factors for each k‐block disaggregating the monthly values. The resulting c_ϵ factors exhibit bell‐shaped and uniform distributions for Methods 1 and 2, respectively, which also verifies (as aforementioned in the previous section) that the error from method 1 is typically smaller than that of Method 2 (see histograms of Figures 3e and 3f). This is expected as in Method 2 the first k‐block with error ϵ ≤ p_ϵ ∣x^(k)∣ ends the optimization while in Method 1 typically more than one k‐block fulfills the error goal and the best of them is selected. Now given that each k‐block is rescaled with a different c_ϵ we can investigate the effects of this correction by assuming that c_ϵ is an r.v. itself; thus, the r.v. describing the final disaggregated time series could be expressed by the r.v. W = C_ϵX. We can show that the error correction factor c_ϵ, given that ϵ ≤ p_ϵ · ∣ x^(k)∣, is confined in [(1 + p_ϵ)⁻¹, (1 − p_ϵ)⁻¹], while to ease the mathematical analysis we investigate the second case (Method 2) where we can assume that C_ϵ is distributed uniformly within this range (see Figure 3f), that is, . Now if f_X (x) and are the pdf's of X and C_ϵ, respectively, and X and C_ϵ are assumed to be independent, then the pdf of W is given by

(8)

Figure 3

(a and b) Hourly time series generated by disaggregating the monthly totals shown in Figure 2c; (c and d) error correction factors c_ϵ for each hourly block disaggregating the corresponding monthly values; (e and f) corresponding histograms of the c_ϵ values shown in panels c and d. Method 1 disaggregates a target value by generating ν k‐blocks and choosing the one with the smaller error, while Method 2 is a trial‐and‐error iteration framework seeking the first block to fulfill the error goal and allowing a maximum of ν iterations.

This enables us to compare the r.v.’s X and W, quantify the effect of this correction, and finally assess the relative error p_ϵ value that does not modify the properties of X.

In order to demonstrate this framework, we use a versatile three‐parameter distribution (any other distribution can be used), which we apply in all case studies, that is, the generalized gamma ( ) distribution (Papalexiou & Koutsoyiannis, 2012, 2016; Stacy, 1962) with pdf

(9)

where β > 0 is a scale parameter, γ₁ > 0 and γ₂ > 0 are shape parameters, and Γ(·) is the gamma function. For γ₁ < 1 and γ₁ > 1 the pdf is J‐ and bell‐shaped, respectively, while the parameter γ₂ adjusts mainly the “heaviness” of the right tail, and thus, the behavior of extreme events. Applying equation 9 for and we get

(10)

with . For example, for using equation 10 we can plot and compare the pdf's for various values of p_ϵ, that is, p_ϵ = {5,  10,  20,  and 30%}. We see (Figure 4a) that the pdf's f_W(w) are almost identical with f_X(x) and the difference is only revealed if we plot the distribution function difference ΔF ≔ F_X(x) − F_W(w) (Figure 4b), which clearly indicates very small deviations even for p_ϵ = 10%; for p_ϵ = 5%, the difference is essentially zero. Of course, different distributions may show larger deviation for the same p_ϵ values; for example, selecting a bell‐shaped we observe (Figure 4c) more apparent differences in the pdf's, yet ΔF is still very small for p_ϵ = 10% and essentially zero for p_ϵ = 5% (Figure 4d). Also, note that ΔF is not constant in the whole r.v. range but peaks at some value close to the mean of X and vanishes for large values. As already mentioned, the choice of the pdf f_X(x), as well as the values of its parameters, can affect the difference ΔF, yet it is rational to expect that an a priori selected small p_ϵ like 5% is a safe choice. Of course, what is considered a safe choice is subjective as it also depends on the ΔF we are willing to accept.

Figure 4

Comparison of the r.v.'s X and W = C_ϵ · X for various relative error values p_ϵ in terms of pdf's and cdf differences for: (a and b) a J‐shaped generalized gamma and (c and d) a bell‐shaped generalized gamma.

Finally, we note that the previous analysis cannot be easily performed assuming a normal distribution for c_ϵ (Method 1), as the corresponding product r.v. W = C_ϵX does not have an analytical form. Yet it should be clear that ΔF is expected to be even smaller in this case, as there are more values close to 1 (which implies less disturbance; Figure 3e) and less values in the edges of the [(1 + p_ϵ)⁻¹, (1 − p_ϵ)⁻¹] range.

2.5 Blueprint for Stationary and Nonstationary Disaggregation

Summarizing and linking the methods described in the previous sections, we provide here a complete operational scheme, in order to disaggregate a given coarse‐scale time series { } into a fine‐scale time series {x_s(t)} generated from a cyclostationary process {X_s(t)} with prescribed stochastic characteristics. Particularly:

1. Define (assess) the marginal distributions F_Xs(x) and the ACSs ρ_Xs(τ) at the target fine time scale, and the corresponding parent‐Gaussian ACSs ρ_Zs(τ). Then fit AR_s(p) models of sufficiently large order p that reproduce the ρ_Zs(τ) ACSs (see section 2.2).
2. Choose a desired relative error p_ϵ (e.g., 5%) and estimate for each target value the number ν of random k‐blocks (or iterations) that have to be generated in order to find at least one fulfilling the prescribed p_ϵ with given confidence P, for example, P = 99% (section 2.3).
3. For a target value generate ν random Gaussian blocks using the fitted AR_s(p) model.
4. The ν generated blocks are transformed into blocks by using the transformation .
5. The sum (or average) of each block is estimated, that is, , in order to calculate a list of ν errors | from the target value.
6. The block corresponding to the smallest error ϵ is selected.
7. The values of the selected block are multiplied by the error correction factor so that .
8. The next target value is disaggregated by repeating steps (3–7).

To provide some remarks for the application of the scheme, we note that first, if Method 2 is selected to disaggregate a target value , then steps (3–5) are modified accordingly; that is, a single is generated and transformed into ; then the is estimated, and if |, the process ends, otherwise is repeated for a maximum of ν iterations. In the case of reaching the maximum number of iterations ν and the error goal has not been achieved, then the block with the smallest error among the ν generated blocks is selected. Second, as the algorithm sequentially disaggregates the coarse scale values, it builds up in parallel two time series, that is, the disaggregated one {x_s(t)}, but also the {z(t)} comprising the corresponding Gaussian variable values, that is, those that were transformed to the final disaggregated values. This is necessary as the AR_s(p) model uses always as starting values previous values of the {z(t)} time series in order to preserve the correlation structure. Third, in the case of mixed‐type r.v.’s like precipitation it is obvious that a zero precipitation value at the time scale k is always disaggregated into a k‐block of zero values, then obviously step (7) is omitted as , and thus, the correction factor cannot be defined. Also, as previously noted, the use of the correction factor (step 7) is not strictly required and can be omitted if the prescribed error is negligible for all practical purposes.

Now the previously described scheme was based on the assumption that the disaggregation kernels are cyclostationary; that is, the stochastic process {X_s(t)} is the same as the {X_s(t + n_p)} process, where n_p denotes the period. However, it is straightforward to modify this assumption and form a fully nonstationary framework. Particularly, we can assume (or define) that any of the stationary processes at the target time scale are nonstationary; that is, their stochastic characteristics change in a predefined way, and thus, are functions of time; in this case {X_s(t)} ≠ {X_s(t + n_p)}. For example, we can define that statistical characteristics such as the mean, standard deviation, or shape measures change over time in a prescribed way, which in turn, allows the estimation of the marginal distributions F_Xs(x) at any t and consequently of the disaggregation kernels. We demonstrate the generality and the flexibility of this nonstationary framework is section 4.3.

As a general comment, we stress that this scheme is essentially parsimonious as the only true parameters we need to estimate are those that describe the process at the target fine scale, and that is, the parameters of the marginal distribution and of the ACS. For example, if one wishes to disaggregate monthly precipitation into hourly, assuming that in a given month the nonzero rainfall follows a Pareto II distribution (two parameters) and the ACS is described by a Weibull parametric form (two parameters), then the total number of parameters is four, plus one related to the estimation of probability dry. All the rest of the parameters used, for example, in the large order AR models, or in the Bernoulli framework, are not essentially parameters as they are deterministically defined (or entail) by the selection of the marginal and of the ACS. Clearly, these parameters should be estimated for any stationary period (e.g., month), and of course, additional parameters are needed when nonstationarity is assumed.

3 Stationary Disaggregation

3.1 Proof‐of‐Concept: Disaggregating Monthly Precipitation Into Hourly

As a proof‐of‐concept for illustrating the proposed method, we disaggregate monthly precipitation values into hourly. We use a randomly selected station (code: 3240_154202) from the hourly precipitation data set of National Climatic Data Center of the USA (see Hammer & Steurer, 1997) covering the period 1 February 1981 to 30 November 2011. As anticipated, the hourly precipitation time series and the corresponding monthly values exhibit different statistical properties (Figures 5a and 5b). The former is the result of an intermittent process with hourly probability dry ranging from 88 to 96% (depending on the month) and high positive skewness, while the latter can be characterized as continuous (all monthly values were found nonzero) with values tending towards normality due to the central limit theorem.

Figure 5

(a) Observed hourly precipitation, (b) the corresponding observed monthly (aggregated) time series, and (c) hourly precipitation time series created by disaggregating the monthly time series of panel b.

Following the steps described in section 2.5 we first assume hourly precipitation corresponding to each month as a stationary intermittent process; thus, for every month we estimate the probability dry p₀ and we fit the distribution (equation 9) to describe the nonzero values. Its mean , standard deviation , and coefficient of skewness are given, respectively, by

(11)

(12)

(13)

Note that maximum likelihood led to parameter estimates indicating unrealistically heavy tails and theoretical and values deviating significantly from the observed ones (that is probably due to the discretization errors of the hourly data especially for small values, i.e., the presence of the so‐called statistical ties). For this reason, we fitted the using the method of moments (equations 11–13), and thus, the emerging distributions preserve exactly these characteristics. Comparison of the empirical distributions with the fitted ones indicates an excellent agreement (see Figures 6a and 6b for a summer and winter month and Figure S1 for all months). Recalling that the parameter γ₂ controls the tail behavior, its range (0.41–0.92) indicates subexponential tails, and it is in agreement with an extensive analysis on hourly extremes based on more than 4,000 hourly precipitation records across the United States (Papalexiou et al., 2018).

Figure 6

(a and b) Empirical distributions (red dots) of observed nonzero hourly precipitation (Figure 5a) and the fitted generalized gamma ( ) distributions (solid line) for a summer and a winter month; the legend shows the probability dry p₀ and the estimated parameters. (c and d) Empirical autocorrelation structure (ACS) (red dots) of the observed hourly precipitation (Figure 5a), the fitted parametric Pareto II (PII) ACS (solid line), and the ACS of the parent‐Gaussian process (dashed line); the legend shows the estimated parameters of the fitted PII ACS.

Next we calculate the empirical autocorrelation values up to lag 15, as autocorrelation is essentially 0 for higher lags and for all months. Then, we fit a parametric power‐type ACS (see Papalexiou, 2018, for details on parametric ACSs), that is, the Pareto II, given by

(14)

where b > 0 and c > 0. This parametric ACS interpolates the empirical values almost perfectly in all months (see Figures 6c and 6d and S2 for all months). For demonstration, we depict also the parent‐Gaussian ACS (dashed line in Figures 6c and 6d and S2), which is highly inflated as expected.

Finally, we reproduce the parent‐Gaussian ACSs of each month by using an AR(p) model of order p = 72 (i.e., taking into account the previous 72 hr of precipitation) as the parent‐Gaussian ACSs of some months show positive correlations up to this lag. We stress again the parsimonious character of this model, as the p coefficients of the AR(p) model are analytically determined by the parent‐Gaussian ACS, which in turn emerges by the fitted two‐parameter PII ACS in this case; thus, the Gaussian process model is essentially a two‐parameter model. Subsequently, we disaggregate the monthly time series (Figure 5b) according to the steps in section 2.5 and we obtain the disaggregated time series (Figure 5c).

A first inspection of the observed (Figure 5a) and the disaggregated time series (Figure 5c) shows a remarkable similarity. Yet we assess in detail the statistical quality of the disaggregated time series by comparing the empirical distribution and the ACS of the synthetic time series with those of the target (observed) series. Referring to Figures 7a and 7b for two characteristic months and Figure S3 for all months, the empirical distributions (blue dots) of the disaggregated data match very well the target distributions (solid lines) that were fitted using the observed data. Of course, minor deviations on the tails are expected due to random variations and are not the result of a systematic error. Comparing the empirical ACSs (blue dots) with the target ACSs (solid line, Figures 7c and 7d and S4) also reveals an outstanding match with small deviation emerging by sampling fluctuations.

Figure 7

Verification of the statistical properties of the disaggregated hourly time series shown in Figure 5c. (a and b) Empirical distributions (blue dots) of disaggregated nonzero hourly precipitation in comparison with the target distribution (solid line) assessed from observed data (Figure 6a,b). (c and d) Empirical autocorrelation structure (ACS) of disaggregated hourly precipitation (blue dots) in comparison with the target ACS assessed from observed data (Figure 6c,d).

Finally, Figure 8 shows a side‐by‐side comparison of summary statistics between the observed and the aggregated time series for all months. Particularly, the probability dry, the mean, and the standard deviation are almost indistinguishable, while small deviations in skewness are due to sampling fluctuations. In fact, it is well known that sample estimates of skewness are not robust for samples drawn from highly skewed and subexponential distributions and in general should be avoided and alternatives like L‐moments (Hosking, 1990; Lombardo et al., 2014; Papalexiou & Koutsoyiannis, 2016; Peel et al., 2001) should be preferred.

Figure 8

Observed hourly time series (Figure 5a) versus disaggregated (Figure 5c). Monthly comparison of (a) probability dry, (b) mean value, (c) standard deviation, and (d) skewness.

4 Treating Nonstationarity

By definition a stationary process has statistical properties that remain time invariant. In contrast, nonstationarity implies changes in the marginal distribution attributes such as the mean value, scale (standard deviation), and shape characteristics (e.g., heavier tails), and in the joint distributional properties often expressed by the ACS. This time‐varying complexity creates difficulties in identifying the proper nonstationary model based on the limited information from the observed time series (see, e.g., Lins & Cohn, 2011; Serinaldi et al., 2018). Yet by necessity, nonstationary approaches (see, e.g., Cheng & AghaKouchak, 2014; Cheng et al., 2014; Obeysekera & Salas, 2016; Ragno et al., 2018; Read & Vogel, 2016) often grounded on a priori assumptions about the specific attributes exhibiting a trend (e.g., trend in the mean) and the form of the trend (e.g., a linear trend). Developing nonparametric or adaptive methodologies for nonstationary modeling remains a challenge, but it is outside the scope of the present paper.

4.1 Crash Test: Adaptation of Stationary Kernels to Non‐stationarity

In the previous section, we showed that the disaggregation scheme developed here can generate fine‐scale time series consistent with a priori known marginal distribution and ACS. Yet an important question is how this scheme adapts under nonstationary conditions, and particularly, when nonstationarity is assumed for the process at the coarse scale, which also implies nonstationarity for the target (fine) scale. The purpose of this crash test is to investigate the disaggregation of a nonstationary process at the coarse (monthly) scale but using stationary kernels which will generate time series at the finer (daily) scale while preserving the coarse‐scale totals. This approach could be of practical interest when potential nonstationarity is observed at the coarse scale, yet the exact form of nonstationarity at the fine‐scale cannot be robustly assessed, for example, due to insufficient information or lack of theoretical justification. Here we explore the most common case of nonstationarity consisting of a linear trend. Particularly, we assume that the monthly precipitation (Figure 5b) changes at a rate of 0.00039 mm/hr per month, that is, doubling its mean in 30 years (see Figure 9a). We disaggregate the trending time series by using the stationary disaggregation kernels described in section 3.1, that is, estimated from the observed hourly precipitation (Figure 5a).

Figure 9

(a) A modified monthly time series exhibiting a trend (red line); (b) hourly precipitation time series created by disaggregating the monthly time series in panel (a) and the mean value of the largest 2% of hourly values (red line).

The hourly disaggregated time series (Figure 9b) show the same pattern as the observed ones (Figure 5a). Still, a careful inspection shows an “intensifying” behavior over time, which is more apparent by depicting, for example, the monthly mean value of the largest 2% of hourly values (Figure 9b, red line). The estimated change, based on the fitted trend to the disaggregated values, is the same as in the monthly time series, that is, 0.14 mm/hr over a 30‐year period.

A more detailed comparison shows (a) a decrease in probability dry (Figure 10a), especially for nonsummer months, (b) an increase in the mean of nonzero precipitation (Figure 10b) especially in summer, (c) standard deviation increases (Figure 10c) mainly in summer, and (d) no change in shape (Figure 10d) as quantified by the skewness coefficient, implying that more extreme values result by location and scale changes. Of course, the statistics shown in Figure 10 for the nonstationary case express an average stage as in reality these statistics should be time varying. This adaptation to nonstationarity is rational and anticipated, because feeding systematically larger monthly values in the disaggregation algorithm forces the selected random block to have either more wet days, or, larger values, or, both, in order to fulfill the constraint and add up to the monthly values. Clearly, since there is no reason for the procedure to choose one of these two options, we observe changes in both. The reason we observe a different “adaptation” over the summer months relates probably to the disaggregation kernels used; that is, the target stochastic processes for the summer months have the less intense ACS (see Figure S2) and this implies less variability in terms of probability dry, which in turn forces the process to choose blocks with larger nonzero values.

Figure 10

Observed hourly time series (Figure 5a) versus disaggregated (Figure 9b) from the modified monthly time series (Figure 9a). Comparison of (a) probability dry, (b) mean value, (c) standard deviation, and (d) skewness. Note that statistics shown for the nonstationary case express an average state.

This crash test shows that although the disaggregation kernels are stationary, the optimization process creates a nonstationary process at the fine scale in order to respect the coarse‐scale nonstationarities. Of course, this approach does not provide full control on the statistical properties that become nonstationary at the fine scale. For example, in this crash test we observed changes mainly in probability dry and the mean, yet we may wish to generate time series from a process where the shape characteristics also change; for the latter case the scheme using nonstationary kernels should be chosen. Finally, we stress that when we use stationary kernels, their properties (assessed e.g., from a past stationary period) should be generally nonvalid in the nonstationary projection period as changes in the coarse scale should be reflected also in the fine scale. The resulting nonstationarity in some statistical properties is a “by‐product” due to the forcing constraints of the nonstationary coarse process. According to this argument, there is not a theoretical justification for the use of a kernel assessed by the historical marginals and ACS apart from the fact that it offers a rational starting point and probably a more robust option than “speculating” the future properties of a kernel without sufficient information. Clearly, any reasonable kernel parameterization, or even different parameterizations, to explore sensitivity and different assumptions can be used.

4.2 Disaggregating GCM Projections

It is well known that general circulation models (GCMs) present substantial issues in the representation of precipitation, such as overestimation of light rainfall, and underestimation of extreme precipitation (Stephens et al., 2010; Trenberth et al., 2003; Wilcox & Donner, 2007). These problems still persist today (see, e.g., Gehne et al., 2016; Herold et al., 2016; Langousis et al., 2016; Liersch et al., 2016; Svoboda et al., 2017) and even amplify as the resolution of the new generation of models increases (Trenberth et al., 2017). Since monthly GCM simulations are more reliable that daily (e.g., Chun et al., 2014), a common approach is to disaggregate the monthly, or even annual, time series to (sub) daily scales.

Here we use the CPC unified gauge‐based analysis of global daily precipitation data set (Xie et al., 2010), which is a gridded data set that has been utilized in numerous evaluation studies of remote sensing, reanalysis, and GCM data products (e.g., DeAngelis et al., 2013; Gehne et al., 2016; Sheffield et al., 2013; Yong et al., 2015). The CPC data set comprises approximately 17,000 stations covering the area defined from 89.75°N to 89.75°S and from 0.25°E to 359.75°E in 0.5° × 0.5° spatial resolution, and for the 1 January 1948 to 31 December 2006 period. The observed daily precipitation time series (Figure 11b) used here is from a 0.5° × 0.5° square region (41.0–41.5°N and 238.5–239.0°E) aggregated to the monthly scale for the sake of comparison (Figure 11b). We also used the BCSD‐CMIP5‐Climate‐monthly data product from the Downscaled CMIP3 and CMIP5 Hydrology Projections Version 3.0 (Reclamation, 2014; https://gdo‐dcp.ucllnl.org/downscaled_cmip_projections/). The data used are downscaled in space by the data provider using the “Bias‐Correction and Spatial Disaggregation (BCSD) technique” (Reclamation, 2013; Wood et al., 2002; Wood et al., 2004). Specifically, we use three GCM simulations of the CMIP5, that is, the monthly precipitation estimates under the rcp8.5 emission scenario from the CNRM‐CM5 (Figure 11c), GISS‐E2‐R, and IPSL‐CM5B‐LR models at a 0.625° × 0.625° grid for the period 1950–2100 CE. Note that the rcp8.5 refer to future projections from 2006 to 2100 and do not include the period 1950–2005. The time series used here (e.g., Figure 11c) were the merged data product of both historical and future simulations (downloaded from http://iridl.ldeo.columbia.edu).

Figure 11

(a) Observed monthly and (b) observed daily precipitation over a 0.5° × 0.5° grid in California; (c) monthly projection time series (including the historical 1950–2005 period) from a GCM model (CNRM‐CM5); (d) daily time series by disaggregating the monthly CNRM projection.

Following the method described in section 2, we estimate, on a monthly basis, the probability of dry days p₀ and we fit the Generalized Gamma ( ) distribution to the observed nonzero daily precipitation using the method of L‐moments (see Figure S5) assuming that the process is stationary. Subsequently, we use the Weibull parametric ACS, given by

(15)

with b > 0 and c > 0, to describe the empirical autocorrelation of daily precipitation and we estimate the parent‐Gaussian ACS (Figure S6) of each month, which can be reproduced with sufficient accuracy by an AR(p) of order p = 7 days. The disaggregation scheme was finally applied to the three monthly GCM projections. Comparing Figures 11b and 11d highlights the similarity between the daily disaggregated time series of the CRNM model and the observed daily series.

Of course, a more detailed comparison (see Figure S7) of the empirical distributions of nonzero disaggregated daily precipitation from the three GCM monthly projections with the target distribution reveals some differences for some months, yet this is exactly what we expect, and actually what we desire, as monthly GCM projections differ statistically from the observed monthly precipitation, and thus, the disaggregation scheme (as demonstrated in section 4) has to adapt. In any case, we stress the realistic results, which are also apparent when comparing the empirical ACSs of the disaggregated daily precipitation from three GCM monthly projections with target Weibull ACS assessed from the observed data (see Figure S8).

Similarly, a straightforward comparison of basic statistics (Figure 12), that is, probability of dry days, mean daily precipitation, second L‐moment, and L‐skewness between the observed daily time series versus the daily disaggregated, clearly shows their similarity, yet in some months differences are apparent. Of course, we stress again that if the GCM projections are nonstationary, then this should be reflected also at the fine‐scale process, and thus statistics shown in Figure 12 (or the box plots in Figure 13) express an average state. For example, the daily mean value of December for the CNRM projection (≅6 mm) is almost 50% higher than the observed (≅4 mm), or, higher L‐skewness (usually implying more extremes) is observed for August for the GISS projection. Finally, an side‐by‐side comparison among monthly (Figure 12a) and among daily values (Figure 12b) in terms of box plots (Figure 13, whiskers, indicate the 90% range of values) shows a “transfer” of the difference among the monthly values to daily disaggregated values. For example, the pattern of rank‐based indices such as medians or the interquartile range (as expression of variance) at the monthly scale is also “transferred” at the daily scale for almost all months. Yet it is interesting to note how this pattern is smoothed in many months due to contrasting effects. In fact, the disaggregation kernel drives the statistics toward the observed ones, while the constraints force the scheme to adapt in order to replicate the monthly differences.

Figure 12

Observed daily time series (Figure 11b) versus daily disaggregated from three monthly GCM projections. Comparison of (a) probability dry, (b) mean value, (c) second L‐moment, and (d) L‐skewness.

Figure 13

Box plots (whiskers indicate the 90% range of values) comparing (a) the observed monthly precipitation over a grid box in California with the projections of three GCM models, and (b) the observed with the disaggregated daily precipitation of the three monthly GCM projections.

Ideally, a climate model has to adequately simulate the process of interest at the timescale of interest (a complete discussion on statistical downscaling and bias correction in climate research can be found in Maraun and Widmann, 2018). Yet as climate models are still not able to resolve important fine‐scale features, disaggregation methods still offer a valuable alternative. As a general comment we stress that disaggregating climate model projections is by necessity based on specific assumptions about the processes at the fine scale. Clearly, any statistical parameter of the disaggregation model could be time variant or nonstationary introducing a large degree of complexity and uncertainty. There is evidence, for example, that the space‐time structure of mesoscale convective systems will be changing in the future (e.g., Lenderink & van Meijgaard, 2008; Westra, Alexander et al., 2012) altering not only marginal statistics but also intricate temporal dynamics. This imposes demands on the disaggregation models as to the imposed complexity of the nonstationary marginal and ACS. Here we do not claim that disaggregating a monthly GCM projection, for example, to hourly, results in a process that encapsulates precisely the future hourly characteristics. Rather, we show that it is technically feasible to disaggregate a projection based on the precise statistical properties of the past, or, as we show in the next section, to disaggregated by assuming general nonstationary scenarios.

4.3 Explicit Nonstationary Disaggregation

We demonstrate the generality and the flexibility of the proposed nonstationary framework by disaggregating the monthly precipitation of the CNRM projection (Figure 11c) using only the projected July values from 2020 to 2099 (80 years; see Figure 14b). Clearly, this framework can be applied likewise for the other months too. We stress that the purpose of this demonstration is to show that it is feasible to disaggregate based on general explicit nonstationary scenarios and not to assess future changes as this is out of the scope of this study. Particularly, based on the empirical (past) daily values of July (Figure 14a), we estimate (using the method of moments) the parameters of the distribution for the nonzero precipitation, that is, β = 0.5, γ₁ = 0.68, and γ₂ = 0.53 (Figure 14c, red line); the ACS (Figure S6); and the probability dry value p₀ = 0.84. Note that the fitted distribution has mean value  mm and standard deviation  mm matching of course the sample statistics of the observed nonzero values. Now let us assume (Scenario 1) that the mean value of the nonzero daily precipitation will increase linearly by 40% by the end of 2099, that is, from 1.24 to 1.74 mm, while the standard deviation, the ACS, the probability dry, and the parameter γ₂ that controls the behavior of the extremes will remain the same. Of course, this change in the statistical characteristics (mean in this case) is reflected in the parameters of the distribution; that is, we need a new parameter set to express the distribution of the nonzero daily precipitation for every year, which, however, can be easily estimated since the statistics are a priori known. For example, the initial parameters expressing the distribution of the past nonzero daily precipitation at the end of the projection in 2099 become β = 0.2, γ₁ = 1.26, and γ₂ = 0.53 (Figure 14c, green line); essentially, over this 80‐year period we have a progressive transition from the distribution shown in red to the distribution shown in green (Figure 14c). A realization of a daily time series disaggregating the monthly totals in Figure 14b and incorporating this nonstationary scheme is shown in Figure 14d.

Figure 14

Disaggregation of monthly projected precipitation to daily based on the explicit nonstationary framework. (a) Observed daily precipitation for July. (b) Projected monthly. (c) Starting (red) and ending (green) marginal distributions of the disaggregation kernels, over the simulation period (2020–2099), expressing a 40% linear increase in the mean value of nonzero precipitation (Scenario 1); red dots show the empirical distribution of the historical data. (d) Disaggregated daily time series based on Scenario 1. (e and f) Similar to (c) and (d) assuming 40% linear increase in the mean value and 30% in the standard deviation (Scenario 2).

Clearly, we can implement any nonstationary scenario and acquire changes not only in the mean but in any other summary statistic, such as the standard deviation, skewness, or L‐moments. Similarly, nonstationary scenarios can be implemented by assuming explicit changes in the marginal distribution's parameters; for example, we may wish to explore an increase in the heaviness of the distribution's tail, which will result in more extremes for the future. Note also that there is no limitation on the function we can choose to express trends; for example, it could be linear or any concave or convex function. For example, let us assume another case (Scenario 2) where in addition to the 40% increase in the mean (over the 80 years) there is also a linear increase in the standard deviation by 30%; that is, by the end of 2099 the standard deviation is expected to increase from 2.25 to 2.96 mm. In terms of the marginal distribution this implies a progressive shift from the distribution depicted in red to the shown in green (see Figure 14e). A corresponding realization of the disaggregation process according to this nonstationary scenario is shown in Figure 14f.

An important aspect we need to stress is the difference between the theoretical statistical properties of the disaggregation kernel and those of the generated blocks. These properties differ as the generated block is constrained to match a specific total at a coarser scale. To clarify by example, on July 2020, the monthly total is 2.87 mm, and the disaggregation kernel (generating the daily values) uses a mixed‐type marginal (p₀ = 0.84 and nonzero values described by the ), which has theoretical mean , or else, the theoretical expected monthly total is 31 × 0.20 = 6.20 mm. Yet the optimization procedure forces the selected daily block to match a monthly total of 2.87 mm. This implies that the disaggregated time series cannot be used to verify that indeed the kernels are nonstationary; for example, the theoretical increase in the mean, used in the previous scenarios, will not be reflected in their monthly totals as the monthly totals are specific values that do not coincide with the theoretical expected total of the kernels, which increases every year. However, the nonstationary framework, apart from being mathematically consistent, can be practically verified by simply dropping the disaggregation constrain. Particularly, if we generate a large number (e.g., 1,000) of daily blocks from a nonstationary scenario (Figure 15a shows a single realization from Scenario 1) and isolate the daily values of every year from all realizations and estimate their statistics, we can clearly reveal that the underline process is nonstationary and in agreement with the desired scenario. This is depicted in Figure 15b, which verifies the 40% increase in the mean and the constant value of the standard deviation (Scenario 1). Likewise, Figure 15c shows a single realization from Scenario 2 and Figure 15d verifies that the model generates a 40% increase in the mean and 30% increase in the standard deviation over an 80‐year period. For both scenarios, of course, we have verified that the rest of the statistical properties are for every year the desired ones, that is, p₀ = 0.84 and γ₂ = 0.53.

Figure 15

Nonstationary simulation of daily precipitation for July (see Figure 14a). (a) Generated time series based on 40% linear increase in the mean by 2099 (Scenario 1), (b) verification of the Scenario 1 model from 1000 simulations, (c) generated time series based on 40% linear increase in the mean and 30% in standard deviation by 2099 (Scenario 2), (d) verification of the Scenario 2 model from 1000 simulations.

Finally, we note that although the proposed method makes an explicit nonstationary modeling and disaggregation scheme technically feasible, its use should be ideally supported either by physical arguments (e.g., forcing emissions or temperature increases that are expected to cause a known evolution in the precipitation at fine scales) or by deterministic factors that are well identified and introduce nonstationarities in related processes (e.g., land use changes or increasing groundwater withdrawals modifying streamflow). As aforementioned, special care should be taken when nonstationarity is assumed based on trends or fluctuations resulting from fitting procedures on data. Of course the framework is “by definition” justified, for example, in sensitivity analysis or in exploring arbitrary nonstationary scenarios. For example, we may wish to investigate how a hydrological model reacts when fed with hourly precipitation that evolves in a specific way.

5 Concluding Remarks

The problem of producing fine‐scale precipitation time series (or of any other hydroclimatic variable) consistent with larger scale observations or model outputs while respecting known, albeit often limited, statistics at the finer scale is termed disaggregation. Disaggregation is essential for practical applications bridging the gap between the observational or climate modeling scales and the scales needed for impact assessment studies, planning, and infrastructure design (e.g., based on the so‐called intensity‐duration‐frequency curves) and risk analysis.

Despite substantial advances, existing disaggregation models exhibit shortcomings as at best they target to preserve a limited and insufficient number of statistics, for example, the first three moments and linear correlations up to only the first few lags, risking underestimation of the distribution's tail, and thus, the behavior of extremes. This stems from the nonunique mapping of a few statistical moments to the full form of a marginal distribution apart from a limited number of parametric distributions, such as normal or lognormal, which do not cover the full range of hydroclimatic variables of interest. Another shortcoming is the lack of generalized methods that can accommodate time‐varying statistics, that is, nonstationarity.

Here we present a precise and general disaggregation scheme, called DiPMaC that allows disaggregating a coarse‐scale time series into any finer temporal scale. The DiPMaC reproduces (a) the actual probability distribution and linear correlation structure at the fine scale, under stationarity, and arbitrary nonstationary scenarios, and (b) the coarse‐scale values. Regarding nonstationarity, we explored two approaches: the first, which is simpler, uses stationary disaggregation kernels, and nonstationarity in the fine‐scale process is introduced by forcing the fine‐scale stationary process to adapt to a nonstationary coarse‐scale process. In the second, which offers more flexibility, the nonstationary properties of the fine‐scale process are defined a priori. Moreover, we show how we can optimize the efficiency of the disaggregation procedure using a Bernoulli‐trials framework. In this direction, we introduced also an analytical method to evaluate the error caused by the rescaling adjustments (necessary if one wishes the fine scale time series to add up exactly to the coarse‐scale series), which allows us to preselect the level of the approximation error in balancing the preservation of the target distribution at the fine scales and the coarse‐scale amounts.

Finally, through a number of case studies, we demonstrate the performance and robustness of the DiPMaC algorithm in disaggregating monthly precipitation to hourly, and monthly GCM precipitation projections to daily. We show that DiPMaC meets both objectives of disaggregation methods; that is, it reproduces exactly the target marginal distribution and the ACS at fine scales, as well as, it allows to disaggregate coarse time series based on general nonstationary scenarios. As a future direction, the proposed framework offers the basis for a generalized approach for precise space‐time disaggregation across different spatio‐temporal scales.

Acknowledgments

We thank the three reviewers and the Associate Editor for providing meticulous reviews, which helped us to significantly improve the initial manuscript. This study was partially supported by National Science Foundation (NSF) grants (NSF WSC: EAR‐1209402, NSF LIFE: EAR‐1242458, and NSF CMMI‐1635797), National Aeronautics and Space Administration (NASA) grant (NNX16AO56G), National Oceanic and Atmospheric Administration (NOAA) grant (NA14OAR4310222), and California Energy Commission grant (500‐15‐005). The data used in this study are freely available at https://www.ncdc.noaa.gov/ and https://cmip.llnl.gov/cmip5/data_portal.html.