Analyzing input and structural uncertainty of nonlinear dynamic models with stochastic, time‐dependent parameters

Abstract

[1] A recently developed technique for identifying continuous‐time, time‐dependent, stochastic model parameters is embedded in a general framework for identifying causes of bias and reducing bias in dynamic models. In contrast to the usual approach of considering bias in model output with an autoregressive error model or a stochastic process, we make the attempt to correct for bias within the model or even in model input. This increases the potential of learning about the causes of bias and of subsequently correcting deficits of the deterministic model structure. The time‐dependent parameters as formulated in our approach can also consistently be used for adding stochasticity to the model without losing precise fulfilment of conservation laws used for deriving the model equations. An additional advantage of the suggested procedure is that it makes it possible to derive more realistic uncertainty bounds of internal model variables than is the case when bias is only considered for measured model output. This is important for mechanistic models in which internal variables have a direct physical meaning. The concept is illustrated by an application to a simple eight‐parameter conceptual hydrological model. This application demonstrates the feasibility of the proposed approach and gives an impression of its potential for application to a large class of nonlinear, dynamic models.

1. Introduction

[2] Statistical inference of parameters of deterministic models is often done by assuming the data to be independently and identically distributed around the model predictions at “true” parameter values (to be inferred from the data). The concept underlying this statistical approach is that the deterministic model (at “true” parameter values) describes the “true” system behavior, and the probability distributions centered at the predictions of the deterministic model describe the measurement error that is assumed to be independent for different measurements. For a well‐calibrated measurement device, this assumption seems often to be reasonable.

[3] However, when plotting time series of residuals of dynamic models, it often becomes apparent that the assumption of identically and independently distributed errors is strongly violated. Heteroscedasticity of the residuals is often caused by a relative error contribution to the total error. If error variance is mainly dependent on output it can be accounted for by applying an adequate transformation to model results and data [Box and Cox, 1964]. If it depends on other model variables as well, parameterization of the error variance may still be possible. However, the significant autocorrelation of the residuals that is often observed is rarely caused by the measurement device. In most cases it is caused by input and model structure errors. Such errors are propagated through (part of) the model which often has internal states that make its output dependent on past conditions. This leads to autocorrelated output errors even in the absence of autocorrelation in input errors. When only data sets with time steps larger than the correlation length of such induced output errors are available, this problem is insignificant. However, with increasing temporal density and precision of measurements becoming available with modern instrumentation this problem becomes more and more apparent. If this bias is not explicitly addressed in the formulation of the inference problem, inferred uncertainty bounds of parameters and results become unreliable (usually, uncertainty is significantly underestimated).

[4] The problem of violation of the assumption of independent errors has been discussed in the hydrological literature (e.g., discussion of the work by Thiemann et al. [2001] by Beven and Young [2003] and Gupta et al. [2003a]). Earlier recognition of this problem, or more generally of calibrating models in hydrology, led to the development of alternative approaches to calibration than statistical inference. One of these approaches is the generalized likelihood uncertainty estimation (GLUE) technique [Beven and Binley, 1992; Beven and Freer, 2001] that is widely used in hydrology (this is only an alternative approach if a “generalized” likelihood function is used; otherwise it is a numerical approach to Bayesian inference). In this approach, the likelihood function is replaced by an empirical, “generalized” likelihood function that is flatter and thus leads to larger parameter uncertainty bounds. However, the empirical nature of the “generalization” of this approach violates some of the concepts underlying the Bayesian approach to statistical inference [Mantovan and Todini, 2006; Beven et al., 2007; Mantovan et al., 2007; Beven et al., 2008]. Another interesting approach is multiobjective calibration promoted in hydrology by Gupta et al. [1998], Yapo et al. [1998], Gupta et al. [2003b], and Boyle et al. [2003]. It was developed following the experience that least squares or similar calibration techniques do not only have the statistical deficiencies mentioned above, but that they not even lead to a choice of parameter values a hydrologist would select as physically most appropriate. However, the problem still remains of how to use the results of multiobjective calibration for probabilistic prediction.

[5] The problem of all of these approaches is that they either deal exclusively with parametric uncertainty of a model or try to map all uncertain elements to an “effective” parametric uncertainty. However, even highly overparameterized distributed hydrological models are not able to reproduce observed hydrographs within measurement accuracy, even when considering all errors of the measurement process (sampling error, instrument calibration error, measurement device error, extrapolation error from point measurement to areas, etc.). For this reason, not even refined identifiability analysis techniques that analyze which parts of a time series are particularly informative with respect to which parameters can provide sufficient help for calibration [Wagener et al., 2003]. This is also the reason why Bayesian model averaging performs poorly [Vrugt and Robinson, 2007], although this is in general a very powerful technique for dealing with model structure uncertainty. If applied to a set of poor models (as conceptual, deterministic rainfall‐runoff models are from a statistical point of view), Bayesian model averaging cannot lead to satisfactory results. Satisfactory calibration can only be achieved by increasing the flexibility of the underlying model in describing the observed system output.

[6] The main problems of all the difficulties mentioned above are therefore the simplifying assumptions necessary for constructing a model of a very complex natural system. Such a model will never be able to reproduce system behavior within measurement error. Not accounting for model structure and input error is, in technical terms, use of an incorrect likelihood function. Thus, we need a better likelihood function, not an alternative inference procedure (see discussion above about suggested alternative inference procedures). However, this insight does not solve our problem as we cannot expect that our models will ever be perfect. Nevertheless, it implies that it is a very important research direction to try to incorporate a description of model deficiency and input error into statistical inference approaches. The idea here is not to modify the inference approach, but to perform it with a more adequate likelihood function. The construction of such a likelihood function requires a procedure that supports the modeler in (1) improving the structure of the deterministic part of the model and (2) adding a random part to the model that accounts for the elements that are not considered in the deterministic model description. This is the research direction in which we are focusing in the remainder of this introduction and paper.

[7] The simplest approach for formulating a likelihood function that corrects for the error outlined above is to explicitly formulate a model output deficiency or bias term in nonparametric form and to jointly estimate this term with the model parameters. This methodology gained attention in the statistical literature in recent years [Craig et al., 1996; Craig et al., 2001; Kennedy and O'Hagan, 2001; Bayarri et al., 2007]. Note that this follows a longer tradition in the applied sciences where it has been realized that residuals do not represent errors of the measurement process only but that they include the effect of input error and model structure deficits (some early examples for such efforts in hydrological modeling are the works by Clarke [1973], Sorooshian and Dracup [1980], and Kuczera [1983]). In time series models, and in particular in hydrological modeling, the more complex structure of the residuals is often accounted for by the use of an autoregressive error model [Kuczera, 1983; Bates and Campbell, 2001; Engeland and Gottschalk, 2002; Yang et al., 2007a, 2007b]. Note that this is a special case of the use of Gaussian processes as suggested by the references from the statistical literature given above. This statistical description of the effect of model bias on output, and the use of autoregressive error models that serve the same purpose, are important steps toward improving the statistical assumptions underlying the application of dynamic simulation models. In many cases, use of one of these techniques may be a good choice. However, these techniques also have disadvantages. The main disadvantages are that these methods do not significantly contribute to uncovering the causes of the bias and that they do not support the estimation of the uncertainty of internal model variables. As the final goal of modeling is to decrease the bias as far as possible, a technique that would support the identification of possible causes of bias would be more useful. The uncertainty of internal model variables may be of interest for mechanistic models within which these have a physical meaning. The uncertainty of internal model variables should then also consider model structural errors. This is not the case if the effect of structural error is considered only as a bias or autoregressive error term added to model output variables that correspond to measured time series.

[8] Instrumental variable techniques were developed to avoid biased inference results for noncausal models [Young, 1984; Söderström and Stoica, 1989; Heckman, 1997; Young, 2008; Heckman, 2008]. However, these techniques rely on the identification of adequate instrumental variables to extend the model structure. Techniques that attempt to identify potential model deficits without making additional structural assumptions would therefore be more promising. For analyzing systematic deviations of the results of dynamic models from observations such a technique was proposed more than 30 years ago [Beck and Young, 1976; Beck, 1987]. The idea was to make (time‐invariant) model parameters time‐dependent to allow them to compensate for model structure deficits. The identified time variation of the parameters can then be used to gain hints about the potential of model structure extensions that would improve the behavior of an extended model with time‐invariant parameters. Technically, this idea was implemented by applying an extended Kalman filter in a state space of a linear model augmented by the parameter vector. Local linearization in the extended Kalman filter approach is required because the augmentation with the parameter vector makes the extended model nonlinear even if the original model was linear.

[9] The idea of using time‐dependent parameters for model structure improvement led to the development of the data‐based mechanistic (DBM) modeling approach [Young and Lees, 1993; Young and Beven, 1994; Young, 2002, 2003; Romanowicz et al., 2006]. This technique applies to models consisting of a nonlinear transformation of input and a linear transfer function model with state‐dependent parameters. Because of the state‐dependent parameters, this is a quite large class of models. Taking advantage of this class of model structures, the method is very efficient because it is based on the recursive estimation of transfer function model parameters that vary over time because of their dependence on other measured or modelled variables (states). The estimation algorithm is in the form of a back‐fitting algorithm in which the recursive Kalman filter and smoothing algorithms are used to estimate these state‐dependent parameters on the basis of data that have been reorganized into a nontemporal order, so as to reveal the state dependency. The class of model structures is very well suited for hydrological modeling as it allows for a nonlinear description of the runoff generation process followed by linear flood routing by the transfer function. This led to many applications in hydrology [see Young and Lees, 1993; Young and Beven, 1994; Young, 2002, 2003; Romanowicz et al., 2006]. On the other hand, the technique is not directly applicable to an arbitrary nonlinear dynamic model.

[10] A related development consisted of replacing the extended Kalman filter by a recursive prediction error algorithm applicable to linear and nonlinear model equations [Stigter and Beck, 2004]. This led to the suggestion of synthesizing data‐based and theory‐based models of environmental systems [Lin and Beck, 2006, 2007b] that was also applied to hydrological and environmental systems [Lin and Beck, 2007a].

[11] Starting from the same basic idea of identifying the causes of model structure deficits as outlined by Beck and Young [1976], in this paper, we follow a similar pathway as the one cited in the previous paragraph. Instead of developing an efficient technique for a special (but rather large in the case of DBM) class of models, we intend to design a technique that is, at least in principle, applicable to any nonlinear, dynamic model. As we want to avoid combining continuous‐time models with discrete‐time, time‐dependent parameters, in contrast to all approaches cited above, we use a continuous‐time stochastic process to describe the time‐dependent parameter. This is conceptually more satisfying and does not require observations at equally spaced points in time. This is realized by combining a recently developed technique for the estimation of continuous‐time, stochastic, time‐dependent parameters of nonlinear, dynamic models [Brun, 2002; Buser, 2003; Tomassini et al., 2009], with a systematic approach of model structure deficit analysis. The fundament of the generality of the approach is that it does not rely on linear submodels or the need for local linearization. Still, convergence properties will depend on the model structure and the available data. Instead of correcting model output as it is done by modeling the bias term or using an autoregressive error model, model parameters are made time‐dependent in an attempt to account for structural deficits or sources of stochasticity already where they occur instead of correcting their effect on model output. The analysis of the behavior of the time‐dependent parameter can then lead to an improvement of the deterministic model structure or the stochastic, time‐dependent parameter can be integrated as a stochastic element in the model to account for intrinsic randomness of the system (at the aggregation level used to describe the system by the model). We prefer including randomness in the model structure through random parameters instead of making the state equations random [Moradkhani et al., 2005; Vrugt et al., 2005; Vrugt and Robinson, 2007] for two reasons. First, it is conceptually more satisfying to make fluxes instead of state equations random, as this does not violate conservation laws and seems to more adequately represent processes in nature. Second, there is a higher potential of learning from the behavior of time‐dependent parameters than from corrections in model states. The reason is related to the first, conceptual point: the modeler may have ideas about fluxes that may be influenced by factors that are not considered in the model.

[12] Our approach can thus be viewed as an extension of the original idea of using time‐dependent parameters for system identification [Beck and Young, 1976], as a generalization of data‐based mechanistic (DBM) modeling [Young and Beven, 1994; Young, 2002, 2003; Romanowicz et al., 2006] to more general, nonlinear, dynamic model structures, or as an extension of “bias modeling” [Craig et al., 1996, 2001; Kennedy and O'Hagan, 2001; Bayarri et al., 2007] by moving the bias correction from model output to intrinsic model variables. As the time‐dependent parameter can also be a modification factor of or an additive term to a model input, it can also be viewed as a generalization of the concept of introducing storm‐dependent input modification parameters, as it has recently gained attention in hydrology [Kuczera, 1990; Kavetski et al., 2003, 2006a, 2006b; Kuczera et al., 2006]. To make this link stronger, we use the same illustrative case study as it was used by Kuczera et al. [2006] to demonstrate the usefulness of storm‐dependent input modification parameters.

2. Methodology

[13] Figure 1 outlines the suggested procedure for model deficiency and input error analysis, deficiency reduction, and uncertainty estimation with the aid of time‐dependent, stochastic parameters. In a first step, (subsets of) the model parameters are replaced by stochastic processes. The states of these parameters are then jointly estimated with the other (time‐invariant) model parameters and with selected parameters of the stochastic process. In a second step, the results of the replacement of each parameter subset obtained in the first step are analyzed for their degree of bias reduction. For the parameter subsets leading to a significant bias reduction, exploratory analyses are done in the third step, for identifying potential dependences of the time‐dependent parameters on external influence factors or internal model states. If such dependences are found, in a fourth step, the deterministic model is modified to account for these dependences and the analysis is restarted with step 1. If no more dependences can be identified, the remaining stochasticity (if any) is considered in the model in step five of the procedure by making one or more model parameters stochastic. If several parameters can explain the remaining bias, this selection must be based on prior knowledge of dominant mechanisms and modeling uncertainties of the system. In environmental models, a stochastic parameter can account for changes in coefficients of rate or substance flux parameterizations due to aggregation errors, adaptation, influence factors that are not considered by the model, or other model simplifications. Because of the necessity of such simplifications, it cannot be expected that a deterministic model leads to a good description of an environmental system. As we want to respect mass conservation principles that often underly the deterministic model equations, we prefer adding stochastic variations to parameters (i.e., in rate or flux formulations) rather than states.

Figure 1

Recommended procedure for model deficiency and input error analysis, deficiency correction, and uncertainty estimation with the aid of time‐dependent parameters.

[14] The five steps outlined above and visualized in Figure 1 are described in more detail in sections 2.1–2.5 and will be illustrated in sections 4.1–4.5, with results for a hydrological model introduced in section 3. As mentioned in section 1, this scheme extends a procedure that was suggested more than 30 years ago [Beck and Young, 1976; Beck, 1987] and led to the development of the concept of data‐based mechanistic modeling [Young and Beven, 1994; Young, 2002, 2003; Romanowicz et al., 2006] and to the application of discrete‐time, time‐dependent parameters in mechanistic environmental modeling [Stigter and Beck, 2004; Lin and Beck, 2006, 2007a, 2007b]. We will extend this framework to continuous‐time, time‐dependent parameters for arbitrary nonlinear, dynamic models using a recently developed estimation technique for time‐dependent parameters [Brun, 2002; Buser, 2003; Tomassini et al., 2009].

2.1. Estimating Time‐Dependent Parameters

[15] The key step of the procedure is the formulation and estimation of continuous‐time, time‐dependent parameters. We apply the procedure for estimating time‐dependent parameters developed by Brun [2002], Buser [2003], and Tomassini et al. [2009]. To introduce the notation used in this paper, we give a short summary of the technique in the following paragraphs.

[16] We represent the discrete output of our deterministic, continuous‐time, nonlinear, dynamic model as a single column vector, y, that consists of the results of specified model variables at given points in time (and space if the model is spatially explicit). Model inputs are divided into inputs to the system described by the model, x, and model parameters, ^M, for which optimal values must be found to make the model a good representation of the system. The output of the deterministic model is then characterized by the function

[17] The simplest modeling concept would then be to assume the deterministic model to perfectly represent the output of the system so that only measurement error, E_y(^M), must be added to the deterministic model results to get a statistical description of observed system output:

The likelihood function of this model would then be given by the joint probability density of the vector of random variables, Y^M(x, ^M):

where is the joint probability density of model results and is the joint probability density of the measurement errors. In most cases, a simple error model is assumed for the measurement error, E_y. The most frequent choice is a multivariate normal distribution without correlation.

[18] As discussed in section 1, because of input and model structure errors (and sometimes also due to a more complex measurement process), this assumption is often inadequate and typically leads to overconfident uncertainty estimates of model parameters and results (in particular if the measurement process is described by a normal distribution). A more adequate statistical description is therefore given by

where b is a description of the bias of the deterministic model [Craig et al., 1996, 2001; Kennedy and O'Hagan, 2001; Bayarri et al., 2007]. Often, a Gaussian process is used for describing the knowledge about this bias term, or an autoregressive error model is applied for describing b + E_y [Yang et al., 2007a, 2007b]. This procedure is often useful, but it lacks the potential of uncovering the causes of bias, and it does not solve the problem of getting corrected uncertainty estimates for internal model variables or for model output that is not observed.

[19] To increase the potential of improving the mechanistic description, we are thus trying to incorporate this bias into the model structure. This is done by replacing a subset, i, of parameters of the model _i^M, by stochastic, time‐dependent processes, Θ_i^M,t(^P,i). The idea is that the time‐dependent parameters would compensate for the bias internally in the model so that we would not longer need an additive output bias term. Thus, equation (4) becomes

In this equation, M⁽ⁱ⁾ represents the model with the parameter(s) i of the original model replaced by stochastic processes in time, _−i^M, represents the set of parameters of the original model that excludes parameter(s) i, Θ_i^M,t is the stochastic process in time that replaces the parameter(s) i of the original model, and ^P,i is the set of (additional) parameters that characterize the stochastic process Θ_i^M,t. The three model structures given by equations (2), (4), and (5) are visualized in Figure 2. When combining the parameters of the original model with the exception of parameter(s) i, _−i^M, with the parameters, ^P,i, characterizing the stochastic process underlying parameter(s) i, we get the set of parameters for the model M⁽ⁱ⁾ used for describing the outcome of the model when the parameter(s) i is (are) replaced by a time‐dependent stochastic process

Note that this extension of the model is not restricted to the original model parameters. By introducing additional additive or multiplicative parameters to model input and output (with default values of zero or unity, respectively), this formulation includes and extends model deficiency or bias terms as suggested in the literature cited earlier [Craig et al., 1996, 2001; Kennedy and O'Hagan, 2001; Bayarri et al., 2007], but it adds additional options of tracking the cause of the bias within the model structure. The extension of the model by making a subset of parameters stochastic processes makes the original model stochastic. As discussed below, this may be necessary to get a more realistic description of reality.

Figure 2

Dependence structure of the original model with (left) time‐invariant parameters, ^M (equation (2)), (middle) the model with additive bias, b (equation (4)), and (right) the model resulting from replacing parameter subset i by a stochastic process in time, Θ_i^M,t, to compensate internally for bias and to consider stochasticity (equation (5)).

[20] Estimation of model parameters of our extended model consists of estimating the time‐invariant parameters of the original model, _−i^M, (some of) the time‐invariant parameters of the stochastic process characterizing the time‐dependent parameter, ^P,i, as well as the realization of the time course of the time‐dependent parameter, Θ_i^M,t. Model prediction only requires the specification of the (joint) distribution of the time‐invariant parameters _−i^M and ^P,i as there will be no information available about the realization of the time‐dependent parameter, Θ_i^M,t, that is an internal variable of the extended model (see right structure in Figure 2).

[21] When discretizing the time‐dependent parameter on a fine grid, the likelihood function of the model M⁽ⁱ⁾, as a function of the time‐invariant parameters (6) only, is approximated by the following integral:

In this equation, the integration extends over the values of the time‐dependent parameter(s) at all points of the grid that discretizes the temporal dimension.

[22] In the following, we limit the set of parameters to be replaced by stochastic processes to one to avoid difficult choices of correlation parameters. The procedure can be extended to the multidimensional case. As a simple choice of a continuous‐time, stochastic process, we choose the time‐dependent parameter, θ_i^M,t, to be replaced by a mean reverting Ornstein‐Uhlenbeck process. This process gives the parameter the freedom for temporary variations without moving too far from its mean. This seems to be a reasonable compromise between the case of a constant parameter and a parameter described by a continuous random walk that would offer even more flexibility but would make predictions extremely uncertain. If more freedom seems to be necessary, a parameterization of the dependence of the parameter on states or external influence factors is required. The mean‐reverting Ornstein‐Uhlenbeck process fulfills the stochastic differential equation

where μ_i is the mean, τ_i is a characteristic time of correlation, σ_i² is the asymptotic variance of the process, and W_t is a random walk based on Gaussian white noise, a so‐called Wiener process [Kloeden and Platen, 1995; Øksendal, 2003]. This process has an analytical solution for the conditional distribution at a given time point, t, given the solution at an earlier time, t₀, described by the following normal distribution

This equation shows that, as a function of time, the mean starts at the given value at t₀ and approaches exponentially the given mean, μ_i, of the process. Parallel to this development, the variance increases from zero and approaches the asymptotic variance, σ_i². This process seems to be a reasonable generalization to allow a previously time‐invariant parameter to vary in time around its mean value. For the parameters we would like to keep stochastic, we do not want to allow more freedom. For identifying model deficits this seems to be sufficient; if there seems to be a trend in the identified time series, the parameter can be replaced by an adequate parameterization of this trend for the next iteration of the procedure shown in Figure 1.

[23] Note that for deterministic models described by a set of ordinary differential equations, equation (8) allows us to describe the model with time‐dependent parameter(s) as a set of stochastic differential equations, where equation (8) is added to the set of equations of the original model that do not have a diffusion term. On the other hand, equation (9) allows us to draw representations of a discretized process without having to apply a numerical integration scheme for stochastic differential equations. The first of these notes is conceptually satisfying, whereas the second leads to significant practical advantages for implementation.

[24] The numerical Markov chain Monte Carlo simulation scheme used to approximate the posterior distribution of time‐invariant and time‐dependent parameters is based on a Gibbs sampler that generates the (k + 1)th point of the Markov chain by sampling the following values [Tomassini et al., 2009].

[25] 1. according to (θ_−i^M∣, , y) = (θ_−i^M∣, y) ∝ (θ_−i^M) · (y∣θ_−i^M, ).

[26] 2. according to (θ^P,i∣, , y) = (θ^P,i∣) (θ^P,i) · (∣θ^P,i).

[27] 3. according to (θ_i^M,t∣, , y) ∝ (θ_i^M,t∣) · (y∣_−i^M, ). The dependence on x is not shown here to avoid an even more complicated notation. The updating step of the time‐dependent parameter θ_i^M,t is based on a Metropolis‐Hastings algorithm within subintervals of the time axis using a conditional solution of the Ornstein‐Uhlenbeck process with fixed values at both ends of the intervals [Buser, 2003; Tomassini et al., 2009]. The parameters _−i^M and ^P,i are sampled by a conventional Metropolis algorithm with normal jump distributions.

2.2. Analyzing the Degree of Bias Reduction

[28] The bias of dynamic simulation models due to input and model structural errors consists usually of three components: insufficient quality of fit (the deviation of model results from measurements is often larger than the measurement error), heteroscedasticity of the residuals (the error variance is often not constant in violation of the simple statistical assumption of identically distributed residuals that is often made), and autocorrelation of residuals (significant autocorrelation of residuals is often in contradiction to statistical assumptions about independent measurement errors). In environmental modeling, there are often the following dominant causes of these sources of bias.

[29] 1. The quality of the fit is not sufficient. If a careful parameter estimation has been done, the dominant causes of insufficient quality of fit are usually input errors and model structure deficits. To assess the quality of the fit, we need an estimate of measurement accuracy. This is often not easy to get, as the overall measurement error can be dominated by errors due to the sampling process or due to extrapolation of point measurements to areas rather than errors of the measurement device. It is often difficult to get independent replicates of this whole procedure when getting data from environmental systems.

[30] 2. There is heteroscedasticity in the residuals. The dominant causes of heteroscedasticity of the residuals are relative error contributions of both, the measurement process and model structure errors. This often leads to a relatively simple dependence of the error variance on model output. In this case, either using a parameterization of this relationship when formulating the statistical model or applying an adequate transformation to data and model results is often a successful strategy to significantly reduce this error. Still, the error will be the combination of measurement, input and model structure errors.

[31] 3. The residuals are autocorrelated. A lag phase of a measurement device, equilibration of concentrations in a sampling volume with surrounding concentrations, or other aspects of the sampling procedure can lead to autocorrelation of measurement errors. However, in most cases these effects are small and the dominant cause of autocorrelation of residuals in environmental modeling studies are input and model structure errors. As dynamic simulation models usually have internal states that depend on the history of driving forces, even uncorrelated input errors will lead to autocorrelated output errors (as they affect these states and through this effect also future behavior). Similarly, errors due to deficits in the model structure either lead directly to systematic deviations in output or they lead to autocorrelated output errors due to their propagation through parts of the model.

[32] The analysis above suggests the following strategy for analyzing the degree of bias reduction achievable when making different parameters time‐dependent: Try to eliminate heteroscedasticity with using an error variance that is parameterized as a function of model results (and, if necessary, other important influence factors) and use quality of fit measures and autocorrelation of residuals as primary indicators for quantifying bias and bias reduction by time‐dependent parameters. As the improvement of the fit will depend on the variance of the time‐dependent parameter, this variance must be chosen carefully by the analyst if the output error is not known (we cannot estimate input and output error jointly [Zellner, 1971]). The results of this analysis should then allow the analyst to rank the parameters according to their potential for bias reduction or at least to separate parameters with a high potential of bias reduction from those without. The understanding of mechanisms in the system described by the model should then allow the analyst to relate the dominant parameters to mechanistic causes of the model structure deficit.

2.3. Identifying Potential Dependences of Parameters on States

[33] The results of steps 1 and 2 of the procedure illustrated in Figure 1 and described in sections 2.1 and 2.2 lead to the identification of the potential of different parameters for bias reduction and to a posterior distribution of these parameters as a function of time. If the residuals are significantly larger than the error of the measurement process (including sampling), improvement of the quality of the fit (with careful assessment of not overfitting) is an important indication of bias reduction. In addition to the degree of improvement of the fit, for each parameter, we also get the time series of the parameter that leads to the best fit of the data. In step 3 of the procedure, we make the attempt to interpret this time series. In particular we are interested to learn if the time‐dependent parameter compensates for a deficit of the deterministic model structure (which should not be described by a stochastic process) or if it describes a random effect of the system described by the model (which may be represented in an improved model structure by a stochastic process). Besides results of statistical analyses, this distinction requires knowledge of the mechanistic structure of the system described by the model.

[34] In simple cases, visual assessment of a plot of the time‐dependent parameter may already uncover obvious systematic effects (e.g., periodic variation with season or day or variations that follow external variables) [Beck and Young, 1976; Beck, 1987]. For more complicated models we suggest the following procedure to follow the visual assessment: Compile time series of all available external influence factors and internal model states and perform an explorative data analysis of the time series of the time‐dependent parameter as a (potential) function of these factors. It may be useful to restrict the explorative analysis to time domains within which the uncertainty range of the time‐dependent parameter is small, as this indicates high identifiability of the temporal behavior. Any explorative statistical methodology may be useful, such as scatterplots, stepwise regression, or cluster analysis. If these analyses uncover potential relationships between some of these factors and the model parameter, these results provide direct hints for model improvement to be implemented in step 4 of our procedure (see section 2.4 below). If no relationship is found, internal stochasticity of the system described by the model may be responsible for the bias. In this case, one or several parameters may have to be made stochastic to get a more realistic description of the system. This is briefly discussed in section 2.5.

2.4. Improving the Deterministic Model

[35] The identification of parameters with a high potential of bias reduction in step 2 of the procedure described in Figure 1 (see section 2.2) leads to the identification of submodels of the deterministic model for which improvements may lead to a better system description. The explorative analyses in step 3 (see section 2.3) may lead to insights that generate more concrete suggestions for the formulation of such improvements. If a significant bias reduction was possible in step 2 and in step 3 significant relationships were found, an improvement of the deterministic model should be possible. In any case, if a significant bias reduction was possible, trials should be made with fits of models that contain modifications to the submodels affected by the parameter that led to significant bias reduction. If a model with less bias can be found (a better fit without overfitting and/or less autocorrelation of the residuals if the assumption of independent measurement errors seems reasonable), the analysis should be restarted at step 1 with the modified model.

2.5. Describing Remaining Stochasticity

[36] If no more significant dependences in step 4 can be identified, possibly after several iterations with modified models through the steps 1 to 4 of the procedure shown in Figure 1, it seems reasonable to attribute any remaining model deficiencies to random sources. Knowledge of the driving forces and internal mechanisms of the system described by the model will then be necessary to identify the dominant inputs or submodels that should be made stochastic to improve the model structure. Input errors, aggregation errors, and influence factors not considered by the model can be important reasons to make a deterministic description not adequate. Knowledge may be available about which model simplifications are the most critical ones for which a stochastic description may be appropriate. In contrast to step 2 (section 2.2), where a relative assessment of different parameters was already very useful (as long as overfitting is avoided), in this step, knowledge of the measurement error is very important. This knowledge, formulated as an informative prior distribution, allows us to infer the variance of the time‐dependent parameter required for improving the fit [Zellner, 1971] and to check for overfitting by comparing the standard deviation of the residuals with the measurement error. A comparison of the inferred variance of the time‐dependent parameter with the knowledge of this uncertain input or model structure element will provide an independent check of the adequateness of making this particular parameter time‐dependent.

[37] After having completed the procedure shown in Figure 1 we can hope having identified an improved structure of the deterministic model and included adequate stochasticity in the model to guarantee a more adequate description of the investigated system. This revised model should then lead to a reasonable description of parameter and model prediction errors within a conventional statistical inference process (see discussion in section 1). In section 3 we will introduce a hydrological model to which this procedure will be applied in section 4.

3. Hydrological Modeling, Model, Watershed, and Model Application

3.1. Uncertainty in Hydrological Modeling

[38] Uncertainty of model predictions has gained a lot of attention in hydrological modeling over the past 20 years (Duan et al. [2003] give a current overview). This led to the development of many different techniques to address input, parameter, model structure, and output measurement uncertainty. Some of them are generalized likelihood uncertainty estimation (GLUE) [Beven and Binley, 1992; Beven and Freer, 2001; Beven, 2006], parameter solution (ParaSol) [van Griensven and Meixner, 2006], sequential uncertainty fitting (SUFI‐2) [Abbaspour et al., 2004], Bayesian inference based on Markov chain Monte Carlo [Kuczera and Parent, 1998; Vrugt et al., 2003; Yang et al., 2007a, 2007b], and Bayesian inference based on importance sampling [Kuczera and Parent, 1998]. In a recent comparison of these techniques [Yang et al., 2008] we concluded that, because of its sound theoretical foundation, it seems to be the better strategy to improve the formulation of the likelihood function within Bayesian inference instead of inventing alternative inference procedures that have a poorer conceptual foundation. In particular, it was concluded that it is important to explicitly consider input and model structure uncertainty at their source instead of modeling their effect as an autoregressive output bias term. There has been considerable development in this field in recent years [Kavetski et al., 2003; Vrugt et al., 2003, 2005; Kavetski et al., 2006a, 2006b; Kuczera et al., 2006; Vrugt and Robinson, 2007]. It is the goal of this case study to contribute constructively to this development.

3.2. Hydrological Model logSPM

[39] The simple hydrological model logSPM consists of mass balance equations for the three compartments of soil, groundwater and river as visualized in Figure 3 [Kuczera et al., 2006]. The water balance in the soil is formulated as a differential equation for the water in the soil per unit watershed area, h_s:

The amount of water stored in the soil increases because of rain minus surface runoff, (q_rain − q_runoff), and it decreases because of evapotranspiration, q_et, lateral subsurface flow, q_lat, and percolation to groundwater, q_gw. The water balance in the groundwater is formulated as a differential equation for groundwater per unit watershed area, h_gw:

The amount of water stored in groundwater aquifers increases because of percolation from the soil, q_gw, and it decreases because of release as base flow to the river, q_bf, and because of percolation to deep aquifers, q_dp. Finally, the water balance in the river is formulated as a differential equation for river water per unit watershed area, h_r:

The amount of water stored in the river(s) increases because of surface runoff, q_runoff, lateral subsurface flow, q_lat, and base flow from groundwater, q_bf, and it decreases because of river flow out of the watershed, q_r. The fluxes in equations (10)–(14) are parameterized as follows:

The model described by equations (10) and (11) contains two nonlinearities. The fraction of the watershed with saturated soil, f_sat, and the ratio of actual to potential evapotranspiration, f_et. Both of these quantities are parameterized as functions of the mean water depth in the soil, h_s. They are given by

and

respectively. These nonlinear dependences are visualized in Figure 4. The final output of the model, river flow, Q_r, is given as the product of the watershed area, A_w, and river flow per watershed area, q_r,

(see below for an explanation of the factor f_Q).

Figure 3

Schematic diagram of the model logSPM [Kuczera et al., 2006].

Figure 4

Shape of the nonlinear functions used for describing the fluxes shown in Figure 3 and given by equations (11a)–(11h). (left) Fraction of saturated area, f_sat, given by equation (12) and (right) fraction of actual evapotranspiration, f_et, given by equation (13). The solid parts of the curves represent the range of values covered in the base simulation shown in Figure 5.

[40] The hydrological model defined by equations (10)–(14) has two input time series, rainfall, i_rain(t), and potential evapotranspiration, i_pet(t). In addition, the watershed area, A_w, is considered as an external input to the model, rather than a model parameter.

[41] The model has eight parameters and three initial conditions, and we added three additional parameters to manipulate the input time series i_rain(t) and i_pet(t) and the output, Q_r. These three factors, f_rain, f_pet, and f_Q, will be equal to unity for the base simulation. As time‐dependent parameters, they serve the purpose of investigating input uncertainty and multiplicative output bias. All parameters and their marginal prior distributions are summarized in Table 1. Note that this model deviates in three details from the original model by Kuczera et al. [2006]: First, the parameter s_F in the model used in this paper is equal to the original parameter sF by Kuczera et al. [2006] plus 99. Second, the parameter k_et was (implicitly) set to unity by Kuczera et al. [2006]. And third, we added a deep percolation flow, q_dp, to the model.

Table 1. Parameters of the logSPM Model and Their Marginal Prior Distributions as Used in our Application^aa LN(μ, σ) means a lognormal distribution with mean μ and standard deviation σ. The intervals given in brackets are 95% prior uncertainty intervals.

Parameter	Units	Prior Value	Meaning
ks	1/mm	LN(0.01,0.02),	coefficient describing increase of saturated area
		[0.00037,0.054]	with soil water depth
sF		LN(300,600),	coefficient describing increase of saturated area
		[11,1613]	with soil water depth
ket	1/mm	LN(0.01,0.02),	coefficient describing increase of evapotranspiration
		[0.00037,0.054]	with soil water depth
qlat,max	mm/d	LN(2,4),	maximum lateral flux per unit of watershed area
		[0.074,10.8]
qgw,max	mm/d	LN(6,12),	maximum percolation flux to the groundwater
		[0.22,32]	per unit of watershed area
kbf	1/d	LN(0.002,0.004),	coefficient for base flow water discharge from
		[0.000074,0.011]	groundwater to the river
kdp	1/d	LN(0.002,0.004),	coefficient for deep percolation from ground
		[0.000074,0.011]	water to deeper aquifers
kr	1/d	LN(1,2),	coefficient for river water discharge to
		[0.037,5.4]	downstream river sections
hs,ini	mm	LN(100,200),	initial water volume per unit area in the soil
		[3.7,538]
hgw,ini	mm	LN(100,200), [3.7,538]	initial water volume per unit area in the groundwater
hr,ini	mm	LN(0.1,0.2),	initial water volume per unit area in the river
		[0.0037,0.54]
frain	‐	1	rain multiplier
fpet	‐	1	potential evapotranspiration multiplier
fQ	‐	1	river discharge multiplier

• a LN(μ, σ) means a lognormal distribution with mean μ and standard deviation σ. The intervals given in brackets are 95% prior uncertainty intervals.

3.3. Likelihood Function

[42] To decrease the heteroscedasticity of the residuals, a Box‐Cox transformation was applied to model results and data. We then assumed the measurement error to be homoscedastic, independent and normally distributed in the transformed units:

where λ₀ is a constant to make (y + λ₂) nondimensional, λ₂ is an offset parameter with the same units as y, and λ₁ is a nondimensional exponent. On the basis of previous experience [Yang et al., 2007a, 2007b], we applied this transformation with y₀ = 1 m³/s, λ₂ = 0.1 m³/s, and λ₁ = 0.3. Linear error propagation applied to the transformation (15) yields the following approximate dependency of the standard deviation in original units, σ_y, on the standard deviation in transformed units, σ_y′, as a function of y expressed in the original units,

For an exponent λ₁ < 1 and a constant standard deviation in the transformed units, σ_y′, this leads to an increase in the standard deviation in the original units, σ_y, with increasing values of y. The offset λ₂ > 0 guarantees that the standard deviation in the original units does not approach zero if y approaches zero.

[43] In transformed units, we use now the likelihood functions discussed in section 2.1. Equation (3) without bias correction is used for comparative purposes and equation (7) with internal bias correction based on time‐dependent parameters for the analysis of model deficiencies.

3.4. Abercrombie Watershed

[44] The model was applied to the Abercrombie watershed, New South Wales, Australia. The watershed area, A_w, of this watershed is 2770 km². The data were kindly provided by George Kuczera, to allow for comparison with other studies on input uncertainty [Kuczera et al., 2006].

3.5. Model Application

[45] The marginal prior distributions of all parameters are summarized in Table 1. We chose wide prior distributions to account for our lack of knowledge of watershed characteristics and the high aggregation level of the model that makes it difficult to transfer knowledge about physical properties of the watershed into values of model parameters. Independence of the parameters was assumed when building the joint prior distribution.

[46] Step functions were used to describe the two model inputs of rainfall intensity and potential evapotranspiration to account for the fact that only daily sums of these variables were available. The model output, Q_r, according to equation (14), was averaged over 1 day for comparison with measured data. In contrast to the work by Kuczera et al. [2006] the model was applied to the whole time series kindly provided by George Kuczera to better allow for an assessment of the presence or absence of a long‐term trend in ground water level.

[47] To get samples approximating the posterior parameter and result distributions, Markov chains of length 50,000 were run for the time‐invariant parameter cases, of length 5000 when exploring the effect of making different parameters time‐dependent, and of 40,000 for the final simulation with making the selected parameter time‐dependent. To save storage space, only each fifth step was kept for the time‐invariant parameter case, each second step when including time‐dependent parameters. Multiple realizations of the chains were calculated while improving the start values and (normal) jump distributions based on the previous run. This finally led to good convergence even of relatively short chains in both cases, for time‐invariant and time‐dependent parameters. This was confirmed by running the long chain for the faster case of time‐invariant parameters.

4. Results and Discussion

[48] We started the analysis with a base parameter estimation of the time‐invariant parameters of the model described in section 3. The simulation results shown in Figure 5 demonstrate that the model is able to reproduce the main characteristics of the data.

Figure 5

Results of the base model with time‐invariant parameters as a function of time in days since 31 December 1971. (a) Detail of Figure 5b at a higher temporal resolution (section between the dashed vertical lines). (b) Measurements (dots), results for parameter values corresponding to the maximum posterior distribution (solid lines), and calculated 95% probability confidence intervals of model predictions without measurement error (grey area) of river discharge at the watershed outlet (left axis) and measured rainfall intensity (right axis). (c) Water level in the groundwater reservoir (solid line and left axis) and in the soil (dashed line and right axis). (d) Runoff (solid line and left axis) and lateral flow, base flow, and deep percolation flow (long‐dashed, short‐dashed, and dash‐dotted lines and right axis). (e) Residuals of Box‐Cox‐transformed data and model results.

[49] However, systematic deviations remain. In particular, the normality and independence assumptions of the error term are not fulfilled, and with a posterior mean of 1.2 the standard deviation of the transformed residuals is too large to represent measurement error only. Note that according to equation (16), in linear approximation, this would correspond to a standard deviation in original units increasing from 0.24 m³/s at a discharge of 0–30 m³/s at a discharge of 100 m³/s. As most of the deviation between data and model results is explained by the “measurement error,” the model prediction without measurement error is overconfident. The 95% confidence intervals of the prediction are hardly distinguishable from the prediction line (see Figures 5a and 5b). The Nash‐Sutcliffe index [Nash and Sutcliffe, 1970] of our simulation was significantly smaller (0.51) than that found by Kuczera et al. [2006] (0.74). This is probably due to the longer simulation period and the use of the Box‐Cox transformation that decreases the weight of large discharges (see equation (16)). This can affect the Nash‐Sutcliffe index (calculated in the original, not transformed scale) significantly.

[50] In addition to these deficits of the statistical description, there is also a physical deficiency in the model simulation. Despite having introduced a deep percolation flux into the model (see Figure 3 and equations (10b) and (11g)) and applying the model over a longer simulation period than that of Kuczera et al. [2006], the simulation at the maximum of the posterior distribution shows an increasing trend of groundwater level over the simulation period (see Figure 5c), as was also found by Kuczera et al. [2006].

[51] In the following, we try to reduce the deficits of the deterministic model and to add appropriate stochasticity to the model to account for nondeterministic system and input characteristics. We do this according to the procedure illustrated in Figure 1. The following subsections 4.1–4.5 correspond to the steps 1–5 of this procedure and implement the techniques described in the subsections 2.1–2.5 for our case study.

4.1. Estimating Time‐Dependent Parameters

[52] To try to explore the causes of systematic deviations of model results from measurements, we replaced sequentially the logarithms of all eight model parameters and three modification factors listed in Table 1 by random Ornstein‐Uhlenbeck processes and identified their joint posterior with the other, time‐invariant parameters as described in section 2.1. The mean of the log of the modification factors was kept equal to zero, the means of the logs of the other parameters were estimated on the basis of the prior given in Table 1.

[53] The degree of bias reduction of different parameters depends on the sensitivity of the model results to the parameter, the amount of memory effects of the submodels depending directly or indirectly on the parameter, and the degree of variability allowed for the parameter. To eliminate the last factor from the comparison, we chose the same standard deviations, σ_i, for the logs of all time‐dependent parameters. Our choice of a value of 0.2 for this standard deviation implies a comparison of the degree of bias reduction possible with relative parameter changes of the order of −33 to +50% (based on the 95% confidence interval). The characteristic times, τ_i, were also set to the same value of 1 day. This allows the parameter to change at the same time scale as new measurements are available. This high flexibility seems meaningful at this exploratory stage of the procedure; this choice has, however, to be reassessed for step five of the analysis.

4.2. Analyzing the Degree of Bias Reduction

[54] In the second step of the procedure illustrated in Figure 1, we quantify the bias reduction achievable with making different parameters time‐dependent. As outlined at the beginning of section 4, the most significant contributions to bias are the poor fit (the estimated standard deviation of the error term is significantly larger than measurement error), the autocorrelation of the residuals, and the nonnormality of the residuals. As we can hardly expect a model improvement that reduces the autocorrelation of the residuals without improving the fit, we can use the degree of improvement of the fit as a primary indicator of bias reduction. However, we have to be careful that we do no longer apply this criterion when the residual error approaches the measurement error, as then a further reduction would be an indication of overfitting.

[55] Using the Nash‐Sutcliffe index [Nash and Sutcliffe, 1970] as an indicator of the quality of fit, Table 2 shows to which degree different time‐dependent parameters can reduce the bias. On the basis of the top four parameters in this ranking there are three options for model improvement.

Table 2. Nash‐Sutcliffe Indices for Simulations With One Parameter Made Time‐Dependent and Base Simulation Without Time‐Dependent Parameter

Time‐Dependent Parameter	Nash‐Sutcliffe Index
frain	0.90
ks	0.84
fQ	0.67
sF	0.63
fpet	0.60
kr	0.57
ket	0.54
qlat,max	0.54
kdp	0.53
qgw,max	0.52
kbf	0.52
none	0.51

[56] 1. The rain multiplier, f_rain, represents input uncertainty of rainfall. The significant improvement of the fit achieved by making this parameter time‐dependent is an indication that input uncertainty could be a major cause of systematically deviating model results. However, this interpretation needs a careful check of the size of input “corrections” applied by this parameter as it is not astonishing that modification of the main driving force improves the fit. On the other hand, as precipitation must be extrapolated from a small number of measurement sites to the whole catchment, input uncertainty is known to be significant in rainfall‐runoff modeling of large catchments [Kuczera, 1990; Kavetski et al., 2003; Kavetski et al., 2006a, 2006b; Kuczera et al., 2006]. This result also corresponds to similar results in data‐based mechanistic modeling [Young and Beven, 1994; Young, 2002, 2003; Romanowicz et al., 2006], in which time dependence of the parameter related to input was particularly influential.

[57] 2. The parameters k_s and s_F are used to quantify the nonlinear dependence of the soil water household on soil water level (see equation (12) and Figure 4). According to equations (11b), (11d), and (11e) and the flow scheme shown in Figure 3, runoff, infiltration, lateral flow, and percolation are affected by the function f_sat(h_s) determined by these parameters. The significant improvement of the fit achieved by making these parameters time‐dependent leads to the hope that an alternative function f_sat(h_s) or an alternative improvement of the soil submodel could lead to better results.

[58] 3. The output multiplier, f_Q, represents (multiplicative) bias correction of the output. This is the classical way of dealing with model bias [Craig et al., 1996, 2001; Kennedy and O'Hagan, 2001; Bayarri et al., 2007].

[59] As we want to account for causes of bias instead of providing bias description, we concentrate on improvement of the soil runoff submodels and on describing and quantifying input uncertainty.

4.3. Identifying Potential Dependences of Parameters on States

[60] The time series of the state of the time‐dependent parameter that leads to the best fit can be hoped to provide hints for the formulation of model improvements. In the third step of the analysis illustrated in Figure 1, we try to find such hints by searching for relationships between the time‐dependent parameter and model states or external influence factors. This analysis can be constrained to time domains within which the posterior is significantly narrower than the prior. This guarantees that only values are used that could be identified from the data. Despite doing a careful analysis for the present application, no significant relationships could be found between time‐dependent parameter estimates and internal or external influence factors. This is a hint that stochasticity may be a main contributor to model deficiency or that improvement of the deterministic model will only lead to improved model performance for a small fraction of the data points.

4.4. Improving the Deterministic Model

[61] Steps 2 and 3 of the procedure illustrated in Figure 1 led to somewhat diverging indications. The significant bias reduction achievable in step 2 (section 4.2) leads to the hope that a significant fraction of this reduction can be achieved by improving the deterministic model. According to the results of this analysis, the soil runoff submodels are the most promising candidates for model improvement. On the other hand, the explorative data analysis performed in step 3 (section 4.3) did not lead to more precise hints for improvement of the deterministic model. Nevertheless, in step four of the procedure illustrated in Figure 1, we try to improve the deterministic model.

[62] The results from section 4.2 indicated that there may exist opportunities for model improvement of the soil runoff submodels. On the basis of this result, we implemented two model modifications. Modification 1 provides a more flexible parameterization of the nonlinear saturated area fraction function:

This function allows the differentiation of two regimes of increase of saturated area fraction. Modification 2 improves the runoff model. As Figure 5 demonstrates, the model is unable to match the highest peaks during flood events. This may be caused by soil areas that are more quickly saturated then the mean response in the catchment. This cannot be considered by our simple hydrological model that does not resolve the spatial dimension. A simple description of the effect of fast runoff can be achieved by allowing part of the rain to directly leave as runoff, if the rain intensity is very high. The modified runoff flux is then parameterized as follows:

with

Exponent 4 was chosen to guarantee that the term is very small at small and intermediate rainfall intensities. These modifications are visualized in Figure 6.

Figure 6

Suggested model modifications according to (left) equation (17) and (right) equation (18). The solid parts of the curves represent the ranges of values covered in the base simulation shown in Figure 5.

[63] The more flexible parameterization of the saturated area function did not significantly increase the Nash‐Sutcliffe index (of 0.51 in the base simulation) despite three additional parameters. However, the other model improvement defined by equation (18) led to an increase of the Nash‐Sutcliffe index from 0.51 to 0.73. This is quite substantial even when considering that this modification requires two additional parameters. For this reason, we chose to use the model extension given by equation (18) but to keep the original saturation area parameterization.

[64] Figure 7 shows the results of the model with the modifications given by equations (18a) and (18b). The results of the simulations are very similar to those of the original model with the exception that five of the seven largest positive residuals (too low prediction of high floods) decreased significantly (compare Figures 5e and 7e). However, we still remain with a too large “measurement” error (σ_y′ = 1.1 in transformed units corresponds according to equation (16) approximately to an increase from σ_y = 0.22 m³/s at a discharge of zero to σ_y = 28 m³/s at a discharge of 100 m³/s) and too narrow uncertainty bands (without measurement error) that are still hardly distinguishable from the prediction line.

Figure 7

Results of the model modified according to equation (18) with time‐invariant parameters as a function of time in days since 31 December 1971. See caption of Figure 5 for an explanation of the plots.

4.5. Describing Remaining Stochasticity

[65] Our inability to improve the fit by extending the parameterization of the saturated area function and the minor improvement of most of the discharge hydrograph achieved by the modified description of runoff are indications that further mechanistic model improvements may be difficult to find. Note that the Nash‐Sutcliffe index improved considerably for the second modification given by equation (18) because the improvement of a small number of high discharges has a significant influence on this indicator. As there are still significant deficiencies in the description of the observed hydrograph by the model, it seems reasonable that these are caused by random mechanisms not (yet) described by the model. Step 5 of the procedure illustrated in Figure 1 makes the attempt to include the dominant random mechanisms in the model.

[66] The major mechanisms leading to a failure of the deterministic description of river discharge by a simple, conceptual hydrological model are aggregation and input errors.

[67] 1. Because of the spatially aggregated description by the model, states of the system with the same amount of water stored in soil and groundwater but differing in the spatial distribution of the water are described by the same state of the model. Applying the deterministic model formulation, this results in the same calculated river discharge. In the real system, the spatial distribution of water affects its release to the river. Keeping the model structure simple, this mechanism can be described statistically by temporal variations in parameters characterizing water release from soil and groundwater.

[68] 2. It is well known that rainfall often has a very high spatial variability. As usually only point measurements at rain gauges are available, this leads to a very high uncertainty when deriving watershed‐integrated rain intensities from measured rain data. In our approach, this uncertainty can be taken into account by the rain intensity multiplication factor, f_rain (see equations (11a) and (11b)).

[69] Despite the conceptual difference of these sources of stochasticity, they may be hardly distinguishable in model applications, because heterogeneous rainfall is the cause for “nonequilibrated” filling of soil and ground water reservoirs. For this reason, it seems meaningful to describe the combined error of rainfall uncertainty with respect to spatially averaged flux and spatial inhomogeneity by the single time‐dependent parameter f_rain. We have to be aware, however, that this factor will then be larger than if it would only represent the uncertainty in rainfall averaged over the catchment.

[70] In contrast to step 1 of the procedure illustrated in Figure 1 (see section 4.1), we are now interested in inferring the variation required for this parameter to explain the model deviations except for the measurement error. This requires the specification of an informative prior for the measurement error. Therefore, for this analysis, we use a lognormal distribution with a mean of 0.5 and a standard deviation of 0.05 as a prior for the standard deviation of the error of Box‐Cox‐transformed model results and data. According to equation (16), in linear approximation, this corresponds to a standard deviation of discharge measurements increasing from 0.1 m³/s at a discharge of zero to 12.6 m³/s at a discharge of 100 m³/s. We then include the standard deviation of the modification factor, f_rain, with a noninformative prior proportional to in our inference procedure. As capturing precipitation at a daily resolution can be assumed to be only weakly correlated between successive days, we kept the characteristic correlation time at 1 day as in the exploratory analysis done in section 4.1. Note, however, that for very high intensity sampling of precipitation we may want to use a larger correlation time than the measurement interval to account for the correlation structure imposed by the movement of precipitation cells. But this is not relevant for daily averaged measurements.

[71] Figures 8, 9, and 10 show the results of the model with the extension (18) and with making the parameter f_rain time‐dependent. Figure 8 shows that the systematic deviations decreased significantly, the 95% uncertainty band of the prediction is no longer confined to the line width of the simulation, the groundwater level does not show a significant trend, and the residuals are smaller and considerably less heteroscedastic and autocorrelated. Figures 9a and 9b shows the realization of the rain input modification factor leading to the best fit, and Figures 9c and 9d show the median and 95% uncertainty band of its posterior distribution. The wide 95% uncertainty bands in Figures 9c and 9d clearly indicate that there exist only a few short time domains of high identifiability. These time domains correspond to time of high precipitation. In the case of the parameter f_rain, it is evident, that it is not identifiable when precipitation is zero. This leads to artificially high values during periods of no rainfall. During the periods within which the parameter is identifiable, it is constrained to values between about 0.4 and 2.5. This seems not an unrealistic range given the high uncertainty when extrapolating data from point rainfall measurement sites to the whole catchment and when having in mind that the parameter also parameterizes uncertainty because of uneven spatial distribution of rainfall (and resulting water transport). Finally, Figure 10 shows the prior and posterior marginals of all model parameters for all three simulations shown in Figures 5, 7, and 8. Given the informative prior for the standard deviation of the error term in transformed units, all parameters with the exception of the initial water level in the river are identifiable (to different accuracy). However, as it is typical for models with systematic errors, the values of the model parameters depend strongly on the model formulation. There is strong correlation between the two parameters of the saturated area fraction function f_sat, k_s and s_F (not visible in Figure 10). The poor identifiability of the initial water level in the river is caused by the short retention time of water in the river of the order of 1.1 d. This leads to only a short‐term effect of this initial condition on the simulation. The last parameter, σ_Q,trans, the estimated standard deviation of discharge in transformed units (see equation (15)), demonstrates the improvement of the fit with the model modification (18) and with the time‐dependent parameter.

Figure 8

Results of the model modified according to equation (18) with f_rain made time‐dependent as a function of time in days since 31 December 1971. See caption of Figure 5 for an explanation of the plots.

Figure 9

(a, b) Realization of the time‐dependent parameter f_rain leading to the best fit and (c, d) median and 95% probability interval of the marginal posterior distributions of the time‐dependent parameter at each point in time. The two dashed lines in Figures 9b and 9d indicate the section shown in Figures 9a and 9c. Dashed lines in Figures 9a and 9c indicate values of 0.5 and 2.0.

Figure 10

Marginals of the parameters of the prior (dash‐dotted lines) and of the posterior marginals for all simulations shown in Figures 5 (original model, time‐invariant parameters, long‐dashed line), 7 (modified model (18), time‐invariant parameters, short‐dashed line), and 8 (modified model (18), time‐dependent parameters, solid line).

[72] Note that this application of time‐dependent parameters to rainfall input time series is very similar to the rain multiplier technique suggested earlier to account for rainfall input uncertainty [Kuczera, 1990; Kavetski et al., 2003, 2006a, 2006b; Kuczera et al., 2006]. However, our technique is more generally applicable to time series models for which it may not be easy to define natural divisions of the time axis in subintervals as it is possible for rain events. In particular it is also applicable to other parameters of the hydrological model that affect continuous processes relevant during dry periods.

5. Summary and Conclusions

[73] We propose a systematic approach to model deficit analysis and improvement. The approach is based on techniques of the estimation of continuous‐time, time‐dependent model parameters developed earlier [Brun, 2002; Buser, 2003; Tomassini et al., 2009]. Our approach can be viewed (1) as an extension (with respect to continuous‐time description and applicability to nonlinear models) of a similar use of discrete‐time, time‐dependent parameters developed over the last 30 years [Beck and Young, 1976; Beck, 1987; Young and Beven, 1994; Young, 2002, 2003; Stigter and Beck, 2004; Romanowicz et al., 2006; Lin and Beck, 2006, 2007a, 2007b], (2) as a further step of bias analysis by moving from a statistical description of the effect of bias on the output [Craig et al., 1996, 2001; Kennedy and O'Hagan, 2001; Bayarri et al., 2007] to correcting the sources of bias in the model structure or input, or (3) as a generalization of the concept of rain multipliers accounting for rainfall input uncertainty in hydrological modeling [Kuczera, 1990; Kavetski et al., 2003, 2006a, 2006b; Kuczera et al., 2006]. Besides offering a systematic approach to improving the deterministic model, it is a very important feature of the technique to provide a pathway to including stochasticity into the (previously) deterministic model. Because of input and aggregation errors inherent in environmental models, we think that this may usually be necessary to make the model a realistic description of the underlying environmental system.

[74] In the particular application to a hydrological model discussed in this paper, our technique led to a clear identification of submodels susceptible to improvement. However, it did not provide good hints on how to improve these submodels. This was in contrast to an application of our technique to the didactical model of a synthetic degradation experiment used by Bayarri et al. [2007]. For this model, from a scatterplot of the identified time‐dependent parameter versus concentration, we got an indication of the functional shape required to improve the model. In the application described in this paper, even after improving the deterministic model, the model still did not lead to a satisfying description of the measured hydrograph. Application of a time‐dependent modification factor to rainfall input to address input uncertainty with respect to average flux and spatial distribution, significantly improved the fit. However, as there may be rain events without precipitation at the measurement site (that cannot be captured with a simple input modification factor), a more specific rain uncertainty model could probably improve the representation of the uncertainty of catchment‐wide rainfall better than our general technique of applying time‐dependent parameters also to input variables. In addition, the intermittent nature of rainfall leads to large sections of the time series within which the correction factor is not identifiable (because it is multiplied by zero). The suggested technique has thus more advantages over other approaches (e.g., to the use of rainfall multipliers) if it is applied to parameters to which the model output is sensitive permanently. This is typically the case for model‐internal parameters.

[75] The main problem of the proposed approach is its lack of numerical efficiency. This makes it difficult to apply to computation‐intensive computer codes. For the class of models used in data‐based mechanistic modeling, more efficient techniques exist [Young and Beven, 1994; Young, 2002, 2003; Romanowicz et al., 2006]. More research is required to achieve a similar level of efficiency for the continuous‐time, stochastic parameters and for general nonlinear, dynamic models that we address with our approach. This could be achieved by increasing the efficiency of the numerical solution technique, by using a less computation intensive technique that leads to similar results, or by applying the technique to a fast emulator of a computation intensive simulator [Currin et al., 1991; O'Hagan, 2006; Bhattacharya, 2007; P. Reichert et al., Mechanism‐based emulation of dynamic simulation models: Concept and application in hydrology, submitted to Computational Statistics and Data Analysis, 2009].

Acknowledgments

[76] This work profited significantly from previous work with Hans‐Rudolf Künsch, Roland Brun, Christoph Buser, Lorenzo Tomassini, and Mark Borsuk and from various discussion in SAMSI working groups within the program on “Development, Assessment and Utilization of Complex Computer Models” (see http://www.samsi.info). The hydrological application profited from a lot of previous work in this field (cited in section 3.1). In particular, it builds on work by the research group of George Kuczera, who also kindly provided the data. This makes our results directly comparable to previous studies by this group. Extensive comments by Jasper Vrugt, Peter Young, and an anonymous reviewer led to significant improvements in the manuscript. This project was partially supported by the U.S. National Science Foundation under agreement DMS‐0112069. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.