An impact study of updating background error covariances in the ALADIN‐France data assimilation system
Abstract
[1] The operational ALADIN‐France 3D‐Var system is based on static background error covariances calculated off‐line during a few week past period. In this study, the impact of an online updated specification of background error covariances is evaluated in the ALADIN‐France system. This evaluation is done by comparing three experiments, respectively based on (i) covariances calculated from a monthly average over a past period, (ii) covariances calculated from a monthly average over the period of study, and (iii) covariances calculated from a sliding daily average over the period of study. First, it is shown through a comparison between experiments (i) and (ii) that updating the monthly average of error covariances has a positive impact on the short‐range forecast quality. This is related to the specification of covariances which are more representative of average weather regimes at play during the period of study. Second, a comparison between experiments (ii) and (iii) indicates that additional positive impacts of a daily update of error covariances are also visible, although they tend to be somewhat localized and modest during this period. These impacts are illustrated by case studies for humidity during an anticyclonic situation, and for wind during a cyclonic event. These results support the idea to consider an online updated specification of background error covariances.
1 Introduction
[2] Data assimilation can be used to provide accurate initial conditions to Numerical Weather Prediction (NWP) forecast models. Usual data assimilation techniques for NWP rely on a combination of observed information and of a background, which corresponds to a short‐range forecast. These two information sources are weighted by their respective error covariances, and classically [Daley, 1991], it can be shown that the role of background error covariances is to spatially filter and propagate observed information. In practice, these error covariances are however difficult to estimate, for instance, because the true atmospheric state is never exactly known or due to the impact of model errors. Moreover, the size of the covariance matrix is too large to allow for explicit storage and evolution of covariances.
[3] Due to their important role during the analysis step, research efforts are still ongoing to increase the realism of these background error covariance estimates. This includes the use of error simulation techniques [e.g., Houtekamer et al., 1996; Fisher, 2003; Belo Pereira and Berre, 2006], for instance, which can rely on the addition and evolution of perturbations representative of errors existing and cycled in the data assimilation system. In addition to these error simulation techniques, research efforts are devoted to the relaxation of assumptions of temporal stationarity and horizontal homogeneity which have been often used in background error covariance modeling, as reviewed, e.g., in Berre and Desroziers [2010]. For instance, the spatial filtering of flow‐dependent variances and their impact are illustrated for global systems in Raynaud et al. [2009] and in Bonavita et al. [2012]. Ensemble estimates can also be used to specify flow‐dependent 3D and 4D error covariances [e.g., Buehner et al., 2010]. While these studies have been conducted extensively in global models, their possible application to limited area models is also receiving increasing interest. For instance, Storto and Randriamampianina [2010] showed that seasonal variations of covariances had significant impact on regional forecasts based on a high‐latitude version of the HARMONIE/Norway system, and Brousseau et al. [2012] focused on daily variations of covariances and their impact in the AROME‐France configuration.
[4] In the present study, we will characterize the respective impacts of both seasonal and daily variations in the context of the ALADIN‐France 3D‐Var regional data assimilation system. The currently operational version of this system is based on a static and horizontally homogeneous covariance model, and the present paper focuses on the impact of relaxing the static covariance assumption in this 3D‐Var system. This study has been conducted after diagnosing temporal variations of associated covariances [Monteiro and Berre, 2010], which indicates, for instance, that seasonal and daily covariance changes are significant, and that they are related to weather situation variations. Diurnal changes of covariances have been also examined in Monteiro and Berre [2010] and in Bölöni [2012], although their impact will not be studied here.
[5] The structure of the paper is as follows. The experimental framework is described in section 2. The impact of seasonal variations is presented in section 3, while section 4 is about the impact of daily variations. In section 5, results are discussed, and conclusions are drawn.
2 Experimental Framework
[6] The operational ALADIN‐France system is based on a local version of the regional ALADIN model [Horányi et al., 1996] and on a 3D‐Var data assimilation system [Fischer et al., 2005], with boundary conditions provided by the Météo‐France global ARPEGE system.
[7] The regional ALADIN model is spectral (based on a bi‐Fourier representation of the fields) and results from a limited area counterpart of the ARPEGE/IFS global system [Geleyn et al., 1995]. The version used in this study is based on a 10 km horizontal resolution and 60 hybrid vertical levels over a domain that covers France, the Iberian Peninsula, part of surrounding countries, and of the Mediterranean Sea (see illustration in Figure 9). The 3D‐Var ALADIN‐France data assimilation system consists on 6‐hourly assimilation cycling (at the main synoptic hours). It uses surface and upper air conventional observations over land and over sea (e.g., SYNOP, BUOY, TEMP, and PILOT) and also remote sensing observational data (such as AMSU‐A and ‐B, HIRS, MHS, AMV, SEVIRI, AIRS, and IASI data).
[8] The ALADIN‐France background error covariance model is detailed in Berre [2000]. It is based on horizontally homogeneous but scale‐dependent covariance estimates. These covariances are usually calculated off‐line from a few week average of forecast perturbation covariances, which are obtained by running an ensemble of perturbed assimilation cycles [e.g., Houtekamer et al., 1996; Fisher, 2003; Belo Pereira and Berre, 2006]. This ensemble assimilation method is based on the explicit addition of random perturbations to actual observations in order to simulate the effect of observation errors. It also relies on implicit background perturbations which are provided by the previous analysis perturbations, and explicit model perturbations may also be added to represent model error contributions. The ALADIN‐France ensemble assimilation cycle is coupled to the global ARPEGE ensemble assimilation system [Berre et al., 2007], which allows lateral boundary condition (LBC) errors to be simulated through the use of perturbed ARPEGE as boundary conditions. Apart from these LBC perturbations, the current version of the ALADIN‐France ensemble assimilation system is nevertheless based on a perfect model assumption (i.e., no explicit model perturbations are added), and resulting background error standard deviation estimates are increased by a factor 2 typically, to account for unrepresented model error contributions and for other possible approximations in the ensemble simulation.
[9] The version of the ALADIN‐France 3D‐Var which has been operational in 2008 is based on covariance estimates which have been temporally averaged off‐line over a 3 week period in Autumn 2007 (9 September to 4 October 2007) from a six‐member ensemble. This operational run will be referred to by the acronym AUT07S (where the final letter S refers to the use of static covariances). In order to evaluate the impact of relaxing this static covariance approach, this operational version will be compared with two other experimental versions during a 1 month winter period (13 February to 14 March 2008).
[10] The first experimental run is based on covariance estimates which are temporally averaged off‐line over the winter period of study (13 February to 14 March 2008) from a six‐member ensemble. This experiment will be denoted by WIN08S. The comparison between WIN08S and AUT07S allows for the evaluation of the impact of monthly variations affecting covariance estimates during the winter 2008 period compared to the autumn 2007 period. On the one hand, the experiment WIN08S is expected to be better than AUT07S due to the representation of prevailing weather regimes during the considered period. On the other hand, WIN08S is not a configuration that can be implemented operationally in real time, because it includes monthly covariance estimates which are computed a posteriori, i.e., after a first set of monthly deterministic and ensemble experiments have been conducted during the considered period. Despite this limitation, WIN08S is a useful benchmark for the evaluation of a second experimental run, which may be implemented operationally in real time, and which is based on covariances that are temporally averaged in a sliding way over each day preceding each analysis network. This second experimental run will be denoted by WIN08D (where the final letter D refers to the daily approach, compared to the static approach used in WIN08S). The comparison between WIN08D and WIN08S allows the impact of daily changes of covariances to be evaluated.
[11] For instance, for the analysis calculated on 13 February at 0000 UTC, background error covariances in WIN08D are estimated from a temporal average over samples of 6 h perturbed forecasts produced from perturbed analyses calculated on 12 February at 0000, 0600, 1200, and 1800 UTC. The covariance update in WIN08D is applied both to autocovariances and cross covariances (which are represented using linear regressions as in, e.g., Derber and Bouttier [1999] and Berre [2000]). In order to compensate the decrease of sample size when using such four‐network averages (instead of a few week average), a 12‐member ensemble has been used to estimate these daily covariances (instead of a six‐member ensemble in WIN08S and AUT07S). The sample size associated to the temporal and ensemble average is thus equal to 4×12=48, which may be expected to be too small, e.g., compared to at least 60 independent samples required for estimating full rank covariances for a 60‐level configuration. The effective sample size may be even smaller if the 12 perturbations are not fully independent (although quasi‐independence is expected due to the use of 12 independent sets of explicit observation perturbations and of implicit background perturbations). This sample size is however further increased through the spatial average which is implicitly applied when estimating homogeneous and isotropic covariances in spectral space [Berre and Desroziers, 2010]. In our case, the isotropic average is computed over total wave number bands whose spectral width is equal to 3 [see Berre, 2000, Appendix B]: spectral covariances associated to different wave vectors (m,n) are averaged over intervals k∗−1.5k∗+1.5, where k∗is the total wave number (as defined, e.g., in Berre [2000]). The effective number of contributing wave vectors (m,n) to a given wave band then depends on the total wave number k∗: the smallest value is equal to 4 (for the first band of small wave numbers), and this number increases for larger wave numbers (proportionally to k∗ typically). The total sample size is thus typically larger than 48×4=192, which makes feasible the estimation of covariances for a 60‐level configuration. Another potential issue when reducing temporal stationarity as in WIN08D is an increased risk of filter divergence. Although this does not occur in our experiments, a possible solution to this issue would be to revert in such an event to static covariances.
[12] The impact of monthly and daily variations of covariances will be assessed in particular by calculating root mean squared (RMS) values of the differences associated to AUT07S, WIN08S, and WIN08D forecast and analysis fields with respect to TEMP and SYNOP observations, and also with respect to the ECMWF analysis. The RMS of these differences will be denoted by RMSE in the curve labels.
3 Impact of Monthly Covariance Variations
3.1 Impact on the Forecast Quality
[13] As described in the previous section, comparing experiments AUT07S and WIN08S allows for the evaluation of the impact of monthly covariance variations on the forecast quality. These monthly covariance variations correspond to the difference between the few week averages of covariances computed over the considered autumn 2007 and winter 2008 periods.
[14] The impact on the 12 h forecast average quality is illustrated on Figures 1 and 2. While the impact is fairly neutral above 300 hPa, positive impacts of WIN08S are noticeable in the middle and low troposphere, for both temperature (Figure 1) and wind (Figure 2).
[15] Examination of associated temporal variations are shown for 500 hPa, for temperature in Figure 3 and for wind in Figure 4, which is the level where the error reduction is larger. These figures indicate that improvements are relatively frequent over the whole period, leading to significantly smaller RMS (e.g., at 90% level in Figure 3, and at 95% level in Figure 4, according to bootstrap significance tests (Wilks [2006], using two‐tailed nonparametric bootstrap tests)). This is also supported for instance by Figure 5, which corresponds to the impact on 24 h forecasts of relative humidity at 700 hPa.
[16] These results suggest that using covariance estimates averaged during the 1 month period of study has a positive impact on the short‐range forecast quality. The impact is rather neutral beyond the 24 h forecast range (not shown). This is likely to be related to the influence of the lateral boundary conditions, which are identical in the two experiments, and whose impact on the regional forecast increases with the forecast range.
3.2 Changes in Analysis Fit and in Vertical Correlations
[17] In order to understand the positive impact of WIN08S on short‐range forecasts, the analysis fit to observations has been compared between experiments WIN08S and AUT07S. A general increase of the analysis fit to observations can be noticed in WIN08S compared to AUT07S. This is illustrated in Figure 6 for the analysis fit to TEMP observations of temperature at 500 hPa. This suggests that the short‐range positive impact arises from an increased analysis fit to observations, which occurs in more than 3/4 of the days during the considered period (as shown in Figure 6).
[18] These analysis fit changes between WIN08S and AUT07S that arise from differences in the background error covariances that are specified respectively in these experiments. While specified standard deviations and horizontal correlations of background errors are similar in the two experiments (not shown), vertical correlation diagnostics indicate that the increased analysis fit may be connected with the use of sharper vertical correlations in WIN08S compared to AUT07S. The increased sharpness of vertical correlations in WIN08S is illustrated at 500 hPa in Figure 7(top) for specific humidity and Figure 7(bottom) for divergence. These vertical correlation differences are likely to reflect the fact that vertical mixing processes (associated toconvection) were relatively less prevailing during the winter 2008 period than in the autumn 2007 time interval. This tends to be supported by the diagnostic study in Monteiro and Berre [2010], which indicates that broader negative lobes (in vertical correlations) reflect increased vertical couplings in summer convective situations.
[19] These results support the idea that using background error covariances that are representative of prevailing weather regimes during the impact study can be beneficial to the short‐range forecast quality.
4 Impact of Daily Covariance Variations
[20] As described in section 2, comparing experiments WIN08S and WIN08D allows for the evaluation of the forecast quality impact of daily covariance variations within the considered February to March 2008 period.
4.1 Overall Impact
[21] Examination of time‐averaged forecast scores indicates that the average impact of daily variations of covariances is nearly neutral for the considered period, because local improvements of scores tend to be masked by frequent neutral impacts. This is illustrated in Figure 8 for the time series of surface pressure RMSE of 6 h forecasts with respect to SYNOP data. It appears that the difference between the RMSE values of the two experiments is very small, although a tendency to have local positive impacts of WIN08D (compared to WIN08S) can be noticed, for instance, at the beginning of the period, and also during the March part of the period.
[22] To some extent, the overall neutral impact is expected in the sense that daily covariances in WIN08D are relatively similar on average to those in WIN08S, since they are calculated from the same February–March period. On the other hand, one could also expect that the local impact of WIN08D may be daily varying and somewhat larger for dates when the daily covariances (not shown) are relatively different from the 1 month average used in WIN08S. This expectation tends to be supported by the somewhat larger positive impact of WIN08D visible in Figure 8 at the beginning of the period, which is anticyclonic, and in the March part which is cyclonic.
[23] This kind of local positive impacts will be further illustrated by two case studies. Associated weather situations, which contrast with the prevailing zonal regime over most of the considered February–March period are shown in Figure 9.
4.2 Impact on Humidity During the Anticyclonic Period
[24] Figure 10 corresponds to the time evolution of horizontally averaged background error standard deviation estimates of specific humidity near 500 hPa which are specified in WIN08D. It can be noticed in particular that there is a relatively large contrast between small values (around 0.15 g/kg) in the first 6 days of the period and larger values (up to 0.25–0.3 g/kg) during the next 10 days. The static horizontally averaged background error standard deviation of WIN08S is also shown (full line) to contrast with the daily fluctuations of the online approach.
[25] This is related to the fact that the first 6 days correspond to a winter anticyclonic situation with relatively cold and dry air, while the following days are affected by a zonal regime with a warmer and moister atmosphere. This dependence of moisture error standard deviations on the weather situation is also visible, for instance, in the time series of 12 h forecast RMSE of relative humidity at 500 hPa. This is shown in Figure 11, which corresponds to a zoom over the first 10 days over the period, in order to illustrate changes between the first 5 and 6 anticyclonic days (with values around 15%) and the next dates (with values up to 30%).
[26] It can also be noticed in Figure 11 that the 12 h forecast RMSE is smaller for WIN08D than in WIN08S during the anticyclonic 6 day beginning of the period. This suggests that this may be a good example to illustrate local positive impacts of representing daily variations of background error standard deviations.
[27] To examine this more in detail, respective analysis increments (analysis minus background) are plotted in Figure 12 (for 13 February 2008). While the spatial structure of increments is relatively similar between the two experiments, it can be seen that the amplitude of increments is smaller in WIN08D than in WIN08S. This is a direct effect of specifying smaller daily varying humidity background error standard deviations in WIN08D than the monthly averaged values used in WIN08S, as shown in Figure 10. This coherent connection between Figures 10, 11, and 12 shows that specifying daily varying humidity background error variances that are consistent with the weather situation and with the associated water content of the air mass can be beneficial for a better forecast performance.
[28] This tends to be supported by Figures 13 and 14 which correspond to differences between ALADIN‐France analysis and the ECMWF analysis taken here as a reference. It can be seen in Figure 13 that the amplitude of departures is smaller in WIN08D than in WIN08S, for instance over Denmark, England, and also in the surrounding oceanic areas. This coincides well with areas where the amplitude of analysis increments has been reduced in WIN08D compared to WIN08S.
[29] These effects are also visible in Figure 14, which indicate that the amplitude of departures from the ECMWF analysis is noticeably reduced during the anticyclonic first 5 days.
4.3 Impact on Wind During the Cyclonic Period
[30] Figure 15 shows the vertical profile of time‐averaged RMSE for the 24 h forecasts with respect to radiosondes taken as a reference. A slightly positive average impact of WIN08D can be noticed at 850 hPa, in particular. Examination of time series indicates that this kind of average slight positive impact tends to be more pronounced during the March cyclonic period. This is illustrated in Figure 16, which corresponds to a 36 h forecast RMSE for wind at 850 hPa, zoomed over the March cyclonic period. The positive impact of WIN08D is particularly noticeable for the 36 h forecast valid on 11 March at 1200 UTC, which has been launched from the analysis calculated on 10 March at 0000 UTC. It is interesting to investigate whether this may correspond to specific daily changes in the background error covariances specified in WIN08D.
[31] To examine this, an example of respective analysis increments of WIN08S and of WIN08D is plotted in Figure 17 for zonal wind at 850 hPa. This Figure 17 corresponds to increments produced on 10 March at 0000 UTC, zoomed over the southern part of France and surrounding areas. It can be seen for instance that the amplitude of increments is much larger for WIN08D than for WIN08S near the South‐East coast of France. Figure 18 shows that this is consistent with larger specified vorticity background error standard deviations in WIN08D than in WIN08S for this date, which belongs to the cyclonic part of the period of study. Figure 19 indicates that this leads to smaller departures with respect to the ECMWF analysis (taken as a reference here) for the WIN08D analysis than for the WIN08S analysis. This is visible for instance in the area corresponding to the dark blue departure structure that is elongated meridionally near the middle of Figure 19, for instance with maximum values of 8–10 m/s in WIN08S reduced to 6–8 m/s in WIN08D.
[32] This is a case that illustrates a mechanism through which specifying daily varying background error standard deviations in cyclonic situations can lead to daily varying analysis increment amplitudes and potentially more realistic analysis estimates.
5 Conclusions
[33] In this study, the impact of temporally updating specified background error covariances has been studied for the ALADIN‐France 3D‐Var system during a 1 month winter period. This has been carried out by comparing an operational 3D‐Var version, which uses covariances that are estimated off‐line from a few week average over September 2007, with two experimental versions in which updated background error covariance estimates are specified.
[34] The first experimental version uses specified background error covariances estimated from a 1 month average corresponding to the period of study. This allows for using time‐averaged covariances that are consistent with average weather regimes prevailing over the considered period. Results indicate that using such updated covariances has a positive impact on the short‐range forecast quality of temperature, wind, and humidity. This is connected to an increased analysis fit to observations, which arises from sharper vertical correlations for the updated winter covariances than in the operational September covariances.
[35] The second experimental version uses specified background error covariances estimated from 1 day sliding averages preceding each analysis network. Compared to the first experimental version, this allows also daily covariance variations (in addition to monthly variations) to be taken into account in the 3D‐Var system. Results indicate that, during our period of study, these daily variations have more modest and localized positive impacts than the monthly variation impact which has been studied by comparing the operational and first experimental version. Case studies have been shown to illustrate local positive effects of daily varying background error standard deviations for humidity during an anticyclonic situation and for wind during a cyclonic period.
[36] Moreover, it may be underlined that the daily varying experimental version also has a better forecast quality than the operational version, in a similar way as for the monthly updated experimental version compared also to the operational version. This implies that the online daily varying covariance approach may be envisaged for operational applications, as a replacement to the current off‐line static approach (corresponding to the calibration over September 2007). This would allow both monthly and daily variations of background error covariances to be represented.
[37] While this kind of impact studies could be carried out during other periods, there are also other aspects that could be examined. For instance, the impact of diurnal variations of covariances could be evaluated by using a larger ensemble than in the current study. The impact of the domain size would also be interesting to examine. On the one hand, one may expect that the domain should be large enough to allow the regime associated to the forecast error to remain in the domain. On the other hand, the domain should be small enough to avoid that too many differing regimes are mixed in the sliding average. Moreover, while the present study has focused on temporal variations within the currently operational homogeneous covariance framework, the effect of horizontal heterogeneities could also be studied in addition.
Acknowledgments
[38] M. Monteiro's working stay in Météo‐France was supported by the ALADIN flat‐rate funding. The authors would like to thank the reviewers for the instructive and complementary suggestions.
