An Agenda for Land Data Assimilation Priorities: Realizing the Promise of Terrestrial Water, Energy, and Vegetation Observations From Space

Abstract

The task of quantifying spatial and temporal variations in terrestrial water, energy, and vegetation conditions is challenging due to the significant complexity and heterogeneity of these conditions, all of which are impacted by climate change and anthropogenic activities. To address this challenge, Earth Observations (EOs) of the land and their utilization within data assimilation (DA) systems are vital. Satellite EOs are particularly relevant, as they offer quasi-global coverage, are non-intrusive, and provide uniformity, rapid measurements, and continuity. The past three decades have seen unprecedented growth in the number and variety of land remote sensing technologies launched by space agencies and commercial companies around the world. There have also been significant developments in land modeling and DA systems to provide tools that can exploit these measurements. Despite these advances, several important gaps remain in current land DA research and applications. This paper discusses these gaps, particularly in the context of using DA to improve model states for short-term numerical weather and sub-seasonal to seasonal predictions. We outline an agenda for land DA priorities so that the next generation of land DA systems will be better poised to take advantage of the significant current and anticipated shifts and advancements in remote sensing, modeling, computational technologies, and hardware resources.

Key Points

• Land data assimilation has shown significant promise for short-term forecasting applications

• Significant gaps remain in the current land data assimilation systems related to observation utilization and models

• Coordinated development with the modeling and observational community and adoption of technological enhancements are needed in the future

Plain Language Summary

Satellite remote sensing measurements have enabled the monitoring of the Earth's land surface with unprecedented scale and frequency. These measurements allow us to monitor the changes on the land surface and understand the contribution of human activities toward them. The information from such observations is combined with the modeled estimates through data assimilation (DA) algorithms. This article discusses the progress made in the development of land DA systems and the major gaps that remain. The paper also outlines priorities that we need to consider in the development of next generation land DA systems so that the potential of land remote sensing measurements can be fully realized.

1 Introduction and Premise

Warmer global temperatures associated with climate change have been shown to lead to more uncertainties in water availability both regionally and globally (Barnett et al., 2005; Berg et al., 2016; Konapala et al., 2020; Milly et al., 2005; Wu et al., 2020). There is increasing evidence for an uneven distribution of changes in key water flux rates such as precipitation and evaporation leading to more intense and frequent extreme events, with the lack of predictability in water availability exacerbating water scarcity (Cook et al., 2018; Oki & Kanae, 2006; Trenberth et al., 2014), water excess and natural disasters. The scientific literature shows that changes in atmospheric moisture transport will lead to consequences such as longer droughts in arid and semi-arid regions, more intense precipitation and snowfall, and earlier snow melt, among other changes (J. Huang et al., 2016; Ingram, 2016; Trenberth et al., 2003; Wainwright et al., 2021). Additionally, anthropogenic factors such as agricultural water use and industrial expansion, reservoir management, deforestation, and wetland drainage have not only quickened the pace of land transformation, but have also contributed to, often unpredictable, changes in the water, energy, and carbon cycles (Haddeland et al., 2014; McDonald et al., 2011; Mehran et al., 2017; Vorosmarty & Sahagian, 2000; Wood et al., 1997). Given the urgent need to understand the variability in water cycle fluxes, there have been several efforts to develop inventories of key water cycle fluxes (L’Ecuyer et al., 2015; Rodell et al., 2015; Trenberth et al., 2007; Vargas Godoy et al., 2021). The combined use of modeling and remote sensing estimates of water and energy states and fluxes is a key feature of these studies, which typically report closure of water and energy budgets with relatively small residuals and uncertainty (4%–8%) at coarse (monthly or larger) temporal scales. However, these studies also note that the imbalances and closure errors increase with increasing spatial and temporal resolution; consequently, concurrent high-quality Earth observations are needed to further constrain the modeled estimates and enable reliable assessment of water, energy, and carbon cycle budgets.

Equally important is the vegetation state, which is also a key factor in the coupling of the water, energy, and carbon cycles. Vegetation exerts key controls on surface water and energy partitioning and the associated response of terrestrial water, energy, and carbon fluxes to climatic drivers (Powell et al., 2013). Changes in vegetation impact surface albedo and surface roughness, interception of precipitation, and reduction in surface runoff (L. Zhang et al., 2015) and can explain up to 30% of the variance in precipitation and surface radiation (Green et al., 2017). Accurately representing vegetation processes in land surface models (LSMs) is thus critical for capturing the efficiency of carbon uptake and transpiration and—more importantly—how this efficiency is expected to change in a future climate.

Despite the importance of biospheric processes, their representation in many LSMs does not accurately capture observed vegetation behavior. The modeled biosphere-atmosphere coupling is much weaker than in observations, in part because the models generally underestimate the observed response of the biosphere to climate (Green et al., 2017; Papagiannopoulou et al., 2017). This can lead to substantial biases in near-surface states, such as air temperature and relative humidity, and can contribute to the misrepresentation of soil moisture dynamics (Koster et al., 2009)—problems that persist at the monthly and longer timescales associated with such vegetation anomalies.

In addition to their role in hydrological and climate prediction, coupled land surface-atmosphere models are integral to numerical weather prediction (NWP) and sub-seasonal to seasonal (S2S) forecasts, with their ability to represent heterogeneity in land use, soil type, vegetation types, soil moisture, the presence of snowpack, and the impact of these factors on the land-atmosphere exchange of energy and water. For example, through the soil-plant-atmosphere continuum, the land surface conditions can control the surface heat fluxes (both latent and sensible) with consequent impacts on air temperature, the height and stability of the atmospheric boundary layer, and occurrence of precipitation (Dirmeyer & Halder, 2016; Eltahir, 1998). This means land surface initialization can influence NWP and S2S forecast performance when the land-atmosphere coupling is strong and when there is sufficient variability, and also significant memory in land surface states (Koster et al., 2004, 2010, 2021). Over recent years, regions of strong land-atmosphere coupling have been shown to have expanded, to now cover regions that never before exhibited strong land-atmosphere coupling (Dirmeyer et al., 2021). For example, soil moisture conditions have been shown to directly impact temperature and precipitation forecast skill, provided that the forecast model properly represents the processes linking land anomalies, surface fluxes, and atmospheric physics. Models with incorrect land initialization and/or poor land-atmosphere coupling behavior will have biases in temperature, humidity, and precipitation forecasts during periods of drought (Dirmeyer, 2018; Dirmeyer & Halder, 2016). Observational constraints are needed to improve such deficiencies (Dirmeyer et al., 2016).

Data assimilation (DA) is the typical approach used in Earth system modeling to combine information from observational data sources with dynamical process representations in numerical process models to develop estimates that are superior to those obtained by using just the data or model alone (Reichle, 2008). The resulting estimate from DA is typically called the “analysis” and is calculated according to the relative errors and uncertainties in the model and the observations. There is a wide variety of methods that are employed to generate such DA analyses, with the mathematical objective of minimizing the posterior error of the DA analysis, whilst being constrained by the available prior information about the system being modeled. The majority of such methods are founded in optimal control or Bayesian estimation theory (Nichols, 2010), which incorporates observations into models in a sequential or non-sequential manner (Bouttier & Courtier, 2002). While the atmosphere and ocean modeling communities (Ghil & Malanotte-Rizzoli, 1991; Lahoz & Schneider, 2014; Malanotte-Rizzoli, 1997; Navon, 2009) have led the charge in the methodological development and applications of DA, it is being increasingly adopted in land surface and hydrology modeling, particularly during the past two decades (de Rosnay et al., 2014; Houser et al., 2012; Reichle et al., 2009).

Given the importance of land surface initialization and land-atmosphere coupling to applications such as NWP and S2S forecasting, it is perhaps surprising that development of LSMs, and land-focused DA activities in particular, have been a relatively low priority for NWP and S2S development in the past. This prioritization is perhaps a reflection of how land surface modeling has developed in relation to atmospheric modeling. In an editorial piece in the American Geophysical Union's newsletter “EOS,” Dirmeyer (2018) addressed this question directly, suggesting that a lack of observations of land surface states or land-atmosphere fluxes resulted in the development of land modeling schemes “to ameliorate systematic atmospheric model errors…addressing one error by introducing additional ones” (Dirmeyer, 2018). In such a circumstance, the realistic representation of land surface states, as obtained from land DA, may lead to only minimal or disadvantageos impacts on atmospheric forecasts (Crow et al., 2020). This inconsistency was more easily overlooked when high-quality observations of land states were not available. However, the latest generation of satellite and in situ observations now allow for more rigorous verification of process-model improvements and initialization of forecasts.

In recent decades, there has been a tremendous expansion in the number of remote sensing platforms that provide observations of the land surface (Balsamo et al., 2018; Kimball, 2008; Lakshmi, 2014; McCabe et al., 2017). Over the years, monitoring capabilities from these platforms have encompassed a broad range of land surface/hydrology variables including land use, soil moisture, snow cover, snow water equivalent, land surface temperature, vegetation change/dynamics, terrestrial water storage, and surface water. The technological and scientific success of these spaceborne platforms have now yielded multidecadal observations for many variables (e.g., terrestrial water storage, soil moisture, leaf area index, snow, land surface temperature) enabling the production of long-term climate data records (Lettenmaier et al., 2015). Similarly, for some variables such as vegetation dynamics, soil moisture and precipitation, observations are now provided by multiple platforms, allowing the possibility of increased spatiotemporal coverage from remote sensing. More recent advances in land remote sensing have also enabled complementary remote sensing observations of key water and carbon cycle variables, for example, precipitation from the Global Precipitation Measurement (GPM; Hou et al., 2014) mission, surface soil moisture from Soil Moisture Active Passive (SMAP; Entekhabi, Njoku, et al., 2010) and Soil Moisture Ocean Salinity (SMOS; Kerr et al., 2010) missions, terrestrial water storage from Gravity Recovery and Climate Experiment Follow-On (GRACE-FO; Flechtner et al., 2014; Kornfeld et al., 2019) mission, Solar-induced chlorophyll fluorescence from the Orbiting Carbon Observatory-2 (OCO2; Sun et al., 2017) mission, and greenhouse gases observing satellite (GOSAT; Doughty et al., 2022).

Together, these observations facilitate the development of observation-only based water, energy, and carbon budget closure estimates (Trenberth et al., 2014). In addition to conventional space agency driven science missions, sensing platforms from commercial companies have also started to become more commonplace (Dash & Ogutu, 2016; Houborg & McCabe, 2016; Tollefson, 2017). There has been a growing recognition in the community that modeling systems that can effectively exploit such multi-sensor data are key to answering fundamental questions about global and regional water, energy, and carbon cycle changes (Durand et al., 2021). In particular, it is imperative that DA systems be designed to take advantage of this presumably “golden age” of land surface remote sensing (Schimel et al., 2019; Sellars et al., 2013; Stavros et al., 2017) and be designed and positioned to take advantage of upcoming missions as soon as data become available, such as the surface water storage from the upcoming Surface Water Ocean Topography (SWOT; Biancamaria et al., 2016) mission, Surface Biology and Geology-designate observable (Cawse-Nicholson et al., 2021) or ESA's BIOMASS (Quegan et al., 2019).

The primary objective of this commentary article is to provide an agenda for the next generation land DA systems given the significant current and anticipated shifts and advancements in remote sensing, modeling, computational technologies, and hardware resources. Note that this article discusses these priorities primarily in the context of space-borne remote sensing observations of the land surface. Accurate land characterization has major impacts on atmospheric forecasts, water availability quantification, management of water and agricultural resources, and developmental issues facing the World's population growth (Collins et al., 2013). It is also critical for capturing the long-term carbon-cycle feedbacks of the biosphere to climate and meteorology and thus making more accurate projections of vegetation, water, and energy states with elevated conditions (Canadell et al., 2021). Effective DA systems have a critical role to play in realizing the full information potential and promise of Earth science remote sensing measurements. The specific goals of this agenda paper are to:

• Define the most significant gaps in and priorities for the development of regional to global scale land DA systems for water, energy, and vegetation predictions at NWP and S2S lead times.

• Identify priorities for coordination between the modeling, remote sensing, and technology communities to improve the effectiveness of land DA.

• Provide a framework for future enhancements of DA systems.

Note that the present paper focuses on DA in the context of state estimation. Remote sensing observations can also be used to reduce errors associated with model parameters and structure. Challenges and priorities related to LSM calibration and parameter estimation, with a particular focus on coupled carbon-vegetation-water interactions related to longer-term biosphere-climate feedbacks, will be addressed in a separate companion paper (Quaife et al., in preparation). In parallel, a perspective is offered (De Lannoy et al., 2022) on how to address the increased complexity of land DA for either state or parameter estimation (or both) in the future.

2 Background

As described in Reichle (2008), the primary objectives of DA include extending the spatiotemporal coverage of infrequent observational data, science translation of raw measurements from satellites (or other data sources), and optimal merging of information from multiple sources while preserving the basic constraints on the physical system. By incorporating observational inputs, DA also helps in reducing the uncertainty of estimated land states and fluxes within Earth systems models. This is particularly important given the proliferation of models and modeling systems, leading to large uncertainties purely from differences in conceptual formulations and parameterizations (Clark et al., 2017). The observational constraints from DA are a practical approach to reduce such uncertainties, so that reliable estimates of the water and carbon cycles, and their feedbacks, can be developed.

Taking advantage of the increased availability of environmental datasets, atmospheric DA has played fundamental roles in steadily improving regional and global environmental prediction capabilities (Benjamin et al., 2018; Hersbach et al., 2020). In particular, the prediction of precipitation, short-term extreme events, and applications of nowcasting (Kain et al., 2010; Mass, 2012) have hugely benefitted from the advances in DA techniques to the point that DA is considered an integral component of NWP systems (Kalnay, 2003). Similarly, DA has been key in global and regional ocean prediction and reanalysis systems (Moore et al., 2019; Zuo et al., 2019). These communities have also pioneered the development of the majority of variational and ensemble DA techniques used in operational and reanalysis environments. These methodological advances have been adapted by the land/hydrology/biogeochemistry communities to develop similar improvements in the land surface water and carbon cycle predictions (de Rosnay et al., 2014; Fairbairn et al., 2019; Lahoz & De Lannoy, 2014; Rayner, 2010; Rayner et al., 2019; Reichle, 2008; Smith et al., 2020).

In the past several decades, there has been a significant increase in the development and application of DA to LSMs, with the majority using sequential filtering approaches. The use of variational DA approaches, which are popular in the atmospheric and ocean modeling communities, is less common in NWP and S2S land DA applications. Variational DA approaches are more widely used for optimizing vegetation and carbon cycle related parameters LSMs, with the goal of better estimating longer-term (annual to centennial) carbon stocks and fluxes (Kaminski et al., 2013; MacBean et al., 2022; Pinnington et al., 2020), although these approaches have also been used to improve LSM soil moisture predictions (Pinnington et al., 2018; Raoult et al., 2021; Scholze et al., 2016). Variational methods provide updates - by minimizing a specified cost function - that constrain the model forecasts with a set of observations. They require the use of linearized observation operators, and in some cases, a tangent linear or adjoint model that represents the sensitivity of model states backward in time (Courtier et al., 1993). Some variational applications exist for land DA that use tangent linear or adjoint models (e.g., Peylin et al., 2016; Raoult et al., 2016; Schurmann et al., 2016) or adopting a linearity assumption (Balsamo et al., 2004; Hess, 2001) but, in general, the development of such models for the land surface has been more challenging due to the tendency for LSMs to contain non-differentiable threshold processes (Reichle, McLaughlin, & Entekhabi, 2002). In variational DA systems, the temporal evolution of the background error covariance is typically not modeled, whereas Kalman and particle filtering approaches allow dynamic updates of these estimates (Rabier & Liu, 2003).

Variants of Kalman filters are often used in sequential DA systems. The Extended Kalman Filter (EKF), the nonlinear version of the Kalman filter, is used in some land surface DA environments (de Rosnay et al., 2013; Ghent et al., 2010; Rudiger et al., 2010), by employing linearized observation operators and error variance propagation formulations (Reichle, McLaughlin, & Entekhabi, 2002). The ensemble Kalman filtering (EnKF) methods (Evensen, 2003), which are Monte-Carlo approximations of the Bayesian filtering process, provide a more convenient approach that is suitable for weakly non-linear models. In addition, due to its Monte Carlo formulation, the EnKF is easy to implement and provides a convenient means for representing the wide variety of errors impacting LSMs. The use of an ensemble of model realizations in the EnKF also allows for the implicit propagation of model errors, making it more computationally tractable and scalable for large systems. As a result, the EnKF has become widely used in the land surface community. Non-sequential extensions of the EnKF that allow information updates backwards over a time window have also been developed (Evensen & Leeuwen, 2000). Detailed methodological descriptions of various DA approaches are discussed in several prior review papers, including (Houtekamer & Zhang, 2016; Lahoz & De Lannoy, 2014; Lahoz & Schneider, 2014; Reichle, 2008; van Leeuwen, 2009; van Leeuwen et al., 2019).

Land DA applications have been examined for a wide range of land surface water, energy, and carbon variables. Given the developmental legacy of LSMs as the boundary conditions to atmospheric models, a majority of the land DA efforts are focused on constraining variables such as soil moisture and snow that have relatively long-memory and a direct impact on the NWP initialization skill (Benjamin et al., 2022; de Rosnay et al., 2014; Gomez et al., 2020; Santanello et al., 2019). Many NWP centers currently employ some form of screen level assimilation of synoptic measurements of air temperature and relative humidity close to the surface (Mahfouf, 1991), updating soil moisture states within DA. Motivated by the need for improved land characterization for short-term forecasts and hydrology applications, a wide variety of studies have focused on the assimilation of more direct information of soil moisture from ground and spaceborne platforms, with a range of DA techniques (De Lannoy & Reichle, 2016a; De Lannoy et al., 2019; Draper et al., 2012; Drusch, 2007; Houser et al., 1998; Kolassa, Reichle, & Draper, 2017; Kolassa, Reichle, Liu, et al., 2017; Kumar et al., 2012; Lievens et al., 2015, 2016, 2017; Margulis et al., 2002; Parajka et al., 2006; Reichle et al., 2019; Renzullo et al., 2014; Rodriguez-Fernandez et al., 2019; Sabater et al., 2007). These studies encompass the assimilation of soil moisture measurements and retrievals from passive microwave satellite platforms ranging from Special Sensor Microwave Imager/Sounder (SSMI/S) in the early 1990s to more recent missions such as SMOS and SMAP that rely on L-band microwave sensors. While less numerous, other studies have focused on the assimilation of root-zone soil moisture proxy information derived from thermal-infrared remote sensing (e.g., Hain et al., 2012).

In addition to soil moisture, assimilation of snow cover and snow water equivalent (SWE) relevant measurements have also been examined with approaches ranging from direct insertion to variants of Kalman filtering and smoothing. Snow cover observations from optical sensors, which do not directly provide quantitative information on the amount of snow in the snowpack are often assimilated using rule-based update methods (Arsenault et al., 2013; De Lannoy et al., 2012; Rodell & Houser, 2004; Zaitchik & Rodell, 2009). A few studies have also used the EnKF to assimilate snow cover observations (Clark et al., 2006; H. Su et al., 2008; Toure et al., 2018; Y.-F. Zhang et al., 2014). These studies typically demonstrate modest improvements for estimating ephemeral SWE and little benefit over deeper snowpacks, owing to the fact that the information about SWE in instantaneous snow cover observations tends to diminish for larger values of SWE, as snow cover saturates at 100%. To address the weak instantaneous correlation between snow cover and SWE, a number of studies (Durand et al., 2008; Girotto et al., 2014; Margulis et al., 2015; Oaida et al., 2019) employed smoothing approaches that assimilate all snow cover observations within a time window. Estimates of SWE from ground measurements, microwave, and lidar platforms have also been widely used for DA in snow hydrology studies (Andreadis & Lettenmaier, 2006; C. Huang et al., 2017; Kumar, Peters-Lidard, Arsenault, et al., 2015; Liu et al., 2013; Magnusson et al., 2014; Margulis et al., 2015, 2019; Slater & Clark, 2006; Smyth et al., 2019). Efforts to assimilate brightness temperature observations from microwave instruments and radar backscatter measurements using machine learning and radiative transfer model operators for improving snow estimation have also been reported over different regions of the world (Kwon et al., 2019; Park et al., 2021; Phan et al., 2014; Xue & Forman, 2017; Xue et al., 2018). Consequently, some operational centers around the world have enabled a form of land analysis to incorporate remote sensing information for NWP initialization (Barrett, 2003; Carrera et al., 2019; de Rosnay et al., 2013, 2014; Gomez et al., 2020; Pullen et al., 2011; Wegiel et al., 2020). These analyses also range from optimal interpolation to Kalman filter methodologies.

In addition to work within NWP and with offline LSMs, there exists a large set of applications that rely on accurate estimates of land surface conditions. Water resource planning efforts require reliable estimates of water fluxes and their variability to account for water availability issues and hydrological extremes. Consequently, these inputs are also becoming critically important for agricultural and food security assessments. Military agencies rely on accurate land characterization of surface states for mobility and trafficability assessments. Water storage in peatlands controls large carbon fluxes to the atmosphere. Estimates of river flow are important for reservoir management as well as transboundary water applications. Given these significant needs for land surface hydrology information, there have been land-focused systems (sometimes called ’offline’ environments) developed in the past several decades. The Data Assimilation Research Testbed (DART; Anderson et al., 2009), developed at the National Center for Atmospheric Research has enabled ensemble DA for the land surface schemes of several Earth system models. The Global Land Data Assimilation System (GLDAS; Rodell et al., 2004) and the North American Land Data Assimilation System (NLDAS; Mitchell et al., 2004) were the two pioneering efforts that established software frameworks for driving LSMs with observationally constrained meteorology, with the eventual goal of enabling the assimilation of land surface states. Other examples of such land-only simulation products include MERRA-Land (Reichle et al., 2011), ERA-Interim/Land (Balsamo et al., 2015), and ERA5-Land (Munoz-Sabater et al., 2021), which supplement major atmospheric reanalysis products. Another land-only product is LDAS-Monde (Albergel et al., 2017, 2020), which jointly assimilates surface soil moisture and leaf area index. Moreover, the land-only SMAP Level-4 Soil Moisture product focuses on the assimilation of SMAP radiance observations (Reichle et al., 2019). A flexible environment called the Land Information System (LIS; Kumar et al., 2006) was developed at NASA that allows for the interoperable use of multiple land and hydrology models and formal DA capabilities around them (Kumar et al., 2008). The capabilities of LIS are not only used in both GLDAS and NLDAS, but have also fostered many LDAS configurations and instances in different parts of the world (Erlingis et al., 2021; Goncalves et al., 2009; Kumar et al., 2018; McNally et al., 2017). The development of similar LDAS environments has also been reported in other organizations around the world (Albergel et al., 2017, 2020; Carrera et al., 2015; Lewis et al., 2012; Reichle et al., 2014; Sawada et al., 2015).

The development of land-only DA environments has enabled the assimilation of other land hydrology variables, including some that are not viable for operational NWP needs because of their long latency. For example, Terrestrial Water Storage anomalies from gravity missions such as GRACE have been used in DA configurations to develop inferences on subsurface changes (Girotto et al., 2017; Kumar et al., 2016; B. Li et al., 2012; Syed et al., 2008; Zaitchik et al., 2008; Zhao & Yang, 2018) The use of GRACE data has provided unprecedented information about the changes in Earth's groundwater storage and has uncovered unsustainable exploitation from groundwater pumping in many areas of the world (Lo et al., 2016; Nie et al., 2019; Rodell et al., 2009, 2018; Thomas & Famiglietti, 2019). The use of GRACE-based constraints also has enabled critical inputs for drought monitoring efforts, by providing an observation-informed estimate of groundwater changes, a critical water resource that varies at longer timescales than surface water (Getirana et al., 2020; Houborg et al., 2012; B. Li et al., 2019).

With the availability of prognostic representations of carbon processes within the third and fourth generation LSMs (Fisher & Koven, 2020; Pitman, 2003), there have been several efforts to utilize remotely sensed vegetation datasets through DA (e.g., in the aforementioned LDAS-Monde system). There is a long legacy of high-resolution measurements of canopy states from optical sensors such as leaf area index, normalized difference vegetation index, fraction of photosynthetically active radiation, and vegetation biomass. Assimilation of leaf area index to improve the estimation of crop yields, vegetation biomass, root zone soil moisture, and carbon fluxes has been demonstrated in several prior studies (Barbu et al., 2011; Demarty et al., 2007; Dente et al., 2008; Fox et al., 2018; Jarlan et al., 2008; Kumar et al., 2019; Nearing et al., 2012; Quaife et al., 2008; Raczka et al., 2021; Sabater et al., 2008). At coarser resolution, analogs of vegetation changes can also be obtained from passive microwave radiometry (Feldman et al., 2021; Konings et al., 2016). While optical sensor measurements are subject to coverage limitations from cloud obscuration, microwave measurements are nearly all-weather. The assimilation of vegetation optical depth estimates from microwave measurements has also shown promising results in improving land surface states, and it provides the opportunity to extend the spatiotemporal coverage of higher-resolution optical measurements (Kumar, Holmes, et al., 2020; Mucia et al., 2021). Many of these studies have explicitly shown the significant promise of vegetation assimilation in capturing human management impacts such as agriculture, where anthropogenic activities lead to seasonal changes in vegetation growth (Mocko et al., 2021). Vegetation assimilation has also shown effectiveness in capturing vegetation disturbance features from fires, which often lead to changes in regional hydrological response (Kumar et al., 2021, 2022). DA offers a more practical approach to representing these changes on the land surface, which are difficult to characterize with process-based models alone (Pongratz et al., 2018).

Other reported land DA instances involve variables such as land surface temperature, albedo, and water surface elevation. Land surface temperature and albedo from thermal and optical sensors are variables at the heart of the surface energy balance and influence the latent and sensible heat fluxes at the land-atmosphere boundary. Therefore, assimilation of these variables should have a direct influence on weather and climate model forecast accuracy. Unlike variables such as soil moisture, snow, terrestrial water storage, and vegetation, the response timescale of land surface temperature is more rapid, with associated diurnal and seasonal cycles. Additional challenges related to bias correction at these finer timescales are described in studies of land surface temperature assimilation, which have employed both variational and filtering techniques (Bosilovich et al., 2007; Castelli et al., 1999; Draper et al., 2015; Ghent et al., 2010; Han et al., 2013; Lakshmi, 2000; Lu et al., 2017; Reichle et al., 2010). Comparatively, fewer studies examine the assimilation of albedo, despite the classification of albedo as an essential climate variable (Hollmann et al., 2013), particularly over large spatial scales. Most of the reported studies employ direct insertion approaches due to the lack of prognostic representation of albedo variables within LSMs and have reported some positive impacts on characterizing surface fluxes and snow states (Boussetta et al., 2015; Kumar, Mocko, et al., 2020; Malik et al., 2012; Yin et al., 2016).

In addition to NWP and S2S, improving the initial conditions for hydrological forecasting applications through DA has also been the subject of many studies (e.g., Baugh et al., 2020; Liu et al., 2012). Most of these focus on improving soil moisture (e.g., Loizu et al., 2018) and snow states (e.g., D. Li et al., 2019), though a few studies have also examined the assimilation of hydrological states and fluxes such as discharge (Clark et al., 2008; DeChant & Moradkhani, 2011; Fairbairn et al., 2017; Pauwels & De Lannoy, 2009; D.-J. Seo et al., 2009; Thirel et al., 2010). Estimates of water stage or elevation have been derived from radar altimeters since the early 1990s, providing nearly all-weather observations (Silva et al., 2010). The use of DA within hydrodynamic models has been examined to extend the spatiotemporal utility of these typically narrow swath altimeter measurements. These studies include efforts to assimilate river discharge, velocity, and water level (Andreadis et al., 2007; El Gharamti et al., 2021; Emery et al., 2020; Ercolani & Castelli, 2017; Neal et al., 2007; Paiva et al., 2013; Ricci et al., 2011). The improved surface water characterization over data-poor transboundary basins from the assimilation of remote sensing data is a highly relevant hydrological modeling challenge not routinely employed in hydrological prediction systems.

3 Major Gaps in Land Data Assimilation

As described in the previous section, there have been significant advances in the development of methodologies for land DA, the utilization of land surface measurements from various spaceborne platforms as well as in situ observations, and the use of land DA for routine applications. Despite this progress, there are several significant remaining gaps and challenges, which are described in this section. Figure 1 shows a summary of the key topics described in this section.

Figure 1

List of the major challenges and gaps in land data assimilation discussed in this article.

3.1 Reliance on the Assimilation of Retrieval Products

All DA methods require an “observation operator” or forward model to translate variables from model space to observation space, and vice versa. The simplest observation operator is the identity matrix where the assimilated value is either an observation (in situ soil moisture, atmospheric temperature) or a retrieval from (typically remotely sensed) observations (e.g., soil moisture, snow mass, or leaf area index) of a variable that is also estimated by the LSM. To date, most of the reported land DA studies and applications have focused on the assimilation of retrieved products, whereas the atmospheric and ocean communities have made far more progress in the direct use of radiance measurements. The retrievals are obtained using radiative transfer or empirical models with their own underlying assumptions of parameter, land surface, and climate characteristics. These assumptions are often inconsistent with those simultaneously made in LSMs that are utilized in DA instances. In many cases, retrieval models are developed and calibrated with limited airborne and field campaign data. Subsequently, their application over different or large spatial extents leads to errors. The assimilation of satellite radiances as in the SMAP L4_SM algorithm (Reichle et al., 2019) is likely to reduce errors stemming from such inconsistencies and potentially make observation errors more easily quantifiable (de Rosnay et al., 2022; Lewis et al., 2012; Lievens et al., 2017), though forward radiative transfer models also suffer from uncertainties related to conceptualization and parameterization. As noted earlier, given the complexity of forward modeling for snow mass, machine learning models have also been used as observation operators for snow DA studies.

The historical reliance on retrieval products is a significant obstacle for the more widespread adoption of land DA in near-real-time or operational modeling environments. This is partly due to the fact that latency associated with retrieval products often does not meet the constraints of operational environments. Radiance assimilation also provides an opportunity to update states impacting the forward modeling of land surface emission (e.g., vegetation states controlling canopy opacity). To realize the more common use of radiance measurements within land DA environments, relevant forward modeling tools, whether they use physically based radiative transfer models (e.g., De Lannoy et al., 2013; Quaife et al., 2008) or employ statistical approaches (e.g., Aires et al., 2021; Kolassa et al., 2018), are needed. Compared to the breadth of LSMs, only a handful of land relevant community forward modeling systems and tools are currently available (Parrens et al., 2014; Royer et al., 2017).

3.2 Limitations From Developmental Legacies

The legacy of LSMs has also significantly influenced the application of land DA instances. As mentioned earlier, the first generation LSMs were conceptualized as the boundary conditions to atmospheric models with the primary objective of improving atmospheric prediction. The focus of ensuring physical realism of land surface states has historically been a secondary priority. For example, while most of the major NWP centers around the world maintain detailed scorecards of prediction skill of atmospheric variables (Brown et al., 2021; Ebert et al., 2013; Haiden et al., 2018), such routine evaluations for land surface variables are uncommon. Consequently, process representations, and therefore the climatologies of variables such as soil moisture, have evolved specifically for each associated NWP system. It is well documented that soil moisture climatologies from different models are starkly different (Reichle et al., 2004), to the point that they are essentially model-specific representations of soil wetness (and should not even be called soil moisture; Koster et al., 2009).

As noted earlier, the screen level assimilation approaches used in many NWP centers update soil moisture states instead of atmospheric states assuming a causative relationship between soil moisture boundary conditions and low-level atmospheric forecasts. Since these assumptions are only valid when weakly forced atmospheric conditions are present, the screen level DA scheme to adjust soil moisture often has become an approach that compensates for true underlying errors in other parts of the model (Draper et al., 2009). It is recognized that the resulting improvements in the low-level atmospheric forecasts are not necessarily due to improved representation of land surface states (Douville et al., 2000; Drusch & Viterbo, 2007), and may in fact degrade the surface states and hydrological consistency (e.g., Zsoter et al., 2019). Consequently, the assimilation of observations or retrievals which have a direct relation to soil moisture becomes problematic in such DA environments. For example, studies have demonstrated that assimilation of satellite soil moisture data and screen level temperature and humidity observations often lead to diverging estimates of root zone soil moisture updates (Draper et al., 2011), though satellite soil moisture assimilation itself improves land surface states (Carrera et al., 2019; Munoz-Sabater et al., 2019). As we noted above, with the evolution of LSMs, they are also increasingly being used in environments where there is a direct need for accurate characterization of land surface, without being necessarily connected to atmospheric models and requirements. As the need to improve the physical realism of land surface states increases and direct measurements of land surface states become increasingly available, methodologies that treat the land surface states as an “error sink” are fundamentally undercutting the potential of land DA. Localization strategies (Anderson, 2007) that selectively adjust portions of the model states most closely related to the observation may also be needed to circumvent such limitations.

3.3 Bias Correction and Efficiency Issues

Another key issue related to the lack of physical realism of LSMs is the current practice of bias correction within land DA environments. In real data assimilation systems, systematic errors (including biases) between model estimates and observations are unavoidable, and they are typically caused by a combination of errors from instrument noise, retrieval issues, and model deficiencies, related to model structure and uncertainties in parameters and inputs. The model structural deficiencies are also tied to the legacy issues mentioned in Section 3.2. Additionally, the lack of physical realism in LSM estimates also stems from the difficulties of the complex heterogeneity of the land surface, which makes it difficult to develop spatially distributed assessments of biases from limited in situ measurements (that are not spatially representative). Prior studies have explored both bias-aware and bias-blind approaches in land DA (Kumar et al., 2012). Bias-aware approaches typically attribute the source of the biases exclusively to the model or the observations and use the analysis increments to progressively estimate the bias (Bosilovich et al., 2007; De Lannoy et al., 2007; Reichle et al., 2010). While these approaches are useful for systems with transient changes in bias, the lack of realism in the model estimates or observations makes their application difficult. Bias-blind approaches are more common in land DA systems, where the relative biases between model estimates and observations are removed before assimilation, essentially focusing on the correction of errors in short-term or interannual variations. Approaches such as Cumulative Distribution Function (CDF)-matching and standard normal deviate scaling are commonly used in soil moisture assimilation systems to rescale the observations into the model climatology prior to assimilation (Crow, Koster, et al., 2005; de Rosnay et al., 2020; Reichle & Koster, 2004). The observation rescaling approaches require long data records of observation and modeled soil moisture estimates to develop scaling parameters, which are difficult with new satellite missions. Similarly, long model integrations with a consistent driving meteorology are required to establish the model climatology, which also often do not exist for operational environments because of frequent system updates. Moreover, regional modeling centers also often change their domains of focus, which makes the reliance on an established soil moisture climatology specific to each modeling domain impractical. The rescaling approaches are also documented to reduce the information transfer from retrievals by about 10% due to errors in empirical CDF computations (Nearing et al., 2018) and violation of theoretical assumptions underlying the matching of higher-order statistical moments in the CDF approach (Yilmaz & Crow, 2013).

Further, recent studies have shown that the rescaling approaches are problematic when stationarity assumptions about the model-observational biases do not hold true, dynamic changes in bias occur, or when human management factors or unmodeled processes are present in the observational signal and missing from the models (Girotto et al., 2017; Kumar, Peters-Lidard, Santanello, et al., 2015). In this regard, the land surface is unique compared to the atmosphere and ocean, as the direct influence of human activities (e.g., irrigation, vegetation disturbances, groundwater pumping, urbanization, reservoir management) is ubiquitous on the land. Because many of these processes are subjective in nature, their accurate representation within models is hard. For example, though irrigation representations have been developed within LSMs, the determination of irrigation onset, duration, and magnitude is difficult to specify accurately (Massari et al., 2021). The use of remote sensing observations through DA offers a more practical approach to incorporating such impacts, without the reliance on heuristic rules. However, as the unmodeled signals are often manifested as systematic errors or biases, new approaches are needed that can identify the root causes of bias, including bias from systematic instrument errors, model parameter and parameterization deficiencies, and unmodeled processes; otherwise, it will remain difficult if not impossible to preserve important observational signals (Kumar, Peters-Lidard, Santanello, et al., 2015).

3.4 Methodological Limitations

As noted earlier, owing to the difficulties in deriving adjoint models for highly non-linear LSMs, Kalman filtering approaches have become the most commonly used methods for land DA, providing a computationally efficient alternative to variational DA systems with comparable performance (Caparrini et al., 2004; Kotsuki et al., 2022; Reichle, Walker, et al., 2002). One of the key assumptions in the serial applications of the EnKF is that model and observation errors are Gaussian and mutually and serially uncorrelated (Katzfuss et al., 2016), which are often not realistic in land DA systems. The particle filter (Moradkhani et al., 2005; Smyth et al., 2019) overcomes this limitation, by approximating the model posterior distribution with Monte Carlo sampling, but requires large ensemble sizes (Dong et al., 2015; Weerts & El Serafy, 2006). Another possibility are hybrid ensemble-variational techniques (Bannister, 2017), which solve the variational problem by approximating required properties of the adjoint and tangent linear using a model ensemble and are being increasingly used in NWP. Hybrid approaches also show promise for overcoming some of the limitations inherent in other ensemble DA techniques (such as limited sample size), and have been applied to land surface modeling problems (e.g., Pinnington et al., 2021).

However, all of these ensemble DA systems can suffer from issues of filter collapse and degeneracy when adequate ensemble spread does not exist, which is sometimes addressed with techniques such as covariance inflation (Anderson & Anderson, 1999; Fox et al., 2018; Gharamti, 2018, 2021). These techniques also require significant tuning and knowledge of observation error covariances (Miyoshi & Yamane, 2007). The reliance on the ensemble to derive the model error covariances can be limiting in some instances when the state variable is bound by theoretical limits. For example, since fractional snow cover is constrained by zero as the lower limit and unity as the upper limit, the effective generation of spread in the ensemble becomes problematic when the model estimate is close to these bounds and has been shown to cause disadvantageous impacts within DA (Arsenault et al., 2013). Reliance on a single choice of surface meteorological forcing has also been shown to underestimate the land model error covariance and reduce the efficiency of land DA (Kumar et al., 2017). Studies have also shown that DA can lead to water balance errors (which are particularly important in coupled land-atmosphere systems) when the analysis increments are not explicitly constrained to conserve the water balance (Girotto et al., 2021). The use of weak constraints in land DA systems has been explored to reduce these inefficiencies (Pan & Wood, 2006; Yilmaz et al., 2011). Finally, most DA systems are designed to work with errors that are strictly random. Since systematic errors are more dominant in land hydrology processes, the primary focus on improving short-term errors through DA often limits the level of improvements possible through land DA (Section 3.3).

3.5 Lack of Information Transfer in Land Data Assimilation Efforts

A key appeal of DA is the potential to extend the information in observations to other model states that are connected to the observations. Land DA efforts have yielded mixed results in effectively realizing this goal, due to a variety of reasons. For example, the assimilation of surface soil moisture retrievals has been widely used to develop improvements in deeper soil moisture, with studies reporting varying degrees of success. The low skill of soil moisture retrievals from older sensors was a key factor in early soil moisture DA studies that reported small improvements in root zone estimation. Larger improvements in subsurface soil moisture estimation with DA have been reported with data from newer L-band sensors such as SMOS and SMAP (De Lannoy & Reichle, 2016a; Reichle et al., 2017). Similarly, larger improvements from DA are often reported with studies involving older or simpler soil moisture models (F. Li et al., 2010; Mladenova et al., 2019), as the information from remote sensing data compensates for the lack of skill in those models. When provided with high-quality boundary conditions, modern LSMs are fairly skillful in their soil moisture estimates. As a result, the added improvements from soil moisture DA featuring such models and high quality precipitation inputs are relatively small (Kolassa, Reichle, Liu, et al., 2017; Kumar et al., 2014; Reichle, Zhang, et al., 2021; E. Seo et al., 2021). The strength of the surface-subsurface soil moisture coupling within LSMs is a significant factor in the level of improvements obtained in the root zone through surface soil moisture assimilation (Kumar et al., 2009). Given that the true surface-subsurface coupling strength is largely unknown and significantly heterogeneous, these studies suggest that there is a strong model configuration dependence on the level of improvements in root zone soil moisture realized through surface soil moisture assimilation.

Attempts to extract information from the assimilation of remote sensing soil moisture observations about associated key water cycle variables such as evapotranspiration (ET), runoff, and river discharge have also produced mixed results. For example, most soil moisture and vegetation assimilation studies that examined the impact on ET and/or river discharge either report marginal improvements or degradations (Albergel et al., 2017; Bonan et al., 2020; Fairbairn et al., 2017; Kumar et al., 2018; Munoz-Sabater et al., 2019; Peters-Lidard et al., 2011). Similar to the earlier example on root zone soil moisture, the information transfer issues with other variables such as ET are also related to the inherent assumptions of coupling between the relevant variables (Crow et al., 2018, 2020). In reality, such coupling relationships are dependent on a number of factors including land surface characteristics, surface meteorology, and climate conditions (Dong, Dirmeyer, et al., 2020). For example, soil moisture has a strong influence on ET in a water limited domain but may not be the primary controlling factor in an energy limited domain (Kumar, Holmes, et al., 2020). Similarly, LSMs often underestimate the coupling between simulated soil moisture and subsequent surface runoff and thus potentially squander valuable streamflow predictability associated with soil moisture assimilation (Crow et al., 2017, 2018, 2019). Consequently, the results from DA will be misleading if the LSM does not adequately represent such coupling relationships accurately, even when high quality observations are assimilated. The impact of such inherent model coupling features is also reflected in studies assimilating observations related to vegetation that have shown stronger impact on root zone soil moisture than from the assimilation of surface soil moisture retrievals (Albergel et al., 2017, 2020; Kumar, Holmes, et al., 2020).

In other cases, the lack of success in information transfer is related to a lack of adequate model prognostics or limited physics. For example, the assimilation of land surface temperature within LSMs has been problematic because there are large discrepancies in the representation of surface temperature in models and the skin temperature derived from remote sensing instruments (Bosilovich et al., 2007). Partly due to the lack of an equivalent model prognostic, the land surface temperature assimilation results have yielded little improvement in surface fluxes (Reichle et al., 2010). Similarly, though terrestrial water storage retrievals from the GRACE mission have provided valuable insights on subsurface water storage changes around the world, their assimilation may produce erroneous trends because most LSMs only include a shallow groundwater representation (whereas GRACE may be observing deeper groundwater storage signals; Girotto et al., 2017).

Another key factor in the lack of information transfer is the limited skill in observations themselves. Though there is often a direct connection between streamflow and melt from the snowpack, little success in this area is reported from snow DA studies that solely rely on remote sensing observations (Andreadis & Lettenmaier, 2006; Kumar et al., 2014; Zsoter et al., 2019). On the other hand, the use of more reliable data from airborne or ground measurements of snow has shown greater promise in improving streamflow (C. Huang et al., 2017; Lahmers et al., 2022), suggesting that significant improvements in remote sensing snow retrievals are needed to improve their utility. As noted above, despite the availability of multi-decadal observations of land surface variables such as land surface temperature, albedo, and leaf area index, these observations were used in relatively few DA studies, mainly because LSMs have large model biases or have not considered and developed appropriate prognostic representations of such variables.

3.6 Spatiotemporal Characterization of Error Specifications

The specification of model and observation errors is a critical factor in the performance of land DA systems and remains a significant challenge (Crow & Loon, 2006). Observation errors must take into account instrument errors, errors in observation operators or retrieval algorithms (considered synonymous with representativeness errors [van Leeuwen, 2015]); consequently, observation error parameters are difficult to measure and specify in a spatially distributed manner. Though spatial correlations are present in the observation errors, they are often ignored in DA environments (Ying, 2020). Model error specification must consider limitations in model physics and uncertainties in boundary conditions and parameters, among other factors. It is generally recognized that it is more difficult to specify model errors than observation errors because the general lack of physical realism in most LSM outputs makes it difficult to perform direct comparisons against available ground/truth measurements (Reichle, 2008). Additionally, the significant heterogeneity and complexity of land processes makes it difficult to develop spatially distributed estimates of model errors. Because of these limitations, the derivation of model error covariance specifications is often done through idealized experiments and largely remains a subjective process (Carrera et al., 2015; Fox et al., 2018; Kumar et al., 2008, 2009; Raczka et al., 2021). In fact, most DA studies to-date employ domain-wide uniform error specifications. DA diagnostics such as innovations and analysis increments are routinely used to evaluate error assumptions in DA configurations, by examining their deviations from the theoretical ideal values (e.g., Reichle et al., 2017). These methods have also been employed within DA systems to adaptively diagnose and improve error variance specifications (Crow & Reichle, 2008; Reichle et al., 2008). Similarly, the application of the triple collocation approach has also been shown to improve the estimation of soil moisture retrieval errors by removing the problematic assumption—required by innovation analysis—that errors in assimilated observations are serially uncorrelated (Crow & Berg, 2010). These theoretical approaches, however, become less effective when observational signals are dominated by unmodeled processes, which can be especially true for land DA systems (Kumar, Peters-Lidard, Santanello, et al., 2015). In land-ensemble filtering, the forecast ensemble spread is typically created by adding small perturbations sampled from randomly generated noise to the model states at regular time intervals. Recent studies have shown that this approach creates unrealistic estimates of forecast uncertainty, since the ensemble spread reflects the local model persistence of the applied perturbations, which is not necessarily an accurate representation of the model uncertainty (Draper, 2021). Explicit consideration of uncertainty from several contributing sources for more realistic model spread representation has been reported in some studies (Dokoohaki et al., 2022; Raeder et al., 2021). Additionally, the model background error can be underestimated due to the limitations of the specific model configurations (Kumar et al., 2017).

3.7 Limitations in the Use of Metrics and Evaluation Strategies

The typical approach in DA studies is to use the available (often point-scale) ground measurements as the truth for evaluating the impact of DA, with the inherent assumption being that the spatiotemporal scale of the model and remote sensing data is sufficiently fine to contain information about the in situ measurements. This assumption may not always be valid, as shown in studies that have quantified information use efficiency within DA systems (Nearing et al., 2018). The assessments versus in situ measurements also overlook the larger utility of remote sensing datasets to provide spatially distributed coverage, offering the possibility to capture spatial features and heterogeneity that cannot be obtained with in situ measurements. Finally, the geographic distribution of in situ stations is skewed toward data-rich regions, where model performance is generally good, whereas the impact of land DA is generally larger in data-sparse regions, which was demonstrated using instrumental variable approaches (e.g., Dong et al., 2019; Reichle, Zhang, et al., 2021), including the aforementioned triple collocation method (Gruber et al., 2020). Therefore, spatially distributed evaluation strategies and metrics must be emphasized in demonstrating the potential utility of remote sensing datasets. For example, the impacts of agricultural activity and man-made disturbances such as fires are hard to represent in models or capture through ground measurements. Assimilating vegetation datasets, on the other hand, has been effective in representing such features over large spatial scales (Kumar, et al., 2021, 2022; Kumar, Holmes, et al., 2020; Mucia et al., 2021).

Additionally, when representativeness differences are significant with variables such as soil moisture, DA evaluations mainly focus on the use of temporal anomaly metrics (Entekhabi, Reichle, et al., 2010). The restricted focus on the evaluation of temporal variability has been shown to underestimate errors in soil moisture datasets (Dorigo et al., 2010) and miss out on capturing the impact of unmodeled processes (Kumar, Peters-Lidard, Santanello, et al., 2015). Evaluation strategies based on approaches such as information theory (Shannon, 1948), as used for instance in NWP context (Cardinali, 2009), that allow for the spatially distributed quantification of the information transfer and noise reduction can be considered to better demonstrate the utility of land remote sensing and DA (e.g., Balsamo et al., 2007). Since the land is particularly impacted by anthropogenic impacts, land DA must also focus on demonstrating the characterization of such processes, beyond simply improving processes driven by natural variability. It must be stressed that some of these evaluation limitations are due to a profound lack of observational capability for many land surface variables. For example, the point-to-satellite footprint representativeness issues for soil moisture are more severe for the representation of spatial patterns than for temporal anomaly statistics (Chen et al., 2019). Datasets or observational techniques that allow for a direct evaluation of soil moisture patterns are generally lacking.

3.8 Multivariate and Land Reanalysis Efforts

While there have been numerous efforts to develop consistent multi-decadal estimates of atmospheric and ocean states through reanalysis of all available remote sensing datasets, such efforts are largely non-existent in the land and hydrology community. This is partly because land DA development has historically lagged behind DA development in the atmosphere and ocean communities, despite the availability of long land surface and hydrology observational data records. In fact, multivariate and concurrent DA of land variables have only been reported in more recent years. An obvious beneficial impact from multi-sensor and multivariate DA setups has been in improving the temporal coverage of observational inputs within the DA environment. For example, since 2010, soil moisture retrievals from multiple passive and active microwave instruments have become available, which improves the temporal observational coverage from remote sensing. Similarly, vegetation conditions are available from both optical/thermal infrared and microwave sensors, with each contributing unique advantages (Section 2). Combining both types of vegetation measurements can help in providing a more continuous observational constraint with a DA environment (Kumar, Holmes, et al., 2020; Mucia et al., 2021). This can be particularly important when temporally continuous coverage of land surface conditions is needed for monitoring of hazards such as fires and related vegetation disturbances (Kumar et al., 2021).

The concurrent handling of complementary information from multiple observation types is a significant challenge in multivariate DA environments (Montzka et al., 2012). Because soil moisture and snow are important in warm and cold seasons, respectively, the assimilation is, for the most part, mutually exclusive and thus relatively straightforward. The development of a reanalysis that incorporates soil and snow depth from multiple sensors across three decades has demonstrated improvements in land surface states, particularly with more modern sensors (Kumar et al., 2018). Resolving the disparities in temporal and spatial resolution differences across observational products is another challenge in multivariate DA systems (Tian et al., 2017). Generally, studies have reported added improvements from multivariate DA than those from corresponding univariate configurations (Abbaszadeh et al., 2020; Hain et al., 2012; H. Su et al., 2010; Zhao & Yang, 2018). Multivariate observational constraints may be necessary in instances where observations provide unique insights into related processes. For example, univariate assimilation of terrestrial water storage or soil moisture in places where groundwater pumping is prevalent may produce incorrect results, while their joint assimilation provides more consistent improvements across relevant water storage terms (Girotto et al., 2019; Tangdamrongsub et al., 2020). Given the simultaneous impact from anthropogenic processes and the interlinked nature of water cycle processes, emphasis on multivariate DA environments will be needed for highlighting the utility of land DA. There is also a significant need to develop land reanalysis products so that an observation-informed spatially and temporally continuous climate record of water, energy, and carbon cycle changes on the land surface can be established.

3.9 Coupled DA Environments

Since the initial development of land DA was motivated by the need to improve NWP, most applications of land DA are limited to providing better initial conditions prior to a forecast. As noted earlier, such independent development of approaches can lead to divergent developmental priorities and inconsistent results with land and atmosphere systems. Coupling between the respective DA systems is needed to reduce the imbalance between them. Generally, there are two types of coupled DA systems (Penny & Hamill, 2017). In a weakly coupled DA system, communication and data exchange between the land and the atmosphere only occurs through the physics within a coupled forecast model, which is the land DA approach commonly used in the context current NWP and reanalysis development (Carrera et al., 2019; de Rosnay et al., 2013; Draper & Reichle, 2019; Gomez et al., 2020; Reichle, Liu, et al., 2021). A strongly coupled DA environment, on the other hand, facilitates cross-component interactions during DA, to allow the use of cross-model error covariances during DA (Shahabadi et al., 2019). Though there is a recognition that strongly coupled DA systems enable increased exploitation of the information from observations and are better at addressing the imbalance issues, no such operational systems exist currently (Sluka et al., 2016). Coupled DA approaches are going to be pursued for their consistent surface-atmosphere initialization (for a “spatial-consistency” target), beneficial for NWP prediction or within a climate reanalysis. They, however, carry a substantially higher computational burden. Land-only DA systems are, therefore, likely to thrive and provide valuable and complementary solutions for frequent land reanalysis and permitting to initialize past reforecasts in support of anomaly based predictions, such as those produced in the S2S framework (for a “time-consistency” target). While land DA and coupled DA approaches may have important methodological differences, they do face common challenges related to model and observation errors, characterization, handling of biases, quality control, compensating coupling errors (Vitart et al., 2019). Therefore several of the priorities identified hereafter are crosscutting.

4 Future Priorities, New Opportunities, and Recommendations

Despite the challenges and gaps described above, DA methodologies are vital tools for realizing the promise of land surface observations. In this section, we outline the priorities for advancing land DA including the need for improvements in models and observations, co-development of models with the DA community, and embracing technological advancements, so that land DA environments can not only help to realize the full potential of land surface observations, but also help define the next generation observational needs. These enhancements will likely require an iterative process, and coordination across several interdisciplinary areas, as shown in Figure 2. A number of specific recommendations to realize this vision is outlined below.

Figure 2

A vision for the next generation land data assimilation systems, with closer developmental coordination with the land surface model and observation communities, adoption of technological enhancements in machine learning and cloud computing, and advancement in the use of metrics and evaluation strategies to realize the potential of land surface observations.

4.1 Enable Multivariate, Multi-Sensor DA

As noted earlier, the number and type of space-borne observations relevant to the land surface and hydrology continue to increase. This increased availability of observations presents unique challenges and opportunities for significantly advancing land and hydrologic prediction (McCabe et al., 2017). The simultaneous availability of multiple observation types of land surface processes provides concurrent water and carbon budget constraints, and has the potential to reduce errors in water and vegetation state estimates beyond what is achievable with numerical process models alone. This is particularly relevant in regions of the world where water cycle processes are dominated by anthropogenic impacts and univariate assimilation may provide misleading results.

The wealth of observational information available from multiple platforms is a unique opportunity to embrace a “systems approach” (Jenkins, 1969) to land DA that truly acknowledges the interconnected nature of water cycle processes and the human influence by enabling multivariate and multi-sensor environments. Focused site level analyses (e.g., Fox et al., 2018) can be used to systematically diagnose the utility of specific observational inputs and to minimize representativeness errors. Such strategies can also be used to configure appropriate localization approaches to specify the extent of information transfer, and to reduce sampling errors from ensemble structures.

4.2 Develop Coupled DA

In strongly coupled DA, multiple components from a coupled model are updated in a single DA step. Strongly coupled land-atmosphere DA expands the possibilities for multivariate and multisensor DA, and its potential advantages are being actively explored. There have been significant advances in the development of coupled Earth system models, through efforts such as the Earth System Modeling Framework (ESMF; www.earthsystemmodeling.org) that provide interoperable standards and tools for the community. Similar, concerted efforts to enable such paradigms (e.g., Joint Effort for Data Assimilation Infrastructure; JEDI; Tremolet & Auligne, 2020) and provide flexible and computationally scalable software infrastructure are needed to realize the vision of coupled Earth system DA environments (de Rosnay et al., 2022). Recent studies examining strongly coupled land-atmospheric DA systems have also stressed the challenges of addressing inconsistencies in spatial and temporal error correlation scales (across the land and atmosphere), the need for reliable forward modeling tools, and the recognition of the memory and timescale differences in land and atmospheric dynamics (Draper, 2021; Lin & Pu, 2020).

4.3 Improve the Physical Realism of LSMs

Many of the limitations of land DA environments are closely related to LSM deficiencies, as discussed in Section 3.2. Therefore, there needs to be a strong emphasis on improving the physical realism of LSM process descriptions and of retrieval products so that interpretation errors and efficiency loss issues can be reduced. Calibration of model parameters can be used to reduce systematic errors in models prior to DA—typically performed using optimization algorithms and treating the parameters as time-invariant (Zhou et al., 2020). Online DA tools have also been applied in this regard, simultaneously adjusting the model parameters and states. While computational cost is often prohibitive in employing these algorithms over large spatial extents and fine resolutions, more computationally efficient approaches founded in machine learning have been developed more recently (Tsai et al., 2021). Embracing these technological advances and improving the physical realism of the model and observations should reduce the reliance on rescaling approaches and enable the direct incorporation of observations within land DA systems. In addition to the parameters, LSM development must also include the refinement of the inherent coupling representations between relevant energy, water, and carbon cycle processes. Diagnostic studies that identified inconsistencies in coupling strength representations (Section 3.5) should serve as a guideline for such improvements. The same observational advances that enable the assimilation of higher-quality state and flux estimates into LSMs also provide opportunities for better assessing, and correcting, coupling strength biases in LSMs. If the objective of information transfer to related variables through DA is to be realized, such model process and parameter improvements are essential. Finally, LSM formulations must evolve to include relevant prognostics that can be employed in land DA environments. More broadly, there is a significant need to consider the requirements from assimilation needs in LSM development.

4.4 Enhance Spaceborne Observations

There is also a need for co-producing land observation datasets in coordination with the land DA community so that a consistent characterization of spatiotemporal errors of observations can be established. Given the prevalence and heterogeneity of bias in land modeling and assimilation, bias-aware DA algorithms that are built to diagnose and incorporate systematic errors during DA (as opposed to addressing them a priori) are needed (Dee, 2005; Pauwels et al., 2013; Ridler et al., 2017). However, such bias-aware systems are only practical if the climatology of the observations can be considered as the reference (Dong, Crow, et al., 2020). Efforts to reduce biases in land remote sensing retrievals or forward modeling approaches should be a priority to achieve these goals (Gao et al., 2020; Grant et al., 2008). Further, coordination across mission science teams and data producers to reduce errors and inconsistencies across products will be beneficial in improving the quality of observational products available for assimilation. Such data homogenization and harmonization efforts can reduce interpretation errors and foster the development of universally accepted observation benchmarks (McCabe et al., 2017). DA systems, in fact, offer natural environments for the harmonization of satellite data from multiple platforms, by appropriately handling the respective sources of errors and uncertainties. Land reanalysis and multivariate assimilation efforts incorporating multiple sources of data should be considered important goals for the land DA community. Such efforts are necessary to establish climate data records, which are important benchmarks for climate studies (C.-H. Su et al., 2016).

4.5 Exploit Ground Measurements

As noted in the introduction, the focus of this article is primarily on the assimilation of space-borne, spatially distributed measurements. Ground-based measurements of land surface variables such as soil moisture, snow, ET, and river flow are available from several measurement networks and focused field campaigns. These measurements have played and will continue to play an important role in enabling the systematic evaluation and optimization of DA analyses. Further coordination with the ground measurement community (e.g., the International Soil Moisture Network (ismn.geo.tuwien.ac.at), FLUXNET (fluxnet.org), and the U.S. National Ecological Observatory Network (NEON; www.neonscience.org)) to communicate the specific DA relevant needs is required to overcome some of the gaps discussed in Section 3. For example, reliable, co-located in situ measurements of land surface states and fluxes (soil moisture, ET, runoff, LAI) can help in validating satellite data (e.g., SMOSREX [de Rosnay et al., 2006] and VALERI [Weiss et al., 2001]) and establishing true surface-subsurface coupling relationships, which are needed to improve the realism of such formulations in models and to ultimately improve the efficiency of DA methods. Similarly, focused ground-truthing efforts to characterize human management impacts (e.g., irrigation impact on soil moisture) would also be helpful in validating space-borne measurements over such areas. Much like the anticipated growth in the number of space-based platforms and measurements, the advent of new technologies is fostering an era of crowd-sourced observations from platforms such as mobile phones and inexpensive sensors that can be placed within a virtual network through them. For example, crowd-sourcing via smartphone apps and GPS devices has revolutionized how traffic information is collected and updated in real-time. While there are still challenges in ensuring the reliability and quality of such measurements for science applications, they offer new possibilities not just for evaluation, but also for DA, particularly if their availability becomes routine and spatially distributed.

4.6 Expand Forward Modeling Capabilities

Advances in forward modeling capabilities are needed to reduce the reliance on retrieval products and to enhance the exploitation of observations in coupled DA systems. In addition to radiative transfer models, the land DA community could also leverage advances in machine learning to develop observation operators. Even if the relevant radiative transfer model is available for the translation of a certain type of observation, ensuring their reliability over different types of terrestrial, biosphere, and climatic conditions often requires significant calibration and validation efforts. The modern class of machine learning methods such as deep learning has shown great promise in its ability to perform hierarchical feature extraction and could effectively function as a forward operator within DA systems (e.g., Shan et al., 2022). Generally, machine learning models are often criticized for their non-transparent nature, a hindrance to developing physical understanding and insights into the results generated by them. Recent advancements have also enabled “explainable AI” methods (Montavon et al., 2018) that have specifically focused on increasing transparency of the decisions and inner workings of the machine learning algorithms. These methods are useful for providing improved understanding of the sensitivities of the inputs, which can lead to improved information exploitation to produce more skillful retrieval products or using such translations directly within DA systems. These methods can also be applied to understand when and where remote sensing measurements provide relevant information content, which can then be used in DA environments to screen observations and refine the quality control procedures to improve the observational information utilization. In addition to providing a statistically robust model, machine learning-based emulators can help in reducing the computational expense of retrieval and/or forward modeling (Kolassa et al., 2018; Rodriguez-Fernandez et al., 2015).

4.7 Incorporate Human-Driven Land Processes

A unique challenge and opportunity for land surface DA is to account for human activities for which we either do not have enough knowledge to implement process representations in models or which are inherently impossible to implement due to their subjective nature. As discussed earlier, the identification and inclusion of such signals and their attribution to the systematic differences stemming from them are needed in such instances. The use of machine learning methods can be potentially useful in this regard. Machine learning methods are also increasingly used for object detection and feature extraction applications (Bengio, 2009; LeCun et al., 2015), where the algorithm automatically learns and characterizes features from the data that are presented to it. As these methods are less “feature engineering” dependent, they can be an effective tool for identifying unmodeled features and characterizing the sources of bias in land observations and models. Taking advantage of such capabilities and incorporating machine learning-based modernizations (Reichstein et al., 2019) are likely to result in better characterization of anthropogenic signals.

4.8 Exploit New Space-Borne Observation Technologies

With the availability of remote sensing opportunities afforded by smallsats and cubesats, commercial platforms for Earth observations are becoming more prevalent (Llop et al., 2014; Sandau, 2010). While space-agency missions provide long-term coverage for critical Earth processes, they tend to be costly and often require decades of development. The smallsats are less costly and can provide continuous coverage at fine resolutions, when launched in multiples. Additionally, other sensing opportunities from smart phones, unmanned aerial vehicles, and other close-to-Earth platforms are going to increase, fostering the era of “big data” (Sellars et al., 2013). While these measurements could be exploited within DA environments, they also likely present additional technology hurdles. As the size of the datasets become larger, there is an increasing shift to using cloud-based resources away from local computing resources. Effective and consistent quality control of these measurements from such distributed platforms can also be challenging. Modeling and DA systems must anticipate and adapt to such changing technologies if the significant potential of such datasets is to be realized.

4.9 Employ Land DA in Mission Planning Efforts

Land DA efforts have a critical role to play in the definition of the next generation of observational types and needs. Given the considerable resources required to implement Earth observing missions, their utility must be systematically assessed through observing system simulation experiments (OSSEs), prior to instrument development. DA is key to enabling OSSEs, which can help assess the relative utility of competing mission designs (e.g., Kaminski & Mathieu, 2017; Tan et al., 2007). Mature land DA environments can be used in this regard for the sub-selection of mission configurations and technologies and to define the level of accuracy required from the observations to meet science utility. Comprehensive multi-observation OSSEs can also help to identify the spatiotemporal gaps in the types of observations that are needed to improve the characterization of the water cycle. For example, snow and soil moisture processes may be seasonally dominant for water availability characterization in a region, but the accuracy, revisit, and spatial resolution requirements of those observations are likely to be different for water balance closure or water availability characterization needs. By enabling “what-if” studies, OSSEs can formalize such requirements and influence the recommendations and definitions of future land hydrology observing systems. In a pioneering effort, the SMAP mission was informed by land DA-based OSSEs in its formulation phase (Crow, Chan, et al., 2005; Reichle et al., 2008) and has been operationally generating a soil moisture assimilation product (Reichle et al., 2019). OSSEs have been used to quantify the anticipated utility of snow measurements from different sensors and technologies to guide the development of future snow missions (Garnaud et al., 2019; Kwon et al., 2021; Wrzesien et al., 2022). Similar land DA environments and efforts must be considered part of the lifecycle of future land surface satellite missions.

5 Summary and Conclusions

Land DA has been an active area of research for some decades now and has shown significant promise for improving land surface characterization and benefiting NWP, S2S and operational hydrology forecasting environments. However, to realize its full potential, the land DA community, in conjunction with the relevant observational and modeling communities should address the significant gaps and challenges laid out in this article. Separate configurations of land DA environments specifically tailored for different applications will likely be needed to overcome the constraints of developmental legacies, lack of model realism, and to foster the development and widespread acceptance of land DA as a necessary capability. We hope this agenda article will serve as the pathway for future land DA developmental priorities so that land DA systems will evolve into viable environments for addressing the critical science and water resource problems facing society through the exploitation of observational information.

Acronyms

DA

Data Assimilation

EO

Earth Observation

LSM

Land Surface Model

NWP

Numerical Weather Prediction

S2S

Sub-seasonal to Seasonal

GPS

Global Positioning System

GPM

Global Precipitation Measurement

SMAP

Soil Moisture Active Passive

SMOS

Soil Moisture Ocean Salinity

GRACE-FO

Gravity Recovery And Climate Experiment - Follow On

GRACE

Gravity Recovery And Climate Experiment

OCO-2

Orbiting Carbon Observatory-2

SWOT

Surface Water and Ocean Topography

EKF

Extended Kalman Filter

EnKF

Ensemble Kalman Filter

SSMI/S

Special Sensor Microwave Imager/Sounder

SWE

Snow Water Equivalent

GLDAS

Global Land Data Assimilation System

NLDAS

North American Land Data Assimilation System

LIS

Land Information System

CDF

Cumulative Distribution Function

ET

Evapotranspiration

ESMF

Earth System Modeling Framework

JEDI

Joint Effort for Data Assimilation Infrastructure

OSSE

Observing System Simulation Experiment

NASA

National Aeronautics and Space Administration

USDA-ARS

United States Department of Agriculture - Agricultural Research Service

NOAA

National Oceanic and Atmospheric Administration

Acknowledgments

This manuscript is in part an outcome of an Analysis, Integration, and Modeling of the Earth System (AIMES) workshop “Tackling Technical Challenges in Land Data Assimilation” held virtually in June 2021. In addition, the workshops organized by the International Earth Surface Working Group in past years (2017–2022) have also contributed to the formulation of this article.

Open Research

Data Availability Statement

This article is focused on discussing the challenges and priorities in the field of land data assimilation and does not include the specific use of any particular software or results involving specific data products.