Volume 10, Issue 2 p. 297-319
Research Article
Open Access

The Impact of Parametric Uncertainties on Biogeochemistry in the E3SM Land Model

Daniel Ricciuto

Corresponding Author

Daniel Ricciuto

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Correspondence to: D. Ricciuto, [email protected]Search for more papers by this author
Khachik Sargsyan

Khachik Sargsyan

Sandia National Laboratories, Livermore, CA, USA

Search for more papers by this author
Peter Thornton

Peter Thornton

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Search for more papers by this author
First published: 27 December 2017
Citations: 74

This article has been contributed to by US Government employees and their work is in the public domain in the USA.


We conduct a global sensitivity analysis (GSA) of the Energy Exascale Earth System Model (E3SM), land model (ELM) to calculate the sensitivity of five key carbon cycle outputs to 68 model parameters. This GSA is conducted by first constructing a Polynomial Chaos (PC) surrogate via new Weighted Iterative Bayesian Compressive Sensing (WIBCS) algorithm for adaptive basis growth leading to a sparse, high-dimensional PC surrogate with 3,000 model evaluations. The PC surrogate allows efficient extraction of GSA information leading to further dimensionality reduction. The GSA is performed at 96 FLUXNET sites covering multiple plant functional types (PFTs) and climate conditions. About 20 of the model parameters are identified as sensitive with the rest being relatively insensitive across all outputs and PFTs. These sensitivities are dependent on PFT, and are relatively consistent among sites within the same PFT. The five model outputs have a majority of their highly sensitive parameters in common. A common subset of sensitive parameters is also shared among PFTs, but some parameters are specific to certain types (e.g., deciduous phenology). The relative importance of these parameters shifts significantly among PFTs and with climatic variables such as mean annual temperature.

Key Points

  • We develop a new method to conduct efficient global parameter sensitivity analysis in high-dimensional land surface models
  • Out of 65 parameters, less than one third are identified as sensitive for five model outputs at 100 sites
  • A common subset of sensitive parameters is shared across and within plant functional types

1 Introduction

Uncertainty about land surface processes contributes to a large spread in model predictions about the magnitude and timing of climate change within the 21st century. Land surface models incorporate a diverse array of processes across various temporal and spatial scales, and include from dozens to hundreds of uncertain parameters. As components of complex Earth system models, these land-surface models provide crucial information about fluxes of water, energy, and greenhouse gases to the atmosphere and oceans. However, the signs and magnitudes of these fluxes depend on multiple competing feedbacks, and model projections diverge in the latter half of this century (Friedlingstein et al., 2014). Much of the current understanding about land-surface process uncertainties at regional to global scales derives from model intercomparison projects (MIPs, see, e.g., Friedlingstein et al., 2014; Huntzinger et al., 2013; Piao et al., 2013). However, such MIPs conflate structural and parametric uncertainty, so that combining models or drawing conclusions is difficult (Knutti et al., 2010). Improved understanding about the sensitivity of model outputs to specific parameters and processes, and the contribution of parametric uncertainty to overall prediction uncertainty, is of critical importance for directing future model development and measurements, and for increasing the accuracy of future projections. One method to quantify model parameter uncertainty is sensitivity analysis (SA). A large number of SA methods exist, and different approaches are generally selected depending on the computational demands of the model simulation, the dimensionality of the problem, and the desired accuracy of the result (Gan et al., 2014). For accessing model output uncertainties with respect to parameter variability, it is particularly attractive to apply variance-based decomposition or global sensitivity analysis (GSA) to compute Sobol indices (Saltelli et al. 2006; Sobol, 2001). This involves computation of conditional expectations and variances and, if performed with a Monte-Carlo approach, requires a large number of simulations ranging from thousands to hundreds of thousands depending on the number of parameters and the desired level of accuracy (Saltelli, 2002; Saltelli et al., 2010). It has been used in ecosystem carbon-cycle models, which usually have relatively fast execution time as they do not represent all land-surface processes needed for a coupled climate model (He et al. 2014; Safta et al., 2015; Tang & Zhuang, 2009; Verbeeck et al., 2006), or in more complex land-surface models with shorter simulation lengths (Pappas et al. 2013; Rosolem et al. 2012).

The new Energy Exascale Earth System Model (E3SM), formerly known as the Accelerated Climate Model for Energy (ACME), is both computationally expensive and contains a large number of parameters. The land model component of E3SM, hereafter referred to as ELM, typically requires lengthy equilibration times for hydrologic and carbon cycle variables, and high computational costs limit the number of simulations. Local SA methods have been used, such as the one-at-a-time (OAT) approach that varies parameters around default values (Zaehle & Friend, 2010). While these methods provide useful information about first-order effects, the results can be misleading as they fail to consider parameter interactions, or variations that may occur across the full parameter space (Saltelli et al., 2004). Recently, the Predictive Ecosystem Analyzer (PEcAn) was developed to apply global univariate sensitivity analysis, performing variance decomposition with a manageable number of ensemble members (Lebauer et al., 2013). Generalized linear models (GLM) analyses have been applied to estimate parameter sensitivities in complex models such as the Community Land Model with a low number of samples (Hou et al., 2012). However, for models with a large number of parameters and relatively low number of simulations available, these methods fail to capture important parameter interactions that can be both significant and yielding insights into model functioning (Safta et al., 2015). Similar to GLMs, ANOVA-type representations yield model approximations and allow exact extraction of sensitivity indices. Such methods include Fourier amplitude sensitivity testing (FAST) (Wang et al., 2013), or high-dimensional model representation (HDMR) (Lu et al., 2013). While many SA studies involving land-surface models have shown that first-order effects generally dominate, it is less clear that this will be the case in high-dimensional and computationally expensive models. Even moderately nonlinear models, e.g., requiring second-order interactions, will become difficult to approach when the number of input parameters is large, since the degrees of freedom for underlying FAST or HDMR parameterizations are close to or even exceed the amount of simulation data available. This naturally leads to overfitted model approximations, and correspondingly inaccurate sensitivities. For such underdetermined situations, it is highly desirable to approach the problem adaptively and construct flexible model approximations with the right amount of complexity. In this work, we apply the Bayesian Compressive Sensing (BCS) method—with novel enhancements regarding weighted bases and adaptive growth—to polynomial approximation of the model in order to determine the ELM parameter sensitivities in the high-dimensional parameter regime with a limited number of model simulations. An adaptive model approximation or a surrogate is being constructed in this work with three major ingredients:
  1. Polynomial chaos (PC) expansions provide a flexible approximation for the model behavior over a range of parameter variability. PC expansions are essentially polynomial fits to model input-output map and, given sufficient smoothness, they drastically outperform—in terms of required number of model evaluations—Monte-Carlo methods for forward uncertainty propagation and GSA (Le Maître & Knio, 2010). However, when large number of input parameters are present, it is infeasible to choose polynomial bases a priori without underdetermining the polynomial fit, given a typically low number of model simulations.
  2. Compressive sensing (CS) is applied for selecting only the most relevant polynomial bases, inspired by the breakthrough CS methods in image processing (Donoho, 2006; Candès et al., 2006).
  3. Bayesian inference, well-known as a flexible machinery to incorporate limited information in a formal probabilistic setting, is applied as the main tool for CS-enhanced PC construction. This ensures meaningful PC approximation, and subsequent GSA, with quantified lack-of-knowledge uncertainties even in a regime when a small number of model simulation results are available, i.e., when conventional deterministic methods are infeasible.

This new sensitivity analysis tool provides an opportunity to answer questions about the drivers of uncertainty in land-surface models over large spatial scales. Here we develop the GSA technique for ELM, a complex land-surface model with a large number of uncertain parameters. The most sensitive parameters in ELM are identified across nearly 100 FLUXNET eddy covariance sites representing a wide range of environmental conditions and vegetation types. We seek to answer the following questions about the drivers of uncertainty in terrestrial carbon and water cycles: (1) What are the key parameters and associated processes that drive variability in ELM predictions at each site? (2) How consistent are these parameters within a model plant functional type? And (3) Can we use information about site climate, vegetation type, and location to predict the key parameter sensitivities?

The paper is organized as follows: Section 2 describes details of the methodology, including PC representation, compressive sensing, and Bayesian inference, in order to construct model approximation, a surrogate, that enables simple analytical extraction of sensitivity indices. Then, in section 3 the land model under consideration is described, followed by the results of studying parameter sensitivities across a wide range of locations selected from the FLUXNET database. Finally, the discussion of the results and major conclusions are relegated to sections 4 and 5, respectively.

2 Method

In order to accurately assess the sensitivity information, we employ surrogate construction via Polynomial Chaos (PC) expansions. Over the last two decades, PC methods have gained popularity as very convenient machinery for uncertainty representation and propagation through computational models (Ghanem & Spanos 1991; Le Maître & Knio, 2010). Besides its main use as a surrogate, i.e., an approximate replacement of a computational model in studies that require intensive sampling, PC expansions allow closed-form expressions for variance-based sensitivity indices due to underlying orthogonality of its bases or regressors (Crestaux et al., 2009).

2.1 Polynomial Chaos Surrogate

In the context of this paper, input parameters are associated with simple bounds with no additional probabilistic form. In such cases, PC expansions reduce to simple polynomial fits of the model output urn:x-wiley:19422466:media:jame20533:jame20533-math-0001 with respect to scaled inputs ξ (each physical input parameter λi is scaled to computational input urn:x-wiley:19422466:media:jame20533:jame20533-math-0002 for urn:x-wiley:19422466:media:jame20533:jame20533-math-0003 where d is the total number of model input parameters)

In the expansion (1), multivariate polynomials urn:x-wiley:19422466:media:jame20533:jame20533-math-0005 are employed as basis functions, composed of products of univariate Legendre polynomials urn:x-wiley:19422466:media:jame20533:jame20533-math-0006 of degree αk each, for urn:x-wiley:19422466:media:jame20533:jame20533-math-0007. Details of various options of finding the coefficients urn:x-wiley:19422466:media:jame20533:jame20533-math-0008, given a set of model evaluations at training inputs urn:x-wiley:19422466:media:jame20533:jame20533-math-0009 are given in the Appendix Appendix A. An essential part of the polynomial surrogate approximation is the choice of the basis set urn:x-wiley:19422466:media:jame20533:jame20533-math-0010, as will be described further in this work.

The major challenge in the surrogate construction for the ELM model is the so-called curse of dimensionality. First of all, the number of model evaluations needed to achieve a polynomial fit of comparable accuracy levels grows rapidly with dimensionality d, considerably impacting the accuracy standard one wants to achieve, particularly for expensive forward models. The second major challenge is associated with the rapid growth of the polynomial basis sets urn:x-wiley:19422466:media:jame20533:jame20533-math-0011 in equation 1. In the present work, the surrogate construction is associated with input parameter vector urn:x-wiley:19422466:media:jame20533:jame20533-math-0012 of dimensionality urn:x-wiley:19422466:media:jame20533:jame20533-math-0013. The nonadaptive polynomial expansion truncations typically lead to infeasibly large basis sets. For example, the total order truncation with order p leads to urn:x-wiley:19422466:media:jame20533:jame20533-math-0014 basis terms. For d = 50, only a second-order expansion already requires K = 1,326 basis terms. A full tensor product truncation, i.e., each dimension includes bases up to order 3, would require a much higher number, urn:x-wiley:19422466:media:jame20533:jame20533-math-0015, of basis terms. While there are truncation schemes that have somewhat delayed growth with dimensionality, e.g., hyperbolic cross (Blatman & Sudret, 2011), Smolyak constructions (Conrad & Marzouk, 2013; Smolyak, 1963), high-dimensional model representation (Rabitz et al., 1999)i or anisotropic truncations (Gerstner & Griebel, 2003), working well for moderate dimensionalities ( urn:x-wiley:19422466:media:jame20533:jame20533-math-0016), they rely on strong structural assumptions of the function urn:x-wiley:19422466:media:jame20533:jame20533-math-0017, and generally are infeasible for urn:x-wiley:19422466:media:jame20533:jame20533-math-0018. The main limitation is that the number of model evaluations N is typically smaller than the degrees of freedom, i.e., the number of unknown PC coefficients, in the PC surrogate representation. In such cases, PC surrogate construction is enhanced with sparsity-promoting regularization term that minimizes urn:x-wiley:19422466:media:jame20533:jame20533-math-0019 norm of the coefficient vector urn:x-wiley:19422466:media:jame20533:jame20533-math-0020. This idea is essentially borrowed from the compressive sensing (CS) paradigm that is commonly used in image processing (Candès et al. 2006; Donoho, 2006), and has seen considerable recent interest in uncertainty quantification community for surrogate construction of high-dimensional models, see e.g., Rauhut and Ward (2012); Hampton and Doostan (2015); Jakeman et al. (2015) and references therein.

Further, we employ a Bayesian inference technique for finding polynomial surrogate coefficients, as a Bayesian framework allows probabilistic interpretation of the associated regularized least-squares problem, and leads to meaningful surrogates with quantified polynomial coefficient uncertainties even in the case of extremely low amount of model simulations (Sargsyan, 2016). Subsequently, one arrives at sensitivity indices with quantified uncertainty due to lack of sufficient number of model evaluations. Bayesian compressive sensing (BCS) algorithm (Babacan et al., 2010) is the main driving force behind the PC surrogate construction in this work, with a subsequent extension to incorporate iterative growth of polynomial bases terms, as described in Sargsyan et al. (2014); Jakeman et al. (2015).

Besides, the major methodological novelty of this paper (see Table A1 in the Appendix Appendix A) is the extension of the iterative BCS method to include weighted regularization that allows more targeted and efficient polynomial bases growth. The detailed analysis of the associated improvement is outside of the scope of the current paper, and will be reported in a separate methodological paper (Sargsyan et al., 2017).

2.2 Global Sensitivity Analysis: Variance-Based Indices

In this work, the overarching goal is to perform global sensitivity analysis (GSA) of a model with respect to a large number of parameters. In this regard, Sobol sensitivity indices will be employed (Sobol, 2003; Saltelli et al., 2004). These indices correspond to variance-based decomposition, as they measure fractional contributions of each parameter or group of parameters towards the total output variance. We outline four sensitivity indices:
  1. Main effect sensitivities, also called first-order sensitivities, measure variance contribution due to urn:x-wiley:19422466:media:jame20533:jame20533-math-0021-th parameter only, defined as

    where urn:x-wiley:19422466:media:jame20533:jame20533-math-0023 and urn:x-wiley:19422466:media:jame20533:jame20533-math-0024 indicate variance with respect to the i-th parameter and expectation with respect to the rest of the parameters, respectively.

  2. Total effect sensitivities measure total variance contribution of the i-th parameter, i.e., including interactions with other parameters, and are defined as

    where urn:x-wiley:19422466:media:jame20533:jame20533-math-0026 and urn:x-wiley:19422466:media:jame20533:jame20533-math-0027 indicate expectation with respect to the i-th parameter and variance with respect to the rest of the parameters, respectively.

  3. Joint-main sensitivities measure joint variance contribution due to i-th and j-th parameter only and are defined as

    where urn:x-wiley:19422466:media:jame20533:jame20533-math-0029 and urn:x-wiley:19422466:media:jame20533:jame20533-math-0030 indicate variance with respect to the i-th and j-th parameters and expectation with respect to the rest of the parameters, respectively. These sensitivities are commonly used and often referred to as simply joint sensitivities. However, for overall effect of the parameter pair (i, j) on the output variance, it is necessary to explore somewhat less conventional, the joint-total sensitivities, defined next.

  4. Joint-total sensitivities measure joint variance contribution due to i-th and j-th parameter overall—including their interactions with other parameters—and are defined as
While there are random sampling approaches (Jansen, 1999; Saltelli, 2002; Saltelli et al., 2010; Sobol, 1993) for efficient estimation of the integral quantities in formulae (2)–(5), they all suffer from the generic deficiency pertinent to all random sampling methods. Namely, in order to get accurate enough estimates, one needs prohibitively large number of model evaluations. In this regard, PC surrogates constructed in this work offer a much more efficient alternative. When the function urn:x-wiley:19422466:media:jame20533:jame20533-math-0032 is approximated by a PC surrogate urn:x-wiley:19422466:media:jame20533:jame20533-math-0033, one can compute moments and sensitivity indices using the orthogonality of the PC basis functions,
where urn:x-wiley:19422466:media:jame20533:jame20533-math-0039 and urn:x-wiley:19422466:media:jame20533:jame20533-math-0040 are multiindex subsets that include only the terms of interest for the corresponding sensitivity index. Therefore, having constructed the PC surrogate, one can easily evaluate the sensitivity indices by computing the weighted sum-of-the-squares of appropriately selected PC coefficients (Crestaux et al., 2009; Sargsyan, 2016; Sudret, 2008).

2.3 Model Description

The E3SM land model version zero (E3SMv0) used in this analysis branched from a version of the Community Land Model version 4.5 (Oleson et al., 2013) in 2014, and is identical in most aspects. Primary differences include an updated version of point CLM used in recent model-experiment comparisons (Mao et al. 2016; Shi et al., 2015) for site-level simulations, and a refined version of accelerated decomposition spin-up, as originally described in Thornton and Rosenbloom (2005). Specifically, we use modified options within the CLM4.5BGC configuration, with the vertically resolved soil organic matter carbon and nitrification-denitrification schemes as described in Koven et al. (2013), but retaining the CTC decomposition pathways from CLM4.0, originally implemented in Biome-BGC (Oleson et al., 2013) rather than the CENTURY pathways. The simulations also include active wildfire (Li et al., 2012) and an active CH4 cycle CLM4Me (Riley et al., 2011), which is required for the nitrification-denitrification scheme although CH4 fluxes are not analyzed here. Some model parameters were also recoded as variables rather than constants, so that they could be read from input parameter files created by our UQ framework.

Site locations were chosen from the FLUXNET network of eddy covariance sites (Baldocchi et al., 2001), demonstrated in Figure 1 and listed in Table 1. Site plant functional types (PFTs) were set following the information in the FLUXNET database (www.fluxdata.org) and are listed in Table 2. Some sites had multiple PFTs, indicated by multiple values in Table 1. Other required model site information, including soil properties, were extracted from global gridded data sets at urn:x-wiley:19422466:media:jame20533:jame20533-math-0041 resolution (Oleson et al., 2013). For input meteorological data, including air temperature, shortwave and longwave radiation, air pressure, wind speed, precipitation and specific humidity, and the global urn:x-wiley:19422466:media:jame20533:jame20533-math-0042 6 hourly CRU-NCEP data were used (Oleson et al., 2013). For each site, data from the nearest urn:x-wiley:19422466:media:jame20533:jame20533-math-0043 grid cell were extracted to drive site-level ELM. Because these simulations investigated, the sensitivities of pre-industrial steady state conditions, transient forcing data sets such as land-use history were not applied. Nitrogen deposition, aerosol deposition, and CO2 concentrations were set to 1,850 values and site-level data were extracted from globally gridded data sets.

Details are in the caption following the image

Location of the 96 FLUXNET sites explored in this work. The sites are color-coded according to their PFTs, listed in Table 2.

Table 1. The Names, Locations, and Plant Functional Type Information for 96 Selected FLUXNET Sites, Ordered From Lowest to Highest Latitude
FLUXNET code Site name Latitude Longitude PFT FLUXNET code Site name Latitude Longitude PFT
AU-Wac Wallaby Creek −37.43 145.19 5 IT-PT1 Zerbolo-Parco 45.2 9.06 7
AU-TUM Tumbarumba −35.66 148.15 5 US-Ho1 Howland Forest 45.2 −68.74 1
ZA-KRU Skukuza (Kruger) −25.02 31.5 14 13 7 CA-Mer Mer Bleue 45.41 −75.52 10
BR-Sp1 Sao Paulo Cerrado −21.62 −47.65 4 US-UMB UMBS forest 45.56 −84.71 7
BW-Ghm Ghanzi mixed −21.2 21.75 14 FR-Lq1 Laqueuille 45.64 2.74 13
ZM-MON Mongo −15.43 23.25 13 14 6 US-WCr Willow Creek 45.81 −90.08 7
AU-How Howard Springs −12.49 131.15 6 US-Wrc Wind River Crane 45.82 −121.95 1
BR-Ji2 Ji-Parana −10.08 −61.93 4 IT-Mbo Monte Bondone 46.02 11.05 13
BR-Sa1 Santarem −2.86 −54.96 4 IT-Ren Bolzano 46.59 11.43 1
BR-Ma2 Manaus −2.61 −60.21 4 HU-Bug Bugacpuszta 46.69 19.6 13
BR-Cax Caxiuana −1.72 −51.46 4 AT-Neu Neustift 47.12 11.32 13
CN-DHS Dinghushan Forest 23.17 112.57 5 CH-Oe1 Oensingen grass 47.29 7.73 13
CN-QYZ Qianyanzhou 26.74 115.07 5 CA-Gro Groundhog mixed 48.22 −82.16 1
US-KS2 Kennedy Space Center (Scrub Oak) 28.61 −80.67 7 US-Fpe Fort Peck 48.31 −105.1 13
US-SP1 Austin Cary 29.74 −82.22 1 FR-Fon Fontainebleau 48.48 2.78 7
CN-Do1 Dongtan 31.52 121.96 13 14 1 FR-Hes Hesse 48.67 7.06 7
US-Aud Audubon 31.59 −110.51 10 CZ-BK1 Beskidy mountains 49.5 18.54 1
US-SO2 Sky Oaks old 33.37 −116.62 10 1 13 CA-Qfo Eastern Old Black Spruce 49.69 −74.34 1
US-Goo Goodwin Creek 34.25 −89.87 13 CA-Let Lethbridge 49.71 −112.94 13
US-Fuf Flagstaff unmanaged forest 35.09 −111.76 1 CA-Ca1 Campbell River 49.87 −125.33 1
US-WBW Walker Branch 35.96 −84.29 7 DE-Bay Bayreuth-Waldstein 50.14 11.87 1
US-Dk3 Duke Forest 35.98 −79.09 1 BE-Vie Vielsam 50.31 6 8 2
JP-TAK Takayama 36.15 137.42 7 DE-Wet Wetzstein 50.45 11.46 1
CN-HaM Haibei 37.37 101.18 13 DE-Tha Tharandt 50.96 13.57 1
US-Ton Tonzi Ranch 38.43 −120.97 13 DE-Hai Hainich 51.08 10.45 7
PT-Esp Espirra 38.64 −8.6 5 UK-Ham Hampshire 51.12 −0.86 7
US-Moz Missouri Ozark 38.74 −92.2 7 NL-Ca1 Cabauw 51.97 4.93 13
US-Blo Blodgett Forest 38.9 −120.63 1 IE-Dri Dripsey 51.99 −8.75 13
US-MMS Morgan Monroe 39.32 −86.41 7 NL-Hor Horstermeer 52.03 5.07 7
ES-Lma Las Majadas del Tietar 39.94 −5.77 10 CA-Oas Old Aspen 53.63 −106.2 8
US-NR1 Niwot Ridge 40.03 −105.55 2 CA-SJ3 Young Jack Pine 53.88 −104.64 2
US-GLE GLEES 41.36 −106.24 2 CA-Ojp Old Jack Pine 53.92 −104.69 2
US-OHO Oak Openings 41.55 −83.84 7 CA-Obs Old Black Spruce 53.99 −105.12 2
IT-Cpz Castelporziano 41.71 12.38 1 CA-WP1 Western Peatland 54.95 −112.47 2
US-IB2 Fermi 41.84 −88.24 13 DK-Sor Soroe 55.49 11.65 7
IT-Col Collelongo 41.85 13.59 7 DK-Lva Lille Valley 55.68 12.08 1
CN-CBS Changbaishan 42.4 128.1 8 1 UK-Ebu Easter Bush 55.87 −3.21 13
IT-Ro1 Roccarespampani1 42.41 11.93 7 CA-Man Manitoba OBS 55.88 −98.48 2
US-MLT US-MLT 42.5 −113.41 13 UK-Gri Aberfeldy 56.61 −3.8 1
US-Ha1 Harvard Forest 42.54 −72.17 7 SE-Nor Norunda 60.09 17.48 2
CA-TP1 Turkey Point 42.66 −80.56 1 FI-Hyy Hyytiala 61.85 24.29 2
CN-NMG Inner Mongolia 43.55 116.67 13 US-BN1 Delta Junction 63.92 −145.38 2
IT-Sro San Rossore 43.73 10.28 1 SE-Deg Degero Stormyr 64.18 19.55 12
FR-Pue Puechabon 43.74 3.6 1 FI-Sod Sodankyla 67.36 26.64 2
US-Bar Bartlett 44.06 −71.29 7 SE-Abi Abisko 68.36 18.79 12
US-Me2 Metolius 44.45 −121.56 1 US-Ivo Ivotuk 68.49 −155.75 11
IT-Non Nonantola 44.69 11.09 7 FI-Kaa Kaamanen 69.14 27.3 12
FR-LBr Le Bray 44.72 −0.77 1 US-Atq Atqasuk 70.47 −157.41 12
Table 2. Plant Functional Types (PFTs) Used in ELM and the Number of Sites From Each PFT Investigated in This Work
PFT Name Number of sites
−1 Mixed 9
1 Boreal evergreen needleleaf tree 22
2 Temperate evergreen needleleaf tree 11
3 Boreal deciduous needleleaf tree 0
4 Tropical evergreen broadleaf tree 5
5 Temperate evergreen broadleaf tree 5
6 Tropical deciduous broadleaf tree 1
7 Temperate decidous broadleaf tree 20
8 Boreal deciduous broadleaf tree 1
9 Boreal evergreen shrub 0
10 Temperate decidous broadleaf shrub 2
11 Boreal deciduous broadleaf shrub 1
12 C3 Arctic grass 4
13 C3 non-Arctic grass 15
14 C4 grass 0

A total of 68 model parameters related to biogeophysics and biogeochemical cycling were varied randomly within uniform ranges justified by literature or expert judgment in Table 3. The ranges used in this analysis are intended to represent both uncertainty from insufficient knowledge and spatial or cross-species variability within broadly defined PFT categorizations. While some parameters are directly measurable (e.g., leaf C:N ratio; specific leaf area, etc.) with high accuracy at specific locations, their values may be highly variable within a PFT. For some parameters, PFT-specific ranges representing interspecies variability were used when measurements were available in the literature (White et al., 2000). When no source of information was available to estimate a uniform range, the default range was set as +/–50% of the default value except where bounded by physical or biological constraints.

Table 3. Descriptions, Ranges, and Sources of Information Used for the 68 ELM Input Parameters Varied in This Study
Parameter Description Units Minimum Maximum Source
dleaf leaf dimension m 0.01 0.1 1
mp Ball-Berry slope parameter none 4.5 13.5 2
bp Ball-Berry intercept parameter umol H2O m−1 s−1 5,000 15,000 2
vcma x ha Activation energy for vcmax J mol−1 36,000 108,000 2
Vcma x hd Deactivation energy for Vcmax J mol−1 198,000 202,000 3
xl leaf-stem orientation index None −0.5 0.375 3
roota_par Rooting depth distribution parameter m−1 3.5 16.5 3
rootb_par Rooting depth distribution parameter m−1 0.5 4.5 3
slatop Specific leaf area at canopy top m2 gC−1 0.002 0.03 4
dsladlai Change in SLA with canopy depth m2 gC−1 LAI−1 5.00E-04 2.50E-03 3
leafcn Leaf carbon/nitrogen (C:N) ratio gC gN−1 12.5 70 4
flnr Fraction of leaf N in RuBisco None 0.0231 0.264 2
smpso Soil water potential at stomatal onset mm −125,000 −17,500 3
smpsc Soil water potential at stomatal closure mm −642,000 −125,000 3
lflitcn Leaf litter C:N ratio gC gN−1 70 140 3
frootcn Fine root C:N ratio gC gN−1 21 63 2
livewdcn Live wood C:N ratio gC gN−1 25 75 2
deadwdcn Dead wood C:N ratio gC gN−1 200 1,400 4
froot_leaf Fine root to leaf allocation ratio None 0.3 2.5 4
flivewd Fraction of new wood that is live None 0.06 0.28 4
fcur Fraction of allocation currently displayed None 0 1 1
lf_flab Leaf litter labile fraction None 0.125 0.375 2
lf_fcel Leaf litter cellulose fraction None 0.25 0.6 1
lf_flig Leaf litter lignin fraction None Constrained Constrained 1
fr_flab Fine root labile fraction None 0.125 0.375 2
fr_fcel Fine root cellulose fraction None 0.25 0.375 1
fr_flig Fine root lignin fraction None Constrained Constrained 1
cwd_fcel Coarse woody debris cellulose fraction None 0 1 1
leaf_long Leaf longevity Years 1 7 4
grperc Growth respiration fraction % 0.125 0.375 2
bdnr Bulk denitrification rate None 0.125 0.375 2
dayscrecover Days to recover for XSMR pool Days 10 90 3
br_mr Base rate for maintenance respiration (MR) umol m−2 s−1 1.26E-06 3.75E-06 2
q10_mr Temperature sensitivity for MR None 1.3 3.3 5
cn_s1 C:N ratio for soil organic matter pool 1 gC gN−1 8 25 3
cn_s2 C:N ratio for soil organic matter pool 2 gC gN−1 8 25 3
cn_s3 C:N ratio for soil organic matter pool 3 gC gN−1 6 25 3
cn_s4 C:N ratio for soil organic matter pool 4 gC gN−1 6 25 3
rf_l1s1 Respiration fraction for litter 1 – > SOM1 None 0.2 0.58 2
rf_l2s2 Respiration fraction for litter 2 – > SOM2 None 0.275 0.82 2
rf_l3s3 Respiration fraction for litter 3 – > SOM3 None 0.15 0.43 2
rf_s1s2 Respiration fraction for SOM1 – > SOM2 None 0.14 0.42 2
rf_s2s3 Respiration fraction for SOM2 – > SOM3 None 0.23 0.69 2
rf_s3s4 Respiration fraction for SOM3 – > SOM4 None 0.28 0.83 2
k_l1 Decay rate for litter pool 1 1 d−1 0.9 1.8 2
k_l2 Decay rate for litter pool 2 1 d−1 0.036 0.112 2
k_l3 Decay rate for litter pool 3 1 d−1 0.007 0.021 2
k_s1 Decay rate for SOM1 1 d−1 0.036 0.112 2
k_s2 Decay rate for SOM2 1 d−1 0.007 0.021 2
k_s3 Decay rate for SOM3 1 d−1 0.0007 0.0021 2
k_s4 Decay rate for SOM4 1 d−1 5.00E-05 1.50E-04 2
k_frag Fragmentation rate for coarse wood litter 1 d−1 5.00E-04 1.50E-03 2
dnp Denitrification proportion None 0.001 0.03 3
minpsi_hr Minimum psi for heterotrophic respiration Mpa −15 −5 2
q10_hr Q10 for heterotrophic respiration None 1.3 3.3 5
r_mort Mortality rate 1 yr−1 0.0025 0.05 4
sf_minn solubility of mineral N None 0.05 0.15 2
crit_dayl Critical day length for senescence Seconds 35,000 45,000 3
ndays_on Number of days for leaf on Days 15 45 2
ndays_off Number of days for leaf off Days 7.5 22.5 2
fstor2tran Fraction of storage transferred None 0.25 0.75 2
crit_onset_fdd Critical onset freezing days Days 7.5 22.5 2
crit_onset_swi Critical onset soil water Days 7.5 22.5 2
soilpsi_on Critical soil water potential for leaf onset Mpa −5 −0.75 3
crit_offset_fdd Critical offset freezing days Days 7.5 22.5 2
crit_offset_swi Critical soil water for leaf offset Days 7.5 22.5 2
soilpsi_off critical soil water potential for leaf offset Mpa −5 −0.75 3
lwtop_ann live wood turnover proportion yr−1 0.5 1 4
  • Note. The information used to define the uniform parameter ranges in the “source” column is: (1) Physically constrained (parameters may not exceed these bounds or are constrained by other parameter values); (2) +/– 50% of default value; (3) Expert judgement (in the case where there is insufficient literature, but +/– 50% would not be appropriate); (4) White et al. (2000); and (5) Raich and Schlesinger (1992). In the case where parameters vary by PFT, the maximum range among all PFTs was applied.

We performed 3,000 spin-up simulations on the Titan supercomputer at Oak Ridge National Laboratory, varying the 68 model parameters randomly over their uniform prior ranges. Following the standard procedure for global simulations, we used a repeating 20 year cycle of CRU-NCEP meteorological data from 1901 to 1920. For each of the 3,000 ensemble members, 96 site-level simulations were bundled into individual simulations using unstructured grids on 6 nodes (96 cores). Ten jobs in total were submitted, each consisting of 300 simulations performed in parallel using 1,800 nodes (19,400 cores). The simulations ran for a 300 model years in accelerated decomposition mode followed by 200 model years in nonaccelerated spin-up mode. We then postprocessed the model output to produce average values for each site and ensemble member over the last 20 year cycle of meteorological data for five model outputs: gross primary productivity (GPP), latent heat flux (EFLX_LH_TOT), total vegetation carbon (TOTVEGC), total soil organic matter carbon (TOTSOMC), and total leaf area index (TLAI). These outputs were then used as training samples for WIBCS surrogate construction. Both WIBCS and the following GSA have been performed via UQ Toolkit, a lightweight C/C++ UQ library with an emphasis on PC methods as well as Bayesian inference (Debusschere et al., 2015).

3 Global Sensitivity Analysis for E3SMv0 Land Model

Here we describe the results of our parametric uncertainty quantification (UQ) study in the E3SM land model (ELM), leading to GSA for a set of 96 land model sites and 5 output quantities of interest (QoIs). A model ensemble is evaluated to identify the key parameter sensitivities across a range of sites with different vegetation and climatic conditions. In this study, we investigate following QoIs: Total leaf area index (TLAI), gross primary productivity (GPP), total vegetation carbon (TOTVEGC), total soil organic matter carbon (TOTSOMC), and total latent heat flux (EFLX_LH_TOT). For all five QoIs, the value is taken as an average over the last 20 years of the simulation representing a steady-state preindustrial value obtained by cycling 1901–1920 CRU-NCEP reanalysis meteorology. Detailed site-level results are first presented for three sites to illustrate the capabilities of the approach. This is followed by a broader analysis of the sensitivities at all 96 sites, organized by location, plant functional type, and climate conditions.

3.1 Parameter Sensitivities

Results for the first three sites (AU-Wac, AU-TUM, and ZA-Kru) are shown as an example in Figure 2. In this figure, the size of each solid blue circular symbol is proportional to the main effect sensitivity for its associated parameter, while the thickness of the straight green line connecting two parameters represents the joint-total effect. Joint total effect sensitivities are high when a quantity of interest is sensitive to the covariation of two or more parameters. Only main effect sensitivities for the top six parameters (shown as six points along a red circle) and joint-total sensitivities with a threshold value greater than 20% of the highest joint-total sensitivity are shown on these plots. Parameters that are ranked lower than sixth in main effect sensitivity but have important joint-total contributions are connected to their companion parameter as an extension off of the main circle.

Details are in the caption following the image

Main effect and joint-total sensitivity indices for five output QoIs for the first three sites, AU-Wac (left column), AU-TUM (middle column), and ZA-KRU (right column). The radii of circles correspond to the main sensitivities in equation (7), while the green edge widths correspond to the joint-total sensitivities from equation (10). Only the most influential parameters are listed for each site-output pair.

For the first site (AU-Wac), the base rate of maintenance respiration (br_mr) is the most sensitive parameter for three QoIs (TOTVEGC, TOTSOMC, and EFLX_LH_TOT), while TLAI is most sensitive to the specific leaf area at the top of the canopy (slatop) and GPP is most sensitive to the fraction of leaf nitrogen in RuBisCO (flnr). GPP is also sensitive to the Ball-Berry stomatal conductance slope (mp) and intercept (bp), while EFLX_LH_TOT is also sensitive to the fine root to leaf allocation ratio (froot_leaf) and flnr. A number of parameters appear among the QoIs with smaller sensitivities: the fine root C:N ratio (frootcn), leaf C:N ratio (leafCN), a parameter controlling rooting distribution (rootb_par), and the temperature sensitivity of maintenance respiration (q10_mr). Strong joint sensitivities are indicated between froot_leaf and br_mr (EFLX_LH_TOT, TOTVEGC and GPP) and between bp and mp (TOTVEGC and TOTSOMC), and between bp and slatop (TLAI).

The second site (AU-TUM) is located in Australia several hundred kilometers to the northeast of the first site (AU-Wac) and shares the same PFT category of evergreen temperate broadleaf tree (PFT #5). The subset of parameters that appears as sensitive across the five QoIs is similar at both sites with 11 shared parameters and 2 unique parameters at AU-TUM that do not appear at AU-Wac. However, the relative sensitivities among these parameters are somewhat different between the sites. AU-TUM is more sensitive to the stomatal conductance parameters (mp and bp), with bp the most sensitive parameter for TOTSOMC, TOTVEGC, and GPP. Unlike AU-Wac, GPP at AU-TUM is not sensitive to flnr. Like AU-Wac, br_mr is a highly sensitive parameter for EFLX_LH_TOT, TOTSOMC, and TOTVEGC, and slatop is the most sensitive parameter for TLAI.

The third site is a savanna in Zambia (ZA-Kru) and comprises two PFTs: tropical deciduous broadleaf forests and C4 grasslands. Although this site shares a number of sensitive parameters in common with the Australian sites, other parameters tend to dominate the sensitivities. The critical soil water potential for senescence (soilpsi_off) is the most sensitive parameter for TLAI, GPP, and TOTSOMC. This deciduous phenology parameter is not relevant for the Australian sites, which are evergreen forests. Similarly, the fraction of stored carbon transferred to display pools (fstor2tran) is relevant only for deciduous sites, as evergreen sites do not use the storage pools in ELM (Oleson et al., 2013). Fstor2tran is important for TOTVEGC and EFLX_LH_TOT. Similar to the Australian sites, TLAI is sensitive to slatop and both TOTVEGC and EFLX_LH_TOT are sensitive to br_mr. The remaining sensitive parameters are similar to those found at the first two sites, with the exception of bulk denitrification rate (bdnr), indicating that this site may be more sensitive to denitrification processes.

Figure 3 shows the main effect sensitivities for the same three sites as Figure 2 and for the same five QoIs in the style of a heat map, with each subplot containing information about the most sensitive parameters for that site (these are parameters that have a main effect sensitivity exceeding a threshold value of 0.004 for at least one of the outputs). Blank (white) entries indicate sensitivities below the color bar, with sensitivities over the threshold represented from blue to red as the values increase. Also, the sensitivities are scaled per output, i.e., different rows should not be compared to each other. This figure largely contains the same information as Figure 2, but here it is more evident that several parameters are sensitive across multiple QoIs and sites (br_mr, slatop, and mp), at one site for multiple QoIs (soilpsi_off), while other parameters are only important for one QoI (e.g., cn_s3, rf_s2s3). We also assess the confidence of the parameter sensitivity indices by pushing the posterior distribution of PC coefficients through the associated sensitivity computation (shown in Figure 4 for the same three sites). The availability of such uncertainty range is a major advantage of using Bayesian machinery. The one-standard-deviation interval of these main effect sensitivity indices is smaller than 0.05 in most cases, and the uncertainties are not large enough to impact the ranking of importance for model parameters or change our conclusions. Clearly, our ensemble size of 3,000 simulations, while not enough to produce accurate surrogates in this high-dimensional space for reliable model replacement, is nevertheless sufficient for this sensitivity analysis.

Details are in the caption following the image

Main effect sensitivity indices from equation (7) for the first three sites, (top) AU-Wac, (middle) AU-TUM, and (bottom) ZA-KRU, are shown, across all five output QoIs. Only the most relevant input parameters for each site are shown. To improve visibility, the color code for each row (i.e., output QoI) is scaled according to the highest contributor, hence the color bars correspond to scaled sensitivities.

Details are in the caption following the image

Main effect sensitivity indices from equation (7) are shown, with associated one-standard-deviation error bars as pushed forward from uncertain PC surrogate coefficients. Three sites, (top) AU-Wac, (middle) AU-TUM, and (bottom) ZA-KRU, are selected for illustration.

The main effect sensitivities are shown for all 96 sites in Figure 5. Sites are grouped by PFT, with −1 indicating mixed PFT sites (see Table 2). Because of the distribution of our chosen FLUXNET sites, some PFTs are represented by many sites (e.g., PFT #1—boreal broadleaf evergreen and PFT#7—temperate broadleaf deciduous), while other PFTs are only represented by 1–2 sites. Only parameters with a main effect sensitivity of greater than 0.2 for at least one site are shown in the figure. A total of 17 parameters appear in the figure, indicating that nearly 75% of parameters were relatively insensitive for all sites and all QoIs. In cases where the sum of the main effect sensitivities per site is less than 1, this indicates that there are other minor parameters that are contributing to the output variance and not shown in the figure, and/or joint sensitivities (not shown) are contributing to the variance of the QoI. These results are further summarized by PFT in Figure 6, with error bars indicating the standard deviation of sensitivity across sites within a PFT. For PFTs 6, 8, and 11, only one site is available and standard deviations could not be calculated.

Details are in the caption following the image

Main effect sensitivity indices from equation (7) for five output QoIs across 96 FLUXNET sites grouped by their PFTs, are shown. Only the most influential parameters (with visible colors) for each QoI are listed in the legends.

Details are in the caption following the image

Total effect sensitivity indices, as given in equation (8), are shown for five output QoIs, with respect to the most impactful parameters, grouped by site PFTs. The error bars indicate 25 and 75 percentiles among the sites within a single PFT.

In these plots, consistent patterns begin to emerge both within and across PFTs. Total leaf area index (TLAI) is highly sensitive to specific leaf area at the top of the canopy (slatop) at nearly all 96 sites. Similarly, TLAI is sensitive to froot_leaf, flnr and frootcn in a large majority of sites, regardless of PFT. Other parameters are only sensitive in certain PFTs. Soilpsi_off, a parameter that controls deciduous leaf offset as a function of soil water potential, appears only in mixed PFTs, tropical deciduous sites, shrublands and grasslands. The main reason is because this parameter is only active in PFTs with stress deciduous phenology (Oleson et al., 2013). In a similar manner, leaf longevity (leaf_long) is only active for evergreen sites and is only sensitive across all sites in boreal evergreen needleleaf, tropical broadleaf evergreen, and temperate broadleaf evergreen forest types. However, leaf_long is only sensitive in a subset of temperate needleleaf evergreen sites even though it is active for all of these sites. Other parameters, even while active among all PFTs, only appear for certain sites. The Ball-Berry stomatal conductance intercept (bp) is most sensitive in a majority of needleleaf evergreen sites (both boreal and temperate), and a minority of C3 non-Arctic grasslands and temperate evergreen broadleaf sites. TLAI is sensitive to the base rate of maintenance respiration br_mr in the first 5 PFTs and relatively insensitive in PFTs 6–13. On the other hand, the bulk denitrification rate (bdnr) is only sensitive in PFTs 6–13.

For GPP, the fraction of leaf nitrogen in RuBisCO (flnr) plays a larger role than for TLAI, with a main effect sensitivity of at least 0.2 for a large majority of these sites. Similar to TLAI, frootcn, froot_leaf and froot_cn are all also important for most sites regardless of PFT. The leaf carbon to nitrogen ratio (leafcn) also appears for nearly all sites. The soilpsi_off, bdnr, and mp parameters display similar patterns of sensitivities as for TLAI. The Ball-Berry stomatal conductance slope mp is more sensitive than for TLAI, especially for evergreen needleleaf tree PFTs, and for broadleaf tropical deciduous trees. While slatop remains a sensitive parameter for the first five PFTs, it is relatively insensitive for PFTs 6–13. The temperature sensitivity of respiration (q10) is a sensitive parameter for boreal evergreen needleleaf trees, temperate deciduous broadleaf trees, and for most C3 non-Arctic grasses. The fstor2tran parameter, which controls the fraction of carbohydrate storage allocated to new growth in deciduous systems, is sensitive for most temperate deciduous tree sites and a subset of the grassland sites.

The other QoIs display similar patterns of sensitivities with some differences in the relative importance of individual parameters. For total vegetation carbon (TOTVEGC), the mortality parameter (r_mort) is highly sensitive for nearly all of the forest PFTs, with a main sensitivity index of greater than 0.2 for most of these sites. For the grassland sites, the fstor2tran parameter is highly sensitive. The remaining parameters have similar patterns of sensitivity as for TLAI, with the exceptions of flnr being somewhat less sensitive, and q10, slatop, and soilpsi_off being insensitive. For soil organic matter carbon, q10 is sensitive at many sites, especially in the temperate and boreal PFTs. The carbon-nitrogen ratio of the third soil organic matter pool also plays a key role in controlling the variability of TOTSOMC at many sites. For latent heat flux (EFLX_LH_TOT), the sensitivity pattern is similar to that of GPP, with the rooting depth distribution parameter (rootb_par) and cn_s3 becoming important for a minority of sites.

We also find that for certain parameters and QoIs, there is a clear dependence of the sensitivity of a parameter on a climate variable such as mean annual temperature (MAT). For example, there is an increasing sensitivity of all QoIs to the base rate of maintenance respiration (br_mr) as the mean annual temperature increases (Figure 7). There is also a decreasing sensitivity of total vegetation carbon to leaf carbon-nitrogen ratio (leafcn) as the mean annual temperature declines. These relationships, while not particularly strong, are statistically significant, occurring both within and across PFTs.

Details are in the caption following the image

Correlation between mean annual temperature and total effect sensitivities is illustrated, for select cases that have a correlation coefficient higher than 0.55 in magnitude. Each data point corresponds to a FLUXNET site, color-coded according to PFTs.

4 Discussion

The ELM contains a large number of uncertain parameters, but the sensitivities of key QoIs for climate and carbon cycling are dominated by a relatively small fraction of these parameters. The sensitivities are largely coherent within each model plant functional type, and are similar across different QoIs. This is useful for determining which model processes dominate the carbon cycle, and is a promising development for future model calibration efforts that are computationally expensive and more limited by the curse of dimensionality. The sensitive parameters identified in this study are generally associated with vegetation process: leaf properties (slatop, leafcn, flnr), fine root properties (frootcn), respiration (q10, br_mr), carbon allocation (froot_leaf), stomatal conductance (mp and bp), phenology (soilpsi_off, crit_dayl), and mortality (r_mort). Parameters associated with soil organic matter cycling and litter chemistry tended to be less sensitive given the QoIs analyzed and our model structure. We note that our study analyzes the QoIs only in long-term mean steady state conditions, and insensitive parameters may become more important at different timescales, or when transient responses are occurring.

The methodology described in this study enables efficient global sensitivity analysis that otherwise would have been computationally infeasible using Monte Carlo or quasi-Monte Carlo methods given the complexity and relatively slow evaluation time of ELM. Other efficient sensitivity analysis methods exist and have been used in land models such as the Fourier amplitude sensitivity test (FAST) (Xu et al., 2009), or the High-Dimensional Model Representation (HDMR) (Lu et al., 2013). However, our method provides additional benefits: it allows for the identification of specific higher-order interaction terms, as well as quantifies uncertainties associated with the lack of enough simulations, at the same time constructing surrogate models for the input-output maps for each site and QoI. These surrogate models can be used to predict how specific model QoIs vary over the full parameter space, and potentially for the calibration of model parameters. Although the surrogate models created in this study are have approximately urn:x-wiley:19422466:media:jame20533:jame20533-math-0044 relative error and unlikely to be suitable for accurate calibration, they do provide insight into joint parameter impacts to outputs, as well as allow sufficiently accurate GSA, as the error bars on sensitivity indices suggest, leading to a subsequent dimensionality reduction. Besides, one can envision an iterative feedback in a tuning process, in which initial approximate calibration informs toward smaller parameter ranges. Consequently, follow-up surrogate models with fewer parameters that also vary on smaller ranges will provide the accuracy needed for calibration of ELM parameters across multiple sites.

While other land models often use different parameters than ELM, those that have conducted sensitivity analyses do show some overlap on the key processes involved in controlling the sensitivities of carbon cycle variables. A sensitivity analysis of the CABLE land surface model indicated that Vcmax, leaf area, and stomatal parameters as the strongest controls on photosynthetic and latent heat fluxes Lu et al. (2013). In ELM, Vcmax is not a parameter but a function of the specific leaf area at the top of the canopy (slatop), the fraction of leaf nitrogen in RuBisCO (flnr), and the leaf carbon to nitrogen ratio (leafcn), all of which are identified as sensitive parameters in this analysis. A recent uncertainty analysis of the ED model (Dietze et al., 2014) indicated that a parameter controlling growth respiration dominates the variability in carbon cycle variables, along with mortality and stomatal parameters. Our analysis also points to the importance of autotrophic respiration, but through the influence of maintenance respiration parameters. Photosynthetic parameters are less important in the ED study primarily because the prior distribution of Vcmax is tightly constrained by observations.

In this study, the uniform distributions used as parameter uncertainty bounds are based on expert and literature-based assessments, and also include within-PFT variability (White et al., 2000). These uncertainty bounds have a large impact on the relative sensitivities of each parameter, so it is important that these bounds are informed by existing knowledge to our best ability (Lebauer et al., 2013). While tightly constrained observation-based priors can be constructed for a single site or species, the broad PFT definitions in ELM and other land models cover wide ranges of species and environmental conditions. Therefore, even parameters than can be measured precisely at the site scale necessarily have diffuse priors in this global, PFT-based study. In the near future, incorporating information from trait databases such as TRY (Kattge et al., 2011) will help to provide more detail about the characteristics of parameter distributions and refine our sensitivity indices. Looking further ahead, trait-based dynamic vegetation modeling approaches (Pavlick et al., 2013; van Bodegom et al., 2014) are advancing the state of land modeling beyond the artificial construct of PFTs and will allow measured trait data and uncertainties to be specified (or predicted) with more precision. In general, defining model parameters as measurable quantities allows for more realistic assessments of sensitivities and uncertainty.

In fully coupled climate models, large uncertainties still remain in predictions of terrestrial carbon cycle fluxes (Friedlingstein et al., 2014). However, within-model process and parameter uncertainties are often neglected in multimodel intercomparisons, which include ensembles that are designed to explore uncertainty in initial conditions and socioeconomic scenarios (Knutti & Sedlávcek, 2013). Process and parameter uncertainties are represented in the spread among models from a number of different modeling centers, but interdependencies and numerous differences among these models make it difficult to attribute these differences to specific causes. Given the high computational demands of fully coupled models such as E3SM, only a limited number of simulations can be conducted to examine such within-model uncertainties. Global sensitivity analysis allows a better understanding of the key drivers of uncertainty in the land-surface components in these coupled models, which can then be explored in a efficient manner. In the future, extending this methodology to include model calibration or weighting using benchmarks (e.g., ILAMB) will also allow for the removal of poorly performing model parameter combinations or submodels, and the for the construction of ensembles with improved predictive skill (Schwalm et al., 2015). We expect these and future sensitivity analysis to inform future E3SM experiments that will explore the impacts of uncertainty in biogeochemical processes on the Earth system.

5 Conclusions

Although there have been extensive observations of land processes over the past several decades, many land-surface model parameters remain highly uncertain because they are difficult to connect to measurements, are highly variable, or reflect incomplete process understanding. This parametric uncertainty is a large source of prediction uncertainty in climate models, but it is not often considered in model intercomparison studies or individual model ensembles. Information about the sensitivity of specific model outputs to parameters is valuable for (1) understanding the controlling processes, (2) focusing model development efforts, and (3) for ideally targeting new observations to reduce parameter and prediction uncertainty. Complex models such as ELM limit the number of ensemble simulations that can be performed, requiring efficient methods to conduct sensitivity analysis. Here, a new method was used to conduct global sensitivity analysis in order to examine the sensitivities of 5 model outputs to 68 parameters at 96 FLUXNET sites using ELM. Addressing the research questions posed in the introduction, we conclude that (1) among all 96 sites, a minority of parameters contributed to variability among the five QoIs. Parameters related to photosynthesis, stomatal conductance, and autotrophic respiration and allocation of carbon and nitrogen to plant components were identified as highly sensitive. (2) The main effect sensitivity indices varied among PFTs, but were relatively consistent among sites within a PFT. Finally, (3) climatic indices such as mean annual temperature also control the sensitivity of some model parameters. In future work, we may be able to use site information to predict key parameter sensitivities in advance.

Our workflow allows for efficient construction of surrogate models and evaluation of sensitivities. Now that a subset of key parameters have been identified, future work will focus on constructing more accurate surrogate models with this subset. Construction of these more accurate surrogates will allow us to identify combinations of parameters and plant traits that are unrealistic. For example, a significant minority of simulations result in dead vegetation, which is clearly not the case at these FLUXNET sites. In addition, these regions of dead vegetation in parameter space complicate the fit for the surrogate model. Model calibration may also be performed with these refined surrogate models using FLUXNET data and other terrestrial data sources from experiments, field measurements, and remote sensing. Posterior probability density functions of these parameters obtained from calibration, in combination with more informative priors from plant trait databases, will enable a formal quantification of uncertainty for land surface model and climate model hindcasts and forecasts. Such uncertainty estimates are essential for policy makers and for the assessment of climate impacts on the Earth system.


This research was supported as part of the Energy Exascale Earth System Model (E3SM), funded by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research. This study used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under contract DE-AC05-00OR22725. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA-0003525. The Uncertainty Quantification Toolkit (UQTk), used to perform the analysis in this publication, is available at http://www.sandia.gov/UQToolkit. Model output used in this study from E3SM is publicly available from https://github.com/dmricciuto/E3SM_FluxnetUQdata/. This manuscript has been authored by UT-Battelle, LLC under contract DE-AC05-00OR22725 with the U.S. Department of Energy. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a nonexclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this manuscript, or allow others to do so, for United States Government purposes. The Department of Energy will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).

    Appendix A: Bayesian Framework for Sparse Polynomial Surrogate Construction

    A1. Polynomial Chaos Surrogate

    Consider a forward function urn:x-wiley:19422466:media:jame20533:jame20533-math-0045 that maps the input parameter vector urn:x-wiley:19422466:media:jame20533:jame20533-math-0046 to an output quantity of interest (QoI). The input parameter set urn:x-wiley:19422466:media:jame20533:jame20533-math-0047 is in general viewed as a jointly distributed random vector, but for surrogate construction over ranges urn:x-wiley:19422466:media:jame20533:jame20533-math-0048, for urn:x-wiley:19422466:media:jame20533:jame20533-math-0049, can be written component-wise as
    where urn:x-wiley:19422466:media:jame20533:jame20533-math-0051 is a vector of d independent, identically distributed (i.i.d.) uniform random variables. Thus the computational inputs urn:x-wiley:19422466:media:jame20533:jame20533-math-0052 are simply scaled versions of the physical inputs urn:x-wiley:19422466:media:jame20533:jame20533-math-0053.
    A PC expansion for the output QoI Q views the latter as a random variable induced by the uniform random input urn:x-wiley:19422466:media:jame20533:jame20533-math-0054, and is written as an expansion with respect to standard multivariate polynomials urn:x-wiley:19422466:media:jame20533:jame20533-math-0055,
    where urn:x-wiley:19422466:media:jame20533:jame20533-math-0057 is a multiindex set with size urn:x-wiley:19422466:media:jame20533:jame20533-math-0058.

    Each multivariate polynomial urn:x-wiley:19422466:media:jame20533:jame20533-math-0059 corresponds to a multtindex vector urn:x-wiley:19422466:media:jame20533:jame20533-math-0060 that defines the polynomial degrees per univariate polynomial urn:x-wiley:19422466:media:jame20533:jame20533-math-0061 as urn:x-wiley:19422466:media:jame20533:jame20533-math-0062. Also, by convention, the sum of all degrees urn:x-wiley:19422466:media:jame20533:jame20533-math-0063 is called the order of the multivariate polynomial urn:x-wiley:19422466:media:jame20533:jame20533-math-0064.

    The standard polynomials are orthogonal with respect to the PDF of urn:x-wiley:19422466:media:jame20533:jame20533-math-0065,
    and in this work are normalized such that urn:x-wiley:19422466:media:jame20533:jame20533-math-0067, where urn:x-wiley:19422466:media:jame20533:jame20533-math-0068 is the PDF of ξ. Since the inputs are assumed uniform as in (A1), we employ Legendre polynomials that are orthogonal with respect to uniform measure urn:x-wiley:19422466:media:jame20533:jame20533-math-0069. More generally, the standard polynomial-variable pairs can be chosen from the Wiener-Askey generalized PC scheme (Xiu & Karniadakis, 2002) depending on the assumed or expected behavior of the outputs of interest.
    The goal of PC surrogate construction is then finding the coefficient vector urn:x-wiley:19422466:media:jame20533:jame20533-math-0070 such that the approximation (A2) is established. Considering the function or the physical model urn:x-wiley:19422466:media:jame20533:jame20533-math-0071 as a black-box, the construction methods rely on evaluating the model urn:x-wiley:19422466:media:jame20533:jame20533-math-0072 at a set of training points, corresponding to underlying samples of urn:x-wiley:19422466:media:jame20533:jame20533-math-0073. Regression-based approach directly minimizes a distance measure between the training set of evaluations of the function urn:x-wiley:19422466:media:jame20533:jame20533-math-0074 and the surrogate, with a possible regularization term urn:x-wiley:19422466:media:jame20533:jame20533-math-0075. The evaluations of the surrogate can be written in a matrix form
    denoting the measurement matrix by urn:x-wiley:19422466:media:jame20533:jame20533-math-0077, where urn:x-wiley:19422466:media:jame20533:jame20533-math-0078 is a counting index of the multiindex set urn:x-wiley:19422466:media:jame20533:jame20533-math-0079. The minimization problem can then generally be written as
    where urn:x-wiley:19422466:media:jame20533:jame20533-math-0081 is a distance measure between two vectors urn:x-wiley:19422466:media:jame20533:jame20533-math-0082 and urn:x-wiley:19422466:media:jame20533:jame20533-math-0083. Most commonly, one chooses an urn:x-wiley:19422466:media:jame20533:jame20533-math-0084 distance urn:x-wiley:19422466:media:jame20533:jame20533-math-0085 and an urn:x-wiley:19422466:media:jame20533:jame20533-math-0086 regularization term urn:x-wiley:19422466:media:jame20533:jame20533-math-0087 (also called ridge regression or Tikhonov regularization) with a weight matrix urn:x-wiley:19422466:media:jame20533:jame20533-math-0088, leading to a closed form solution for regularized least-squares

    A2. Compressive Sensing

    When the input dimensionality is large, and the forward model is expensive, one often has an underdetermined case, i.e., when the number of model evaluations N is lower than the number of polynomial terms. In such situations, the classical least-squares regression is not well-defined, while the urn:x-wiley:19422466:media:jame20533:jame20533-math-0090 regularization is strongly biased. In order to tackle such underdetermined systems, one often uses urn:x-wiley:19422466:media:jame20533:jame20533-math-0091 regularization, i.e., urn:x-wiley:19422466:media:jame20533:jame20533-math-0092, that enforces sparsity in polynomial expansions, in line with the compressive sensing (CS) paradigm, which is a machine learning technique for sparse signal recognition that made a breakthrough in image processing a decade ago (Candès et al., 2006; Donoho, 2006). The key premise is that if a sparse signal is present in sufficiently incoherent measurements, one can efficiently recover it with urn:x-wiley:19422466:media:jame20533:jame20533-math-0093 regularization. In our context, measurements are model evaluations at randomly selected parameter inputs, i.e., training points. While the most classical CS formulation relies on direct urn:x-wiley:19422466:media:jame20533:jame20533-math-0094 minimization under sufficiently accurate reconstruction constraint, it is generally equivalent to a regularized urn:x-wiley:19422466:media:jame20533:jame20533-math-0095 minimization problem, which in the PC regression setting reads as
    corresponding to a more general regularization term urn:x-wiley:19422466:media:jame20533:jame20533-math-0097 with urn:x-wiley:19422466:media:jame20533:jame20533-math-0098. In the simplest setting, the regularization parameter urn:x-wiley:19422466:media:jame20533:jame20533-math-0099 is typically chosen with cross-validation methods (Jakeman et al., 2015). It controls the relative importance of the penalty with respect to the goodness-of-fit. The sparsest solution, i.e., the solution with the fewest nonzero PC coefficients, corresponds to the urn:x-wiley:19422466:media:jame20533:jame20533-math-0100 norm, while the urn:x-wiley:19422466:media:jame20533:jame20533-math-0101 solution provides the reconstruction, while remaining a computationally tractable convex optimization problem, with high probability given sufficiently mild conditions on the sample set urn:x-wiley:19422466:media:jame20533:jame20533-math-0102 and basis functions urn:x-wiley:19422466:media:jame20533:jame20533-math-0103 (Candès et al., 2006; Donoho, 2006).

    A3. Bayesian Framework

    Regression approaches that rely on minimizing a functional (A5) directly extend to a Bayesian framework. Bayesian connection is exploited in this paper as it allows flexibility and meaningful results—with quantified uncertainties—even in presence of limited number of evaluations of the expensive forward model urn:x-wiley:19422466:media:jame20533:jame20533-math-0104. Generally, Bayesian methods (Bernardo & Smith, 2000; Carlin & Louis, 2011; Sivia & Skilling, 2006) are well-suited to deal with a limited number and potentially noisy function evaluations. They allow constructing an uncertain surrogate with any number of samples by describing the uncertainty via posterior probability distribution on PC coefficient vector urn:x-wiley:19422466:media:jame20533:jame20533-math-0105. Besides, Bayesian techniques are efficient in sequential scenarios where the surrogate is updated online, i.e., as new evaluations of urn:x-wiley:19422466:media:jame20533:jame20533-math-0106 arrive (Sargsyan et al., 2012). While computationally more expensive than the simple minimization (A5), the Bayesian approach puts the construction of the objective function urn:x-wiley:19422466:media:jame20533:jame20533-math-0107 within a formal probabilistic context where the objective function can be interpreted as a Bayesian negative-log-likelihood, while the regularization term urn:x-wiley:19422466:media:jame20533:jame20533-math-0108 is the negative-log-prior. For example, the urn:x-wiley:19422466:media:jame20533:jame20533-math-0109 or least-squares objective function corresponds to an i.i.d. Gaussian assumption for the misfit random variable urn:x-wiley:19422466:media:jame20533:jame20533-math-0110. Indeed, Bayes' formula in the regression context reads as
    relating a prior probability distribution on PC surrogate coefficients urn:x-wiley:19422466:media:jame20533:jame20533-math-0112 to the posterior distribution, via the likelihood function urn:x-wiley:19422466:media:jame20533:jame20533-math-0113, which essentially measures the goodness-of-fit of the model training evaluations urn:x-wiley:19422466:media:jame20533:jame20533-math-0114 to the surrogate model evaluations urn:x-wiley:19422466:media:jame20533:jame20533-math-0115 for a parameter set urn:x-wiley:19422466:media:jame20533:jame20533-math-0116. The posterior distribution reaches its maximum at the Maximum a Posteriori (MAP) value. Working with logarithms of the prior and posterior distributions as well as the likelihood, the MAP value solves the optimization problem,
    and is clearly equivalent to the regularized regression (A5). As an additional feature, the Bayesian formulation leads to a probabilistic representation of urn:x-wiley:19422466:media:jame20533:jame20533-math-0118 encoded in the posterior distribution. In the least-squares case, assuming a Gaussian prior with vanishing mean
    and an i.i.d. Gaussian likelihood with, say, constant variance urn:x-wiley:19422466:media:jame20533:jame20533-math-0120,
    one arrives at a multivariate normal posterior distribution for the coefficient vector
    With such probabilistic description of urn:x-wiley:19422466:media:jame20533:jame20533-math-0123, the PC surrogate is uncertain, and is in fact a Gaussian process with analytically computable mean and covariance functions
    where urn:x-wiley:19422466:media:jame20533:jame20533-math-0125 is the basis measurement vector at parameter value urn:x-wiley:19422466:media:jame20533:jame20533-math-0126, i.e., its k-th entry is urn:x-wiley:19422466:media:jame20533:jame20533-math-0127. A key strength of the Bayesian approach is that it leads to a probabilistic surrogate that quantifies the uncertainty due to lack of enough function evaluations. Moreover, Bayesian framework also allows inclusion of nuisance parameters, e.g., parameters of the prior or the likelihood, that are inferred together with urn:x-wiley:19422466:media:jame20533:jame20533-math-0128 and subsequently integrated out to lead to marginal posterior distributions on urn:x-wiley:19422466:media:jame20533:jame20533-math-0129.
    Compressive sensing, or urn:x-wiley:19422466:media:jame20533:jame20533-math-0130 regularization also can be associated with a prior within a Bayesian framework. Namely, the solution of (A7) is the same as MAP estimate of a Bayesian problem with the i.i.d. Gaussian likelihood (A11) and a Laplace prior
    if one chooses the regularization matrix urn:x-wiley:19422466:media:jame20533:jame20533-math-0132. This is the basis for the Bayesian Compressive Sensing (BCS) (Babacan et al., 2010; Ji et al., 2008; Sargsyan et al., 2014) approach, which solves the associated Bayesian problem with a fast approximate method similar to the relevance vector machine (RVM) and Bayesian sparse learning (Tipping, 2001; Tipping & Faul, 1991).
    In this work, we apply the recently developed generalization of the sparse learning algorithm that employs additional structure provided by the polynomial basis and iteratively grows the basis set to include higher-order bases in the dimensions that matter (Jakeman et al. 2015; Sargsyan et al., 2014). This is particularly useful for high-dimensional problems, in which a priori basis selection by any of the truncation rules cannot expose high-order terms due to overfitting. Moreover, we have enhanced the algorithm to include weighted regularization that corresponds to the prior
    with basis-specific weights, which allow better recovery of underlying sparse representation, as et will be reported in a parallel paper (Sargsyan al., 2017). Table A1 illustrates the hierarchy of associated methods that have been described in this section, leading up to the weighted iterative Bayesian compressive sensing (WIBCS) employed in this paper.
    Table A1. List of Relevant Regression Methods for High-Dimensional Surrogate Construction, Leading Up to Weighted Iterative BCS Employed in This Paper
    Classical form Bayesian extension Bayesian solution Iterative growth of polynomial bases
    Classical Least Squares, (A5), (A6) Gaussian prior (A10) Exact: (A12)
    Compressive Sensing, (A7) Laplace prior (A14) Apprx: BCS (Babacan et al., 2010; Tipping, 2001) (Jakeman et al., 2015; Sargsyan et al., 2014)
    Weighted CS weighted Laplace prior (A15) Apprx: weighted BCS (Sargsyan et al., 2017) WIBCS: This paper