Volume 11, Issue 7 p. 2089-2129
Research Article
Open Access

The DOE E3SM Coupled Model Version 1: Overview and Evaluation at Standard Resolution

Jean-Christophe Golaz

Corresponding Author

Jean-Christophe Golaz

Lawrence Livermore National Laboratory, Livermore, CA, USA

Correspondence to:Chris Golaz,

[email protected]

Search for more papers by this author
Peter M. Caldwell

Peter M. Caldwell

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Luke P. Van Roekel

Luke P. Van Roekel

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Mark R. Petersen

Mark R. Petersen

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Qi Tang

Qi Tang

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Jonathan D. Wolfe

Jonathan D. Wolfe

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Guta Abeshu

Guta Abeshu

Department of Civil and Environmental Engineering, University of Houston, Houston, TX, USA

Search for more papers by this author
Valentine Anantharaj

Valentine Anantharaj

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Search for more papers by this author
Xylar S. Asay-Davis

Xylar S. Asay-Davis

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
David C. Bader

David C. Bader

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Sterling A. Baldwin

Sterling A. Baldwin

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Gautam Bisht

Gautam Bisht

Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Search for more papers by this author
Peter A. Bogenschutz

Peter A. Bogenschutz

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Marcia Branstetter

Marcia Branstetter

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Search for more papers by this author
Michael A. Brunke

Michael A. Brunke

Department of Hydrology and Atmospheric Sciences, University of Arizona, Tucson, AZ, USA

Search for more papers by this author
Steven R. Brus

Steven R. Brus

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Susannah M. Burrows

Susannah M. Burrows

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Philip J. Cameron-Smith

Philip J. Cameron-Smith

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Aaron S. Donahue

Aaron S. Donahue

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Michael Deakin

Michael Deakin

Sandia National Laboratory, Albuquerque, NM, USA

Department of Mathematics, University of British Columbia, Vancouver, British Columbia, Canada

Search for more papers by this author
Richard C. Easter

Richard C. Easter

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Katherine J. Evans

Katherine J. Evans

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Search for more papers by this author
Yan Feng

Yan Feng

Argonne National Laboratory, Lemont, IL, USA

Search for more papers by this author
Mark Flanner

Mark Flanner

Climate and Space Sciences and Engineering, University of Michigan, Ann Arbor, MI, USA

Search for more papers by this author
James G. Foucar

James G. Foucar

Sandia National Laboratory, Albuquerque, NM, USA

Search for more papers by this author
Jeremy G. Fyke

Jeremy G. Fyke

Los Alamos National Laboratory, Los Alamos, NM, USA

Associated Engineering Group of Companies, Edmonton, Alberta, Canada

Search for more papers by this author
Brian M. Griffin

Brian M. Griffin

Department of Mathematical Sciences, University of Wisconsin-Milwaukee, Milwaukee, WI, USA

Search for more papers by this author
Cécile Hannay

Cécile Hannay

National Center for Atmospheric Research, Boulder, CO, USA

Search for more papers by this author
Bryce E. Harrop

Bryce E. Harrop

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Mattthew J. Hoffman

Mattthew J. Hoffman

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Elizabeth C. Hunke

Elizabeth C. Hunke

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Robert L. Jacob

Robert L. Jacob

Argonne National Laboratory, Lemont, IL, USA

Search for more papers by this author
Douglas W. Jacobsen

Douglas W. Jacobsen

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Nicole Jeffery

Nicole Jeffery

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Philip W. Jones

Philip W. Jones

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Noel D. Keen

Noel D. Keen

Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Search for more papers by this author
Stephen A. Klein

Stephen A. Klein

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Vincent E. Larson

Vincent E. Larson

Department of Mathematical Sciences, University of Wisconsin-Milwaukee, Milwaukee, WI, USA

Search for more papers by this author
L. Ruby Leung

L. Ruby Leung

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Hong-Yi Li

Hong-Yi Li

Department of Civil and Environmental Engineering, University of Houston, Houston, TX, USA

Search for more papers by this author
Wuyin Lin

Wuyin Lin

Brookhaven National Laboratory, Upton, NY, USA

Search for more papers by this author
William H. Lipscomb

William H. Lipscomb

Los Alamos National Laboratory, Los Alamos, NM, USA

National Center for Atmospheric Research, Boulder, CO, USA

Search for more papers by this author
Po-Lun Ma

Po-Lun Ma

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Salil Mahajan

Salil Mahajan

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Search for more papers by this author
Mathew E. Maltrud

Mathew E. Maltrud

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Azamat Mametjanov

Azamat Mametjanov

Argonne National Laboratory, Lemont, IL, USA

Search for more papers by this author
Julie L. McClean

Julie L. McClean

Scripps Institution of Oceanography, University of California, San Diego, La Jolla, CA, USA

Search for more papers by this author
Renata B. McCoy

Renata B. McCoy

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Richard B. Neale

Richard B. Neale

National Center for Atmospheric Research, Boulder, CO, USA

Search for more papers by this author
Stephen F. Price

Stephen F. Price

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Yun Qian

Yun Qian

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Philip J. Rasch

Philip J. Rasch

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
J. E. Jack Reeves Eyre

J. E. Jack Reeves Eyre

Department of Hydrology and Atmospheric Sciences, University of Arizona, Tucson, AZ, USA

Search for more papers by this author
William J. Riley

William J. Riley

Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Search for more papers by this author
Todd D. Ringler

Todd D. Ringler

Los Alamos National Laboratory, Los Alamos, NM, USA

Congressional Science Fellow, U.S. House of Representatives, Washington, DC, USA

Search for more papers by this author
Andrew F. Roberts

Andrew F. Roberts

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Erika L. Roesler

Erika L. Roesler

Sandia National Laboratory, Albuquerque, NM, USA

Search for more papers by this author
Andrew G. Salinger

Andrew G. Salinger

Sandia National Laboratory, Albuquerque, NM, USA

Search for more papers by this author
Zeshawn Shaheen

Zeshawn Shaheen

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Xiaoying Shi

Xiaoying Shi

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Search for more papers by this author
Balwinder Singh

Balwinder Singh

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Jinyun Tang

Jinyun Tang

Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Search for more papers by this author
Mark A. Taylor

Mark A. Taylor

Sandia National Laboratory, Albuquerque, NM, USA

Search for more papers by this author
Peter E. Thornton

Peter E. Thornton

Oak Ridge National Laboratory, Oak Ridge, TN, USA

Search for more papers by this author
Adrian K. Turner

Adrian K. Turner

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Milena Veneziani

Milena Veneziani

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Hui Wan

Hui Wan

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Hailong Wang

Hailong Wang

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Shanlin Wang

Shanlin Wang

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Dean N. Williams

Dean N. Williams

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Phillip J. Wolfram

Phillip J. Wolfram

Los Alamos National Laboratory, Los Alamos, NM, USA

Search for more papers by this author
Patrick H. Worley

Patrick H. Worley

PHWorley Consulting, Oak Ridge, TN, USA

Search for more papers by this author
Shaocheng Xie

Shaocheng Xie

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Yang Yang

Yang Yang

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Jin-Ho Yoon

Jin-Ho Yoon

School of Earth Sciences and Environmental Engineering, Gwangju Institute of Science and Technology, Gwangju, South Korea

Search for more papers by this author
Mark D. Zelinka

Mark D. Zelinka

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Charles S. Zender

Charles S. Zender

Departments of Earth System Science and Computer Science, University of California, Irvine, CA, USA

Search for more papers by this author
Xubin Zeng

Xubin Zeng

Department of Hydrology and Atmospheric Sciences, University of Arizona, Tucson, AZ, USA

Search for more papers by this author
Chengzhu Zhang

Chengzhu Zhang

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Kai Zhang

Kai Zhang

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Yuying Zhang

Yuying Zhang

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Xue Zheng

Xue Zheng

Lawrence Livermore National Laboratory, Livermore, CA, USA

Search for more papers by this author
Tian Zhou

Tian Zhou

Pacific Northwest National Laboratory, Richland, WA, USA

Search for more papers by this author
Qing Zhu

Qing Zhu

Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Search for more papers by this author
First published: 15 March 2019
Citations: 359
This article has been contributed to by US Government employees and their work is in the public domain in the USA.

Abstract

This work documents the first version of the U.S. Department of Energy (DOE) new Energy Exascale Earth System Model (E3SMv1). We focus on the standard resolution of the fully coupled physical model designed to address DOE mission-relevant water cycle questions. Its components include atmosphere and land (110-km grid spacing), ocean and sea ice (60 km in the midlatitudes and 30 km at the equator and poles), and river transport (55 km) models. This base configuration will also serve as a foundation for additional configurations exploring higher horizontal resolution as well as augmented capabilities in the form of biogeochemistry and cryosphere configurations. The performance of E3SMv1 is evaluated by means of a standard set of Coupled Model Intercomparison Project Phase 6 (CMIP6) Diagnosis, Evaluation, and Characterization of Klima simulations consisting of a long preindustrial control, historical simulations (ensembles of fully coupled and prescribed SSTs) as well as idealized CO2 forcing simulations. The model performs well overall with biases typical of other CMIP-class models, although the simulated Atlantic Meridional Overturning Circulation is weaker than many CMIP-class models. While the E3SMv1 historical ensemble captures the bulk of the observed warming between preindustrial (1850) and present day, the trajectory of the warming diverges from observations in the second half of the twentieth century with a period of delayed warming followed by an excessive warming trend. Using a two-layer energy balance model, we attribute this divergence to the model's strong aerosol-related effective radiative forcing (ERFari+aci = −1.65 W/m2) and high equilibrium climate sensitivity (ECS = 5.3 K).

Key Points

  • This work documents E3SMv1, the first version of the U.S. DOE Energy Exascale Earth System Model
  • The performance of E3SMv1 is documented with a set of standard CMIP6 DECK and historical simulations comprising nearly 3,000 years
  • E3SMv1 has a high equilibrium climate sensitivity (5.3 K) and strong aerosol-related effective radiative forcing (-1.65 W/m2)

Plain Language Summary

The U.S. Department of Energy funded the development of a new state-of-the-art Earth system model for research and applications relevant to its mission. The Energy Exascale Earth System Model version 1 (E3SMv1) consists of five interacting components for the global atmosphere, land surface, ocean, sea ice, and rivers. Three of these components (ocean, sea ice, and river) are new and have not been coupled into an Earth system model previously. The atmosphere and land surface components were created by extending existing components part of the Community Earth System Model, Version 1. E3SMv1's capabilities are demonstrated by performing a set of standardized simulation experiments described by the Coupled Model Intercomparison Project Phase 6 (CMIP6) Diagnosis, Evaluation, and Characterization of Klima protocol at standard horizontal spatial resolution of approximately 1° latitude and longitude. The model reproduces global and regional climate features well compared to observations. Simulated warming between 1850 and 2015 matches observations, but the model is too cold by about 0.5 °C between 1960 and 1990 and later warms at a rate greater than observed. A thermodynamic analysis of the model's response to greenhouse gas and aerosol radiative affects may explain the reasons for the discrepancy.

1 Introduction

In 2013, the U.S. Department of Energy (DOE) developed a report summarizing observed long-term trends that, if continued for several decades, would have major impacts on the energy sector (U.S. Department of Energy, 2013). Among these were regional trends in air and water temperatures, water availability, storms and heavy precipitation, coastal flooding, and sea level rise. The ability to simulate and predict significant, long-term changes in these environmental variables important to energy-sector decisions required capabilities beyond the existing state-of-the-science Earth system models. The Energy Exascale Earth System Model (E3SM) project was conceived to meet this mission need (Bader et al., 2014).

Scientific developments in E3SM are dictated by three science drivers that broadly cover the foundational science for advancing Earth system prediction. Notably, water cycle, biogeochemistry, and cryosphere systems govern variability and changes in water availability and storms, air and river stream temperature, and coastal flooding and sea level rise that are all critical to the energy sector.

E3SM version 1 (E3SMv1) was branched from the Community Earth System Model (CESM1; Hurrell et al., 2013) but has evolved significantly since. E3SMv1 consists of three coupled modeling systems with varying degrees of sophistication. The present work describes the physical Earth system model that represents water and energy cycles in atmosphere, ocean, sea ice, land, and river components. This configuration is aimed at addressing the DOE water cycle science questions relating to interactions between the water cycle and the rest of the human-Earth system on local to global scales, water availability, and water cycle extremes. This physical model also serves as a foundation for two additional configurations: (i) a biogeochemistry configuration with interactive nitrogen and phosphorous for interactions between biogeochemical cycles and other Earth system components and (ii) a cryosphere configuration with added interactive ice shelf cavities for assessing the impacts of ocean-ice shelf interactions on Antarctic Ice Sheet dynamics and the implications for sea level rise.

The focus here is on the physical model at standard resolution useful for simulations like those specified in Coupled Model Intercomparison Project Phase 6 (CMIP6). This includes a 1° atmosphere and land (equivalent to 110 km at the equator), 0.5° river model (55 km), and an ocean and sea ice with mesh spacing varying between 60 km in the midlatitudes and 30 km at the equator and poles. A higher-resolution configuration with a 0.25° atmosphere and land, 0.125° river model, and ocean and sea ice with mesh spacing between 18 km at the equator and 6 km at the poles (roughly equivalent to a 0.1° resolution) will be documented in a subsequent paper.

Given that E3SMv1 features three specific models of components (ocean, sea ice, and river) that have never been used in a coupled Earth System Model and that there were significant developments in the atmosphere and land components, an examination of the model behavior relative to observations and other CMIP-class models is needed. We analyze CMIP6 Diagnosis, Evaluation, and Characterization of Klima (DECK) and historical simulations (Eyring et al., 2016) performed with E3SMv1. This allows for a rigorous comparison of E3SMv1 behavior against observations and many other models. This work will also provide a baseline for all future E3SM developments and experiments.

This paper is organized as follows. In section 2, we present an overview of the model components, with a specific focus on new model components (i.e., those not previously used in coupled Earth system modeling), as well as new developments in the atmosphere and land models. In section 3, we describe the E3SMv1 initialization and spin-up procedure, including model tuning objectives and simulation campaign. Section 4 provides an overview of the preindustrial control simulation and section 5 an analysis of the E3SMv1 climate in the historical simulations including short- and long-term variability. An extended discussion of E3SMv1's effective radiative forcing (ERF) and climate sensitivity is provided in section 6. Finally, we offer a summary of E3SMv1 fidelity and discuss future directions for the fully coupled model in section 7.

2 Model Overview

E3SM started with a version of the CESM1 (Hurrell et al., 2013, http://www.cesm.ucar.edu/models/cesm1.0) from which we developed the fully coupled E3SMv1 system. Notable changes between E3SMv1 and CESM1 include the following:
  1. E3SM Atmosphere Model (EAM) component with a spectral element dynamical core, increased vertical resolution, and substantially revamped physics and the capability of regional grid refinement for multiresolution simulations.
  2. New ocean and sea ice components based on the Model for Prediction Across Scales (MPAS) framework that uses Spherical Centroidal Voronoi Tessellations (SCVTs) for multiresolution modeling.
  3. The river transport model of CESM1 was replaced by a new river model, Model for Scale Adaptive River Transport (MOSART), for a more physically based representation of riverine processes.
  4. E3SMv1 land model (ELM) is based on the Community Land Model Version 4.5 (CLM4.5) with new options for representing soil hydrology and biogeochemistry added to enable analysis of structural uncertainty, with important implications to carbon cycle and climate feedbacks for addressing v1 biogeochemistry questions.

The subsections below provide a more detailed description of the model components. Coupling of the new MPAS-Ocean, MPAS-Seaice, and MOSART models with EAM and ELM provides E3SMv1 with a unique capability for multiresolution modeling using unstructured grids in most of its component models. This capability is critical for future simulation campaigns that have a strong regional focus to meet DOE's needs for Earth system modeling in support of energy-sector decisions.

The project also built a comprehensive infrastructure for code management, development, testing, and analysis to enable development of E3SMv1 and future versions at DOE leadership computing centers. Leveraging DOE investments, a flexible framework provides workflow orchestration, provenance capture and management, simulation analysis and visualization, and automated testing and evaluation capabilities (see Appendix Appendix C for a description of some of the analysis tools).

2.1 Atmosphere

The EAM is the atmosphere component of E3SMv1. It is a descendant of the Community Atmosphere Model version 5.3 (CAM5.3). EAM uses a spectral element dynamical core at 110-km resolution on a cubed sphere geometry. Vertical resolution was increased from 30 layers, with a top at approximately 40 km, in CAM5 to 72 layers with a top at approximately 60 km in EAM. EAM contains many innovations compared to CAM5. While changes in the EAM physics are broadly similar to changes from CAM5.3 to CAM6, many details of tuning, and cloud and aerosols formulations differ in important ways as described in Rasch et al. (2019). A detailed analysis of its cloud and convective characteristics and the rationale for model tuning are provided in Xie et al. (2018) and Zhang et al. (2019). Sensitivity of EAM to a number of its adjustable parameters is explored in Qian et al. (2018). EAM simulated diurnal cycle of precipitation is analyzed in Xie et al. (2019).

Key features of EAMv1 include the following:
  1. Implementation of a simplified third-order turbulence closure parameterization (CLUBB, Cloud Layers Unified By Binormals; Golaz et al., 2002; Larson, 2017; Larson & Golaz, 2005) that unifies the treatment of planetary boundary layer turbulence, shallow convection, and cloud macrophysics.
  2. An updated microphysical scheme, the version 2 of Morrison and Gettelman (2008; MG2; Gettelman et al., 2015). The combination of CLUBB and MG2 enables aerosol-cloud interactions in large-scale and shallow convective clouds to be considered. Significant additional changes to nucleation and ice microphysics have also been incorporated compared to MG2 in CAM6.
  3. The deep convection scheme of Zhang and McFarlane (1995).
  4. The Modal Aerosol Module (MAM4) with revisions to improve aerosol resuspension, convective transport, aerosol nucleation, scavenging, and modifications to sea spray emissions so marine ecosystems can contribute organic matter to aerosols.
  5. A linearized ozone photochemistry to predict stratospheric ozone changes, which provides an important source of stratospheric variability (Linoz v2, Hsu & Prather, 2009).

The computational cost of EAMv1 increased by approximately a factor of 4 relative to CAM5 due to higher vertical resolution and parameterization complexity, along with a larger number of predicted and transported variables (aerosol species and prognostic snow and rain).

2.2 Ocean

MPAS-Ocean, based on the MPAS framework (Ringler et al., 2010), is the ocean component of E3SMv1 (Petersen et al. 2019). MPAS-Ocean uses a mimetic finite volume discretization of the primitive equations and invokes the hydrostatic, incompressible, and Boussinesq approximations on a staggered C-grid (Arakawa & Lamb, 1977; Ringler et al., 2013; Thuburn et al., 2009). MPAS-Ocean grid cells for E3SMv1 simulations are near-hexagons (five or more sides), but the MPAS framework supports cells with any number of sides; the algorithms and code are identical for all cell shapes. The tracer advection scheme is the quasi third-order flux corrected transport (FCT) scheme (Skamarock & Gassmann, 2011) with separate limiting in the horizontal and vertical. The MPAS-Ocean time stepping method is split-explicit, where the barotropic component is subcycled within each baroclinic time step.

The simulations presented here use a z star vertical coordinate within an arbitrary Lagrangian-Eulerian scheme, where the layer thicknesses of the full column expand and contract with the sea surface height (Petersen et al., 2015; Reckinger et al., 2015). The prognostic volume-based equation of motion includes surface mass fluxes from the coupler; thus, virtual salt fluxes are not needed. Vertical mixing is computed implicitly at the end of each time step using the Community Vertical Mixing project implementation of the K-profile parameterization as described by Van Roekel et al. (2018) where our configuration of K-profile parameterization is based on the results of comparison against large eddy simulations.

E3SMv1 standard resolution simulations employ the classic Gent and McWilliams (1990) eddy transport parameterization. The Gent-McWilliams bolus coefficient was tuned, in part, to reduce the transport of heat to depth in the Southern Ocean, to a value of 1,800 m2/s for the simulations presented here. The Redi coefficient, which adds diffusion along isopycnal layers, was set to 0 for this set of simulations. In the horizontal, biharmonic viscosity is used for momentum (1.2 × 109m4/s2). No explicit horizontal tracer diffusivity is included.

2.3 Sea Ice

MPAS-Seaice is the sea ice component of E3SMv1. MPAS-Seaice and MPAS-Ocean share identical meshes, but MPAS-Seaice uses a B-grid (Arakawa & Lamb, 1977) with sea ice concentration, volume, and tracers defined at cell centers and velocity defined at cell vertices. Velocity components at cell vertices are not aligned with the mesh, as in sea ice models with structured meshes and quadrilateral cells. Instead, the velocity components are aligned with a spherical coordinate system that is locally Cartesian and has its poles on the geographical equator. Velocities are determined by solving the sea ice momentum equation (Hibler, 1979; Hunke & Dukowicz, 1997) on cell vertices in an identical manner to the CICE model (Hunke et al., 2015) except for the internal stress term. MPAS-Seaice uses the variational formulation of the internal stress term used by CICE (Hunke & Dukowicz, 2002) but modified to use the nonregular polygons of MPAS meshes. Instead of the bilinear basis functions used by CICE, MPAS-Seaice uses Wachspress basis functions (Dasgupta, 2003), which are integrated with the quadrature rules of Dunavant (1985). Horizontal transport of ice concentration, volume, and tracers is achieved with an incremental remapping scheme similar to that described in Dukowicz and Baumgardner (2000), Lipscomb and Hunke (2004), and Lipscomb and Ringler (2005) but adapted to MPAS meshes. MPAS-Seaice shares the same column physics code as CICE through the Icepack library (Hunke et al., 2018). For simulations shown here, MPAS-Seaice uses the “mushy layer” vertical thermodynamics scheme of Turner et al. (2013) and Turner and Hunke (2015), the level ice melt pond scheme of Hunke et al. (2013), a delta-Eddington shortwave (SW) radiation scheme (Briegleb & Light, 2007; Holland et al., 2012), a scheme for transport in thickness space (Lipscomb, 2001), and a representation of mechanical redistribution (Lipscomb et al., 2007).

Coupling of the sea ice component to the ocean takes advantage of z star ocean coordinates as described by Campin et al. (2008) and is a departure from the coupling of CICE and POP (Parallel Ocean Program) in CESM1. The weight of sea ice contributes to the ocean's barotropic mode, notably affecting the free surface over continental shelves. In shallow water depths at or less than the floating ice draft, the weight passed to the ocean model is limited to prevent evacuation of the underlying liquid column. When frazil ice forms in the ocean model, the volume of newly formed crystals is passed to the sea ice model with a fixed salinity of 4 PSU, rather than exchanging a freezing potential as in other models. Future versions of E3SM will permit progressive brine drainage to the ocean from the mushy layer physics used in MPAS-Seaice (Turner & Hunke, 2015). For E3SMv1, brine drainage occurs internally in MPAS-Seaice for thermodynamic calculations, but for the sake of freshwater coupling, the ocean model only receives mass fluxes back from melted sea ice at the fixed salinity that it originally passed to its cryospheric counterpart (4 PSU). The ocean temperature immediately under the ice is the same as the liquid phase in the lowest layer of the sea ice model and is not fixed at −1.8 °C as is typical of previous generation coupled models (Naughten et al., 2017). For the current version, we have addressed these long-standing ocean-ice coupling issues identified by the modeling community: explicit sea ice mass and salt exchange, a pressure force of the ice on the ocean, a basal sea ice temperature consistent with the ocean model's equation of state, and resolved inertial oscillations (Hibler et al., 2006; Lique et al., 2016; Schmidt et al., 2004).

2.4 Land

The E3SM Land Model, version 0 (ELMv0), is the land component in E3SMv1. ELMv0 adopts many of its capabilities from its parent model, the Community Land Model version 4.5 (CLM4.5). Oleson et al. (2013) provide a detailed technical description of that parent model, including descriptions of all fundamental equations representing the model's biogeophysical and biogeochemical dynamics. The model describes interactions of the land surface with the near-surface atmosphere and includes interactions among land subsystems such as vegetation, soil, snow, groundwater, runoff, urban areas, and managed ecosystems. These interactions are represented through conservation equations of state for energy, water, carbon, and nitrogen, including terms that couple these states and fluxes. While ELMv0 includes prognostic carbon and nitrogen biogeochemistry and dynamic ecosystem structure, the E3SMv1 water cycle experiments do not make use of these capabilities. Instead, these simulations use a static representation of land ecosystem structure with prescribed seasonal changes in vegetation canopies based on a climatology of satellite remote sensing data. This simulation mode is referred to as satellite phenology (SP) and is based on methods described by Lawrence and Chase (2007). In this simulation mode, regardless of long-term climate changes or short-term anomalies, the global distribution and seasonal variation of vegetation structure are constant over the course of the simulations. This includes model representation of vegetation height, timing and amount of displayed leaf area, and vegetation contributions to land surface albedo.

Several new developments have been made within ELMv0 since its branch point from CLM4.5. First, an extended representation of the influence of aerosols and black carbon (BC) deposition on snow was introduced, based on data summarized by Liu et al. (2012), causing aerosol optical properties within the snowpack to vary as a function of snow grain size and aerosol/ice mixing state (Flanner et al., 2012). This development also fixed a bug related to calculation of snow grain size following snow layer division. Second, a minor modification was made to reduce the rate of evaporation from area characterized as pervious road under dry conditions. Third, the numerical scheme for calculation of leaf stomatal conductance was updated to prevent nonphysical simulation of negative internal leaf CO2 concentrations. Fourth, albedo calculation was modified to return a land albedo of 1.0 in land cells and time steps when the sun is below the horizon.

2.5 River

E3SMv1 replaced the River Transport Model (Branstetter & Erickson, 2003), a linear reservoir routing model used in CESM1 and E3SMv0, with the Model for Scale Adaptive River Transport (MOSART), which uses a physically based kinematic wave approach for river routing across local, regional, and global scales (Li et al., 2013, 2015). In the standard E3SMv1 resolution, MOSART uses a regular latitude-longitude grid with spacing of 0.5°. Surface and subsurface runoff simulated by ELM are mapped from the ELM grid to the 0.5° latitude-longitude grid as input to MOSART, which routes the runoff and provides freshwater input to the ocean model. MOSART does not exchange water with the atmosphere or return water to the land model. MOSART divides each grid cell into three categories of hydrologic units: hillslopes that contribute both surface and subsurface runoff into tributaries, tributaries that discharge into a single main channel, and the main channel that connects the local grid cell with the upstream/downstream grid cells through the river network. Two simplified forms of the one-dimensional Saint-Venant equations are used to represent water flow over hillslopes, in the tributary, or in the main channels. MOSART only routes positive runoff, although spurious negative runoff can be generated occasionally by the land model. Negative runoff is mapped directly from the grid cell where it is generated at any time step to the basin outlet of the corresponding MOSART grid cell. More detailed descriptions of MOSART and its input hydrography data and channel geometry parameters can be found in Li et al. (2013). When driven by runoff simulated by CLM4 with observed meteorological forcing data, streamflow simulated by MOSART is shown to reproduce the observed annual, seasonal, and daily flow statistics at over 1,600 stations of the world's major rivers reasonably well in terms of the overall bias and the seasonal variation (Li et al., 2015).

2.6 Coupling and Performance

E3SMv1 uses component coupling software from the Common Infrastructure for Modeling the Earth. The top-level driver is cpl7 (Craig et al., 2012), which provides a main program for forming the single executable of E3SM, directs the time integration of the coupled model, and performs any necessary interpolation or time averaging needed between the components. Communication between the parallel components of E3SM and parallel interpolation is provided by the Model Coupling Toolkit (Jacob et al., 2005; Larson et al., 2005), which is included in Common Infrastructure for Modeling the Earth.

cpl7 implements an online-off-line method for interpolating values between different grids in E3SMv1. Grid intersections and interpolation weights are calculated with an off-line tool and then read in at runtime and applied during the coupled integration. cpl7 also allows flux and state variables to be interpolated with different weights. The number of different weight files needed is relatively small because the atmosphere and land models share a grid, as do the ocean and sea ice models. The interpolation weights for nearly all variables are calculated with the ESMF_RegridWeightGen program from ESMF (Collins et al., 2005) using the first-order conservative option. However, interpolation weights for state variables from the atmosphere to the ocean/sea ice grids are calculated using TempestRemap (Ullrich & Taylor, 2015). Unlike ESMF, the TempestRemap algorithms have native support for vertex centered finite element grids, such as used by the E3SM atmosphere dycore. Future versions of E3SM will use TempestRemap for all mapping weight calculations.

cpl7 allows several time sequencing options for the components and flexibility for how often the components communicate with the coupler. E3SMv1 uses “RASM_OPTION1" where the sea ice, land, and river runoff models execute simultaneously and in sequence with the atmosphere. The ocean model runs simultaneously with all four of those components. Model time steps are as follows:
  1. The main atmosphere physics time step is 30 min, but a few parameterizations use a different time step. CLUBB and MG2 microphysics are substepped together at a time step of 5 min. Radiation is updated hourly. Several layers of substepping are used by the atmosphere dynamics and tracer transport: The Lagrangian vertical discretization uses 15-min time steps, the horizontal discretization uses 5-min time steps, and the explicit numerical diffusion uses 100-s time steps.
  2. Ocean model time step is 10 min with a barotropic sub–time step of 40 s.
  3. Sea ice model time step is 30 min. The 30-min coupling time step between the sea ice and ocean permits transient inertial oscillations in the drift of ice. This can cause instabilities arising from frequent exchange of sea ice weight and sea surface height between sea ice and ocean. Thus, we damp the sea surface height gradient in the sea ice momentum equation with a 24-hr Newtonian relaxation constant.
  4. Land model time step is 30 min.
  5. River runoff model time step is 1 hr.

The coupling frequency for all components is 30 min except the river runoff model which communicates every 3 hr.

Simulations in this work were performed on DOE's National Energy Research Scientific Computing Center (NERSC) Edison supercomputer (http://www.nersc.gov/users/computational-systems/edison). Using 285 nodes (each consisting of two 12-core Intel “Ivy Bridge" processors at 2.4 GHz) with 24 MPI (Message Passing Interface) tasks per node, the coupled model performance averages 10 simulated years per day. Figure 1 illustrates the sequencing, processor layout and relative cost for all components in E3SMv1.

Details are in the caption following the image
Typical processor layout, performance, and sequencing for all components in E3SMv1. Components include coupler (CPL), atmosphere (ATM), ocean (OCN), sea ice (ICE), land (LND), and river runoff (ROF). The hatched area represents unoccupied time. OCN runs concurrently on its own nodes and is load-balanced with ATM + ICE, which run sequentially. Driver overhead is 4%. Other layouts are possible, but we have found this to be the most robust and efficient.

3 Tuning and Initialization

3.1 Final Coupled Model Tuning

Tuning is an integral part of climate model development (e.g., Hourdin et al., 2017; Schmidt et al., 2017). While most of the tuning for E3SMv1 was performed at the model component level, some additional retuning was subsequently required for two reasons: (1) Code errors were discovered in the atmosphere model physics, while the coupled system was being developed and (2) the coupled tuning objectives required an additional level of adjustment to accommodate biases present in other model components and interactions between components. Two minor code errors were discovered and addressed in EAMv1:
  1. An incorrectly positioned parenthesis in the code designed to prevent overdepletion of rain number when multiple microphysics processes act to concurrently deplete rain drops (https://github.com/E3SM-Project/E3SM/pull/1599) led to a reduction in the rain drop number concentration process rate. Correcting the error produced a degradation in overall precipitation pattern, approximately compensated for through increasing the efficiency of convective precipitation evaporation with zm_conv_ke (see Table 1).
  2. Snow crystal number concentration was updated improperly when snow sublimation occurred (https://github.com/E3SM-Project/E3SM/pull/1765). Fixing the error decreased the ice water path, weakening both longwave (LW) and SW cloud radiative effects, which was then countered by a reduction in the sulfate threshold ice nucleation size so4_sz_thresh_icenuc to increase ice number concentration (Table 1).
Table 1. Atmospheric Physics Parameters That Were Altered During the Development of the Coupled Model
Parameter Initial Final Impact Motivation
zmconv_ke 1.5E−06 5.0E−06 Deep convection rain evaporation MG2 bug fix 1
so4_sz_thresh_icenuc 7.5E−08 5.0E−08 SO4 ice nucleation threshold size MG2 bug fix 2
clubb_c14 1.30 1.06 CLUBB dissipation for urn:x-wiley:jame:media:jame20860:jame20860-math-0001 and urn:x-wiley:jame:media:jame20860:jame20860-math-0002 Final coupled tuning
A final tuning of the coupled model with preindustrial (perpetual 1850) forcings was performed to achieve the following:
  1. A near-zero long-term average net top-of-atmosphere (TOA) energy flux.
  2. Minimum long-term drift in global mean surface air temperature.
  3. Reasonable absolute global mean surface air temperature.

This final tuning required adjustments of less than 1 W/m2 because the E3SM atmosphere component was already well tuned from simulations with prescribed SSTs and sea ice concentrations. To simplify the process, we chose to focus on a single parameter in the CLUBB atmosphere turbulence parameterization, clubb_c14, which directly impacts dissipation of the horizontal components of the turbulent kinetic energy. Because of the tight coupling between turbulent kinetic energy and boundary layer clouds, clubb_c14 modulates low-clouds, thus affecting net TOA mostly through SW cloud radiative effects.

While most of the coupled tuning was performed under preindustrial conditions, we also performed a few additional simulations to monitor climate sensitivity and total ERF during the developmental phase of E3SMv1. We evaluated sensitivity using idealized +4-K simulations (Cess et al., 1989) and one abrupt quadrupling of CO2. Total ERF was estimated using pairs of atmosphere simulations with identical SSTs but differing forcing (climatological 2000 vs. 1850). One historical test simulation was also performed with a near final version of E3SMv1 (“beta3rc10” below). The outcomes of these intermediate simulations were consistent with the final model. A pragmatic, but deliberate, decision was made not to attempt to reduce the high climate sensitivity or reduce the aerosol forcing.

3.2 Spin-Up and Initialization

Since the coupled ocean and sea ice system take centuries to spin-up, we chose to accelerate the process by simultaneously performing model spin-up and final coupled tuning. MPAS-Ocean was initially spun up for one year in stand-alone mode, from the Polar science center Hydrographic Climatology data set (Steele et al., 2001) from rest. During this spin-up, sea surface temperature (SST) and salinity were relaxed to an annual mean climatology (from Polar science center Hydrographic Climatology) and the currents were forced by an annual averaged CORE-II normal year wind stress (Large & Yeager, 2009). This was followed by a 10-year Coordinated Ocean-ice Reference Experiments (CORE-II), interannually varying, forced (Large & Yeager, 2009) ocean and sea ice simulation. The ocean and sea ice state at the end of that simulation served as initial conditions for a series of sequential fully coupled simulations with preindustrial forcing (Figure 2).

Details are in the caption following the image
Time evolution of annual global mean surface air temperature for the spin-up simulations. Year 1 marks the first year of CMIP6 DECK piControl.

The first of these simulations consisted of 50 years with a prototype E3SMv1 configuration (“FCT2”). Due to the change in forcing from present-day CORE-II to preindustrial coupled, the simulation experienced a rapid cooling during the first 20 years followed by a relative stabilization. Ocean and sea ice states after 50 years served as initial conditions for a new coupled simulation (“beta3rc10”) that included bug fixes and associated retuning described in Table 1, full implementation of the CMIP6 piControl forcings, and a retuning of clubb_c14 from 1.3 to 1.2 based on a few previous tuning attempts with the “FCT2” model configuration. This beta3rc10 configuration was run for a total of 260 years; it exhibited excessive net influx of energy at TOA and a long-term warm drift with averaged temperature approaching present-day values near the end, even with preindustrial forcings. As a result, a new configuration (“beta3rc11”) was branched off after 150 years reusing all components states as initial conditions, but with a slight retuning of clubb_c14 (1.2 to 1.1). This new configuration was run for a total of 100 years. Initially, it cooled slightly from the impact of the retuning, then drifted warm, albeit at a smaller pace than its predecessor. The process was repeated once again with “beta3rc13,” further reducing clubb_c14 from 1.1 to 1.08. This last simulation was run for a total of 80 years. The drift was smaller than the preceding attempt, but 80 years was too short to conclude that the drift had been removed entirely.

At that point, a decision was made to incorporate a minor code change in the atmospheric dynamical core. This change was a workaround for a compiler bug (https://github.com/E3SM-Project/E3SM/pull/1922). In the interest of time, two new simulations were started simultaneously from the end of beta3rc13, one with identical tuning (“v1a”) and one with a further retuning of the clubb_c14 parameter from 1.08 to 1.06 (“v1b”). After 110 years, both simulations were examined and “v1b” was selected as the official E3SMv1 tuning because it exhibited the smaller drift of the two.

3.3 Simulation Campaign

Table 2 summarizes the E3SMv1 simulation campaign. All simulations were configured to adhere to the CMIP6 DECK specifications (Eyring et al., 2016) as closely as possible (see Appendix Appendix B for details about input data). piControl spans a total of 500 years. Initial conditions for the idealized 1%/year CO2 increase (1pctCO2) and abrupt CO2 quadrupling (abrupt-4xCO2) simulations, as well as the first member of the historical simulations (historical_H1) were taken from 1 January of year 101 from piControl. Subsequent members were branched every 50 years for a total of five ensemble members. Atmospheric Model Intercomparison Project (AMIP) simulations (prescribed SST) were also performed to cover the entire period for which CMIP6 provides surface boundary conditions (1870–2014). Atmosphere and land initial conditions for amip_A1 were taken from year 1870 of historical_H1, and similarly for the other members.

Table 2. Summary of E3SMv1 Simulations
Label Description Period Ens. Initialization
piControl Preindustrial control 500 years Preindustrial spinup
(section 3.2)
1pctCO2 Prescribed 1%/year CO2 increase 150 years 1 piControl (101)
abrupt-4xCO2 Abrupt CO2 quadrupling 150 years 1 piControl (101)
historical_Hn Historical 1850–2014 5 piControl (101, 151,
201, 251, 301)
amip_An Atmosphere with prescribed SSTs and sea ice concentration 1870–2014 3 historical_Hn (1870)
amip_1850allF_An Same but with all forcings held at 1850 values 1870–2014 3 historical_Hn (1870)
amip_1850aeroF_An Same but with all aerosol forcings held at 1850 values 1870–2014 3 historical_Hn (1870)
  • Note. E3SMv1 = Energy Exascale Earth System Model version 1; SST = sea surface temperature.

The entire simulation campaign was performed on NERSC Edison. E3SMv1 did not experience any internal failures during the entire 2,930 simulated years. The only failures were system related (generally node failures or file system issues), and the model could always be restarted from the last available set of annual restart files. The source code git hash for the piControl simulation was 2e145acf(https://github.com/E3SM-Project/E3SM/commit/2e145acf) and for the remaining simulations was 7de18fc7 (https://github.com/E3SM-Project/E3SM/commit/7de18fc7). The difference in source code version arises from code modifications necessary to support transient simulations that were not in place when the piControl simulation was started. The newer 7de18fc7 code bit-for-bit reproduces 2e145acf for piControl. A maintenance branch (maint-1.0; https://github.com/E3SM-Project/E3SM/tree/maint-1.0) has also been specifically created to reproduce these simulations. Bit-for-bit results on NERSC Edison will be maintained on that branch for as long as the computing environment supports it.

4 Preindustrial Control Simulation

The preindustrial control simulation (piControl) is in energy balance at TOA with an average loss of 0.011 W/m2 over the course of 500 years (tuning objective 1; Figure 3a) and almost no long-term linear trend. Among all the model components, the ocean constitutes the largest reservoir of heat. It takes up heat at an average rate of 0.016 W/m2, leaving a small net imbalance of 0.027 W/m2, either from changes in other components or energy nonconservation. Since this imbalance is sufficiently small compared to anthropogenic forcings of interest, E3SMv1 can be regarded as essentially conserving energy. We note, however, that developmental versions of E3SMv1 suffered from much larger imbalances (on the order of 0.5 W/m2). That imbalance was caused by inconsistent definitions of energy in the ocean and the atmosphere, with the ocean properly accounting for changes in water heat content with temperature while the atmosphere did not. The 0.5-W/m2 imbalance was deemed too large to ignore, but rewriting the atmosphere physics more consistently was impractical due to time constraints. As a result, we decided to incorporate an ad hoc correction term. An additional surface (sensible heat) source is introduced to the atmosphere that accounts for the missing energy carried by water molecules leaving the ocean at one temperature and returning (as condensed water) at another temperature. The apparent flux is small (0.4 W/m2) and applied uniformly at every point globally. The correction does not impact the simulated climatology, but its neglect would had led to a long-term drift in the global mean temperature of a few tens of degree by century. With this correction, both atmosphere and ocean communicate consistent energy loss and gain with each other (see Appendix Appendix A for more detail).

Details are in the caption following the image
Time evolution of annual (a) global mean net top-of-atmosphere (TOA) radiation (positive down), (b) global mean surface air temperature, and (c) maximum and minimum of total sea ice area for the Arctic and Antarctic in the piControl simulation. Dashed lines in (a) and (b) represent linear trends. The solid straight line in (a) is the mean TOA energy imbalance of −0.011 W/m2.

The global mean surface air temperature is very stable over the course of the control simulation (tuning objective 2; Figure 3b). There is a slight positive temperature trend of +0.013 °C/century over the entire 500 years and a much smaller trend of −0.003 °C per century over the last 400 years. We note that all additional simulations were branched from piControl in year 101 or later (Table 2), thus ensuring that forcing perturbations are applied atop a nondrifting control simulation. This facilitates subsequent analyses, since it is not necessary to correct for a long-term drift in the underlying control. Another measure of the stability of the climate in the piControl simulation is provided by the annual maximum and minimum sea ice area in Arctic and Antarctic (Figure 3c). Both hemispheres have little long-term drift in their seasonal cycle of sea ice extent.

Among 42 CMIP5 models analyzed by Hawkins and Sutton (2016), a majority of them have preindustrial global mean surface temperatures between 13 and 14 °C, with some as low at 12 °C and as high as 15 °C. With a long-term average surface temperature of 13.8 °C, E3SMv1 falls within the range of the bulk of CMIP5 models (tuning objective 3). This value is also consistent with estimated warming and the present-day global temperature of 14.0 ± 0.5 °C by Jones et al. (1999) for the period 1961–1990 and with leading reanalyses data sets (14.3 to 14.6 °C) for the period 1979–2008 (Hawkins & Sutton, 2016).

Water conservation is also important for a global Earth system model. During the early development of the E3SMv1 model, we identified a number of computational problems, leading to water conservation errors in the atmosphere component (Zhang et al., 2018). These problems were all considered and suitable remedies applied (e.g., borrowing from adjacent cells to avoid more drastic nonconservative fixers) so that in the 500-year piControl simulation, the relative water conservation error (as defined in Zhang et al., 2018) in the atmosphere component is about 0.00226%, equivalent to a computational sea level rise of about 2 mm per century.

Additional problems were also identified in the fully coupled system, including nonconservation caused by inconsistent remapping in the exchange of water between model components, excessive storage capacity in the river routing model, missing perched drainage and ponding in the land surface model, and grid definition inconsistencies between different model components. Some of these errors caused nonconservation errors in sea level rise as large as tens of meters per century but all were resolved and corrected. Unfortunately, a smaller nonconservation error was not uncovered until after E3SMv1 was frozen and the DECK simulations were complete. It manifests itself by a steady loss of water in the ocean of 5 cm per century in the piControl simulation. We note that the water loss does not involve any phase change and therefore does not impact TOA energy balance; however, it does impact ocean heat content. The root cause has since been traced to a nonphysical update of water stored in the unconfined aquifer of urban subgrid land units within the land model. This error, which has been corrected for future versions of E3SM (https://github.com/E3SM-Project/E3SM/pull/2603), was partly masked by incomplete and inconsistent water budget checks within the land model and the coupler. These budgets have also been corrected.

5 Historical Simulations

5.1 Atmosphere Climatology

We first evaluate the atmosphere in E3SMv1 with climatologies from the final 30 years (1985–2014) of simulations. Five ensemble members of the historical coupled experiment (H1 to H5) and three ensemble member AMIP simulations (A1 to A3) are evaluated.

The net TOA radiative flux (positive down) is shown in Figure 4 compared to CERES-EBAF Ed4.0 (Loeb et al., 2009). Bias patterns are qualitatively similar between the atmosphere and coupled simulations, indicating a strong imprinting of atmosphere biases on the coupled system, but the biases are frequently larger in the coupled simulations, where component interactions play a role (root-mean-square error [RMSE] of 9.13 vs. 7.80 W/m2). Significant positive biases are seen in the subtropical stratocumulus regions off the west coasts of North America, South America, and Africa due to an underestimate of cloudiness. The underpredicted coastal stratocumulus is due to both the use of CLUBB and model retuning (Xie et al., 2018). Southern Oceans are also marked by positive biases due to clouds. The remaining oceanic regions have either neutral or negative biases. Tropical land masses have generally positive biases, while northern high latitudes suffer from negative ones. The global mean net flux is +0.32 W/m2 for the coupled simulations and +0.15 W/m2 for the AMIP simulations, both slightly smaller than the observational estimates of 0.86 W/m2 from CERES-EBAF Ed 4.0. (uncertainty range of 0.2 to 1.0; Wild et al., 2012). We note that the present-day energy imbalance was not a target of model tuning. Instead, we tuned the preindustrial control simulation to have a net zero global mean TOA flux. The imbalance for 1985–2014 emerges from the model evolution in response to historical forcings in the case of the coupled simulations or as a result of imposing present-day SST and sea ice concentrations (as boundary conditions) for the AMIP simulations.

Details are in the caption following the image
Annual net top-of-atmosphere (TOA) radiative flux: (a) CERES-EBAF Ed4.0 observational estimate, (b) model bias from the three Atmospheric Model Intercomparison Project (AMIP) ensemble simulations, and (c) model bias from the five ensemble historical coupled simulations (1985–2014). RMSE = root-mean-square error.

Biases in cloud radiative effects (CRE) at TOA compared to CERES-EBAF Ed4.0 (Loeb et al., 2009) are shown in Figure 5 for SW and LW. Many regional biases apparent in net TOA (Figure 4) can be traced to SW cloud biases. LW CRE reveal additional biases, such as a lack of LW cloud trapping in the tropics from high clouds and excessive LW trapping from northern high latitudes clouds. As indicated in Xie et al. (2018), high clouds are significantly reduced in the tropical deep convection regions due to the increase of model vertical resolution from 30 levels to 72 levels in EAM, which results in a much weaker LW CRE over these regions. Global mean CRE for both SW and LW are approximately 3 W/m2 below the observational estimates.

Details are in the caption following the image
Annual top-of-atmosphere cloud radiative effect model biases (W/m2) of the five ensemble historical simulations (1985–2014) against CERES-EBAF Ed4.0: (a) shortwave (SW) and (b) longwave (LW). RMSE = root-mean-square error.

Annual precipitation is depicted in Figure 6 compared to Global Precipitation Climatology Project (GPCP) v2.2 (Adler et al., 2003; Huffman et al., 2009). Simulated global mean precipitation is slightly above 3 mm/day for both AMIP and coupled simulations, approximately 15% larger than the GPCP estimate. A comprehensive review of 30 currently available global precipitation data sets found large differences in the magnitude of global land annual precipitation estimates (Sun et al., 2018). In their estimate of the global energy cycle, Wild et al. (2012) note that the GPCP estimate may be too low due to systematic underestimations in the satellite retrievals (Stephens et al., 2012; Trenberth et al., 2009). Wild et al. (2012) estimated the global mean precipitation at 2.94±0.17 mm/day, while the estimate from Stephens et al. (2012) is 3.04 ± 0.35 mm/day. The global mean precipitation from E3SMv1 falls within both estimates.

Details are in the caption following the image
Annual precipitation rate (mm/day): (a) Global Precipitation Climatology Project v2.2 observational estimate, (b) model bias from the three Atmospheric Model Intercomparison Project (AMIP) ensemble simulations, and (c) model bias from the five ensemble historical coupled simulations (1985–2014). RMSE = root-mean-square error.

Major regional precipitation biases in AMIP simulations (Figure 6b) include a dry Amazon, excessive precipitation over elevated terrain (e.g., Andes and Tibetan Plateau), wet biases over tropical Africa and the Indian Ocean, and a dry bias over the central United States (traced to the summer time). Coupled simulations (Figure 6c) tend to amplify regional biases present in AMIP simulations (except the central United States), as well as develop biases typical of coupled simulations: double Intertropical Convergence Zone and excessive precipitation over the maritime continent. Unsurprisingly, the RMSE increases from 0.93 to 1.13 mm/day and the correlation decreases (0.91 vs. 0.86) between AMIP and coupled simulations.

Figure 7 shows the annual, zonally averaged temperature for the coupled simulations compared to ERA-Interim reanalysis (Dee et al., 2011). Overall, the model captures the thermal structure of the atmosphere. There is a tropical cold bias in the upper troposphere, which correlates with insufficient tropical high clouds (Figure 5b). In the northern midlatitudes, the cold bias extends to the surface because of the colder than observed northern latitudes SSTs (see section 5.2 below). There is also a significant warm bias in the lower portion of the troposphere in the southern high latitudes. The corresponding zonal wind is shown in Figure 8. The locations of the maximum jet locations are displaced equatorward, and the wind magnitude is also too large throughout the midlatitude troposphere. Tropical easterlies are too weak in the upper troposphere and too strong near the surface.

Details are in the caption following the image
Annual zonally averaged temperature (K): (a) ensemble mean of historical coupled simulations (1985–2014), (b) ERA-Interim reanalysis, and (c) model bias. RMSE = root-mean-square error. Vertical coordinate is pressure level (hPa).
Details are in the caption following the image
Annual zonally averaged zonal wind (m/s): (a) ensemble mean of historical coupled simulations (1985–2014), (b) ERA-Interim reanalysis, and (c) model bias. RMSE = root-mean-square error. Vertical coordinate is pressure level (hPa).

The evaluation above provides only a limited view of the performance of E3SMv1 from an atmospheric perspective. For a more exhaustive evaluation, we turn to a comparison with an ensemble of 45 CMIP5 models using metrics computed with the PCMDI Metrics Package (PMP; Gleckler et al., 2008, 2016). The comparison covers the period 1981–2005 of the historical simulations. The historical and AMIP ensemble members of E3SMv1 were also processed with PMP. Figure 9 shows global RMSE for the CMIP5 ensemble using box and whiskers plot, as well as the individual E3SMv1 historical simulations with blue dots and AMIP with red dots. Spatial RMSE against observations collected by PMP are shown for nine fields and each one of them for annual and seasonal averages. Lower values are better. For TOA radiation fields (Figures 9a–9c), E3SMv1 coupled (blue) generally falls within the lowest (best) quartile and is even competitive with some of the best CMIP5 models for certain fields and seasons. We note that we are comparing a newer model against older ones, so we do not expect this to necessarily hold for CMIP6. For surface variables, precipitation (Figure 9d), surface air temperature over land (Figure 9e), and zonal wind stress over ocean (Figure 9f), E3SMv1's coupled performance is better than the ensemble median and often falls within the lowest quartile, with the exception of surface air temperature over land during December–February and March–May (MAM). Precipitation during MAM is also notably worse than other seasons relative to the CMIP5 ensemble. For dynamical variables, zonal wind at 200 and 850 hPa (Figures 9g and 9h) and 500-hPa geopotential height (Figure 9i), E3SMv1 is generally better than the CMIP5 median, except again for MAM.

Details are in the caption following the image
Comparison of RMSE (1981–2005) of an ensemble of 45 Coupled Model Intercomparison Project Phase 5 models (box and whiskers showing 25th and 75 percentiles, minimum and maximum) with the five E3SMv1 historical members (blue dots) and the first member of the Atmospheric Model Intercomparison Project simulations (ret dots). Spatial RMSE against observations are computed for annual and sesonal averages with the PCMDI Metrics Package (Gleckler et al., 2016). Fields shown include TOA net radiation (a), TOA SW and LW cloud radiative effects (b, c), precipitation (d), surface air temperature over land (e), zonal wind stress over ocean (f), 200- and 850-hPa zonal wind (g, h), and 500-hPa geopotential height (i). TOA = top-of-atmosphere; SW = shortwave; CRE = cloud radiative effects; LW = longwave; DJF = December–February; MAM = March–April; JJA = June–August; SON = September–November; RMSE = root-mean-square error.

Unsurprisingly, AMIP simulations (red) perform better than their coupled counterparts (blue). However, the relative degradation between AMIP and coupled helps attribute sources of errors. For example, the difference is relatively small for the zonal mean wind, and thus, improving overall performance would likely require atmospheric improvements. On the other hand, surface variables, in particular precipitation and temperature, are much more strongly affected by the coupled model errors that emerge. The MAM seasonal deficiency also appears to be rooted in coupling errors.

In summary, E3SMv1 performs better than the median of the of CMIP5 ensemble for most atmospheric fields and seasons, which helps establish the credibility of the E3SMv1 simulated climate.

5.2 Ocean Climatology

An annual climatology (1985–2014) of the E3SMv1 ensemble mean of SST is shown in Figure 10, compared to the Hadley-National Oceanic and Atmospheric Administration (NOAA)/OI merged data product (Hurrell et al., 2008) averaged over the same period. Overall, E3SMv1 captures the observed SST well, with an ensemble mean bias of 0.094 °C and an ensemble mean RMSE of 0.939 °C. A few biases do emerge. First, there is a cold bias in the North Atlantic associated with excessive sea ice in the Labrador Sea. Consistent with this sea ice bias, these cold SST biases are stronger in the first half of the year than in the second half (not shown). While the exact cause of the Labrador Sea ice bias is unknown, it is likely that missing critical heat transports (e.g., from the east and west Greenland currents, Irminger current, and the Northwest corner), perhaps from low resolution or excessive horizontal viscosity (Jochum et al., 2008) play an important role. Second, there are warm biases on the eastern sides of ocean basins, coincident with SW CRE biases discussed in section 5.1. Finally, the Southern Ocean is also ≈2 °C warmer than observed. It is possible this last bias is associated with the relatively large value for the Gent-McWilliams bolus coefficient used, which was chosen to prevent excess heat transport to the deep ocean (see section 2.2).

Details are in the caption following the image
Annually averaged sea surface temperature climatology (1985–2014) for (a) E3SMv1 historical simulation (ensemble mean), (b) Hadley-National Oceanic and Atmospheric Administration/OI merged sea surface temperature data set (1985–2014; Hurrell et al., 2008), and (c) model bias. In the upper right corner of panel (c), the mean bias is shown in square brackets and the root-mean-square error is in angular brackets. For each error, the min, mean, max are listed for the historical ensemble.

The E3SMv1 sea surface salinity bias is shown in Figure 11. The model output is compared to the 2011–2014 National Aeronautics and Space Administration (NASA) Aquarius data (Lagerloef et al., 2015). Overall, the surface ocean is too fresh, with the largest bias in the Labrador Sea. The two minor exceptions to this pattern are the positive salinity bias region over the west Pacific warm pool, which is coincident with the atmospheric precipitation maximum being shifted westward relative to observations (Figure 6) and a positive salinity bias in the Arctic.

Details are in the caption following the image
As in Figure 10 but for sea surface salinity (PSU). The data set in (b) is from National Aeronautics and Space Administration Aquarius data (averaged 2011–2014).

The ensemble average mixed layer depth (MLD) annual climatology, based on a critical density threshold (σc = 0.03 kg/m3), is presented in Figure 12. The model output is compared to data described by Holte et al. (2017). The globally averaged model MLD is too shallow relative to observations, with the largest bias coincident with the large fresh bias in the North Atlantic. Unlike many other CMIP5 models (e.g., Sallée et al., 2013), the E3SMv1 MLD in the Southern Ocean is slightly deeper than observed. This is due to the region of deeper mixed layers in E3SMv1 being broader than observed (Figure 12a), possibly due to a positive bias in the Southern Ocean wind stress. The maximum MLD simulated by E3SMv1 is still much shallower than observations, which is consistent with other CMIP5 models.

Details are in the caption following the image
As in Figure 10 but for the annual average mixed layer depth. The data in (b) are from Holte et al. (2017). Data have been averaged from 2001 to 2017.

Figure 13 shows the E3SMv1 maximum Atlantic Meridional Overturning Circulation (AMOC) at 26°N, the site of the RAPID array (Smeed et al., 2017). The mean AMOC simulated in the historical ensemble (approximately 11 Sv) is weaker than the observed mean (16.9 ± 3.35 Sv) and also on the weak end of CMIP model AMOC strength (e.g., Cheng et al., 2013, their Figure 1). There are a number of possible causes for the weak AMOC in E3SMv1. Currently, MPAS-Ocean utilizes a z star coordinate (Adcroft & Campin, 2004), which is broadly consistent with a traditional z level coordinate in the deeper ocean. Z coordinate models are known to experience spurious diapycnal mixing (e.g., Griffies et al., 2000), which could reduce AMOC strength. Second, recent work has shown that Nordic overflows (Wang et al., 2015) and Arctic freshwater transports (Wang et al., 2018) have a strong impact on AMOC. At low resolution, these processes are poorly represented, where the latter is likely too strong as critical passageways being too wide (e.g., Davis Strait). Finally, since E3SMv1 exhibits excess sea ice in the Labrador sea, the simulated MLD (Figure 12) is reduced, which reduces deep convection and hence AMOC strength.

Details are in the caption following the image
Annually averaged maximum Atlantic Meridional Overturning Circulation (MOC) at 26.5°N below 500-m depth. The solid black line represents the ensemble mean and the spread of the shaded gray region represents the maximum and minimum of the ensemble. The observed AMOC and standard deviation at the RAPID array are shown by the vertical bar (16.9 ± 3.35 Sv.).

5.3 Sea Ice

Sea ice in E3SMv1 is too extensive at the end of winter and too confined in late summer, reflecting a heightened model seasonality relative to both the Arctic and Southern Ocean passive microwave record, regardless of the algorithm used to derive ice area from these retrievals (e.g., Cavalieri et al., 1996; Meier et al., 2017). This result is summarized in Figures 14-16. Embedded within this heightened seasonality, the melt season minimum is delayed in the Northern Hemisphere relative to observations. In the high north, the heightened winter seasonal extent relative to observations occurs mainly in the Labrador Sea, as well as the Iceland Sea and Pacific margin of the Sea of Okhotsk, evident in the historical five-member ensemble means (Figures 14a and  14b). Comparison with satellite-derived albedo, shown for June to August in Figure 17, suggests that surface radiative processes leading to an albedo bias in the central arctic are unrelated to the physical exchanges responsible for the Labrador sea bias. As a result, the Labrador Sea bias is unlikely to be attributable to MPAS-Seaice alone and is more likely a result of coupled interactions and unresolved heat advection in the region.

Details are in the caption following the image
Northern Hemisphere ensemble mean March and September sea ice thickness for two decades leading up to year 2000 (a and c), and the first 15 years of the 21st century (b and d). Model ice thickness is truncated at 15% concentration, and magenta represents the Meier et al. (2017) National Oceanic and Atmospheric Administration Climate Data Record ice extent for the same averaging periods. Grid density on the polar stereographic projection is indicated by cell translucence. Numbers in the lower right corner of each panel indicate the mean hemispheric ice thickness thickness for the thinnest ensemble member (blue), the multiensemble mean rendered in the map (black), and the thickest ensemble member (red) for the each period.

The model possesses a negative climatological trend in Arctic sea ice thickness that is qualitatively consistent with observed basin-wide trends (e.g., Kwok & Rothrock, 2009), as quantified in Figure 14. The 1980–1999 hemispheric mean spread among the five ensemble members for March (September) decreases from 1.5–1.71 to 1.16–1.37 m (1.55–2.12 to 0.79–1.35 m) for the period 2000–2014, which includes the thinnest periods known in the Arctic sea ice thickness record. Still, the spatial ice thickness pattern in the central Arctic does not reflect the predominant buildup of thick ice against the Canadian Archipelago seen in observational estimates, including those of Bourke and McLaren (1992), Kwok and Cunningham (2008), and Tilling et al. (2015). This represents a second sea ice bias being addressed in ongoing improvements to the polar components of E3SM. There is no significant climatological trend in mean Southern Ocean sea ice thickness from the historical simulations, in the ensemble mean or spread (Figure 15). Extent bias stemming from the heightened polar seasonality of E3SMv1 is consistent across our chosen 1980–1999 and 2000–2014 analysis periods, seen in Figure 15. While the model does not replicate the observed climatological trend in austral sea ice extent (Figure 16), it does simulate the critical decrease in Northern Hemisphere minimum sea ice extent, which has had a significant impact on planetary albedo in the current century and is therefore an important contribution to the energy balance of E3SMv1 as a whole.

Details are in the caption following the image
As for Figure 14 but for the Southern Hemisphere.
Details are in the caption following the image
Maximum (a) and minimum (b) sea ice area observed during the year for the Northern (green) and Southern Hemispheres (blue). The shaded bounds represent the E3SMv1 historical ensemble spread, and the thick colored lines are National Aeronautics and Space Administration TEAM observations (Cavalieri et al., 1996). Model and observational data are monthly averages.
Details are in the caption following the image
Surface albedo bias (E3SMv1-CERES-EBAF) for Northern Hemisphere spring, averaged over 1985–2014.

5.4 Land and River

Figure 18 shows a comparison of the mean annual total (surface and subsurface) runoff simulated by ELM with the composite runoff map from the Global Runoff Data Center (GRDC; Fekete & Vörösmarty, 2011). Since the GRDC data only provide monthly runoff for 1986–1995, the comparison is shown for the annual total runoff averaged over the same period of the historical_H1 simulation. ELM captures the general spatial distribution of the GRDC runoff, but in relatively arid regions such as Australia and the western United States, ELM has wet biases, while in the Amazon tropical forest, dry biases are notable. These biases are consistent with the precipitation deficiencies shown in Figure 6.

Details are in the caption following the image
Mean annual total runoff (mm/day) simulated by (upper panel) E3SMv1 land model (ELM) and from (lower panel) Global Runoff Data Center (GRDC) averaged between 1986 and 1995.
The seasonal cycle of streamflow is an important metric of water availability, as water deficits resulting from a mismatch in the timing of water supply and water demand have important implications for energy and water. Figure 19 compares the seasonal cycle of simulated and observed streamflow at stream gauge stations in major river basins around the world. A seasonality index (SI) is defined as follows:
urn:x-wiley:jame:media:jame20860:jame20860-math-0003(1)
urn:x-wiley:jame:media:jame20860:jame20860-math-0004(2)
where Qij is the monthly streamflow for month i and year j and n is the number of years in the simulation or observation. With this definition, SI is equal to 1 if the monthly streamflow is uniformly distributed throughout the year and SI is equal to 12 if streamflow only occurs in 1 month. The seasonality of the simulated streamflow is generally comparable with that observed in terms of both magnitude and timing. For example, in North America, streamflow seasonality is stronger in the northwest with a peak timing between November and January but in the central and southeastern United States, streamflow seasonality is weaker with a peak timing generally in spring. In Asia, streamflow generally peaks in the late summer. Larger biases in seasonality are found in Australia (Murray Darling River) and central Asia (Yenisey) where biases in the runoff are also more significant (Figure 18).
Details are in the caption following the image
Seasonality of observed (upper) and simulated (lower) streamflow at stream gauge stations in major river basins around the world. The magnitude of seasonality indicated by the seasonality index (SI) is shown in color, and the timing of the peak streamflow is shown by the position of the arrows. SI is calculated based on monthly streamflow between 1986 and 1995.

5.5 Variability Simulated by E3SMv1

In this section we present a variety of metrics to assess E3SMv1 variability.

The long-term variability of the El Niño–Southern Oscillation (ENSO), as simulated in the piControl and historical ensemble simulations, is examined via wavelet analysis (Torrence & Compo, 1998) of Niño 3.4 SST and is shown in Figure 20. In this analysis, the piControl simulation is subdivided into five 100-year-long sections (as in Stevenson, 2012). For reference, we also include the spectrum from HadISST (Rayner et al., 2003) and ERSSTv4 (Huang et al., 2015). In both the piControl and historical ensemble, E3SMv1 ENSO variability is strong with statistically significant peaks near a 3-year period. We also see a signature of longer term modulation of ENSO variability (6- to 9-year period) in the piControl. This is consistent with other CMIP models (e.g., Stevenson, 2012; Stevenson et al., 2012; Wittenberg, 2009) and proxy data (e.g., McGregor et al., 2013). However, the modulation is a bit weaker than other models. Relative to the CESMv1 Large Ensemble (Kay et al., 2015), Figure 20b, where the CESMv1 PI control is subdivided as in E3SMv1, E3SMv1 variability is slightly closer to observations, but is shifted strongly to a three year period, whereas CESM-LE is dominant at approximately 4.5 years. In both models, the dominant ENSO peak remains consistent between the PI control and historical ensembles. We also note that the spread of ENSO variability in the E3SMv1 historical simulation is much smaller than in the CESM-LE. This is likely due to the small E3SMv1 ensemble size relative to the 39 member CESM-LE (Newman et al., 2018).

Details are in the caption following the image
El Niño–Southern Oscillation (ENSO; Nino3.4) variability of the pre-industrial (PI) control simulation and historical ensemble. The Morlet wavelet of degree 6 is used (e.g., Torrence & Compo, 1998) and the maximum and minimum wavelet power is used for each period. The PI control is subdivided into 100-year intervals to form a five-member ensemble (e.g., Stevenson, 2012). The solid line represents the mean wavelet power for the ensemble. The shading bounds the maximum and minimum power that is above the 90% significance threshold. The black and green lines are two observational data products. (a) E3SMv1 and (b) CESM-LE. SST = sea surface temperature; CESM = Community Earth System Model.

A number of CMIP5 models do not well represent the spatial pattern of ENSO variability, in particular its westward extent (e.g., Menary et al., 2018; Van Oldenborgh et al., 2005). Figure 21 shows the difference of a composite of the strongest El Niño events and strongest La Niña events (following Menary et al., 2018). Broadly, E3SMv1 reproduces the spatial pattern seen in observations well with a few notable differences. E3SMv1 does not capture the signal along the North American coast, suggesting a bias in the coastally trapped Kelvin waves. The westward extent is also larger than observed but better than seen in CESM-LE. Overall, the comparison with observations is good, with low RMSE and a high correlation coefficient.

Details are in the caption following the image
(a–c) Difference of composite El Ninõ events and composite La Ninã events (1950–2015) for the HadleyISST data set, the E3SMv1 historical ensemble, and the CESM-LE, respectively. El Ninõ events are defined as periods when the Ninõ 3.4 SST anomaly exceeds 0.8 °C for more than six consecutive months. The La Ninã criterion is Ninõ 3.4 SST anomaly less then −0.8°C for more than 6 months (these definitions are consistent with Menary et al., 2018). When an El Niño–Southern Oscillation event is identified, the SST is averaged from November to March. For model output, every ensemble member contributes to the mean composite. The Pearson correlation coefficient and RMSE are shown for E3SMv1 and CESM in (b) and (c). SST = sea surface temperature; CESM = Community Earth System Model; RMSE = root-mean-square error.

On subseasonal time scales the dominant mode of variability and predictability in the tropics is the Madden-Julian Oscillation (MJO; Waliser et al., 2003). The MJO is generally thought to play a role in ENSO initiation (McPhaden et al., 2006), monsoon active break cycles (Annamalai & Slingo, 2001), tropical cyclogenesis (Sobel & Maloney, 2000), and remote teleconnection effects (Vitart, 2017); therefore, its accurate simulation is key. The simulation of the MJO in E3SMv1 represents a significant improvement in strength, propagation characteristics, and the explained intraseasonal variance compared to CESM1 (Figure 22). However, significant biases in Pacific propagation remain. In CESM1, the intraseasonal propagation is in the wrong direction, westward, in the Indian Ocean. Weak correlations do make it over the Maritime Continent and into the West Pacific, but they are mostly decoupled from the SST signal. In contrast, E3SMv1 has consistent low-level wind propagation coupled in quadrature with the SSTs from the Indian Ocean to the central Pacific.

Details are in the caption following the image
Tropical lag correlation (averaged 10°N to 10°S) of precipitation (colors) and 850-mb zonal wind (lines) with precipitation in the Indian Ocean region (60°N to 90°E; shown by the vertical dashed green lines) for (a) observed (Tropical Rainfall Measuring Mission for precipitation Modern-Era Retrospective Analysis for Research and Applications [MERRA] for winds), (b) E3SMv1 preindustrial control, and (c) Community Earth System Model (CESM) preindustrial control. Data used are daily anomalies and band-pass filtered between 20 and 100 days.

5.6 Temperature Evolution Over the Historical Record

The 1850–2014 time evolution of the global mean surface air temperature anomaly from the E3SMv1 historical ensemble is compared against three observational products in Figure 23. Observations include NOAA National Climatic Data Center (Smith et al., 2008; Zhang et al., 2015), NASA GISTEMP (GISTEMP Team, 2018; Hansen et al., 2010), and HadCRUT4 (Morice et al., 2012), which are in good agreement with each other. Anomalies are computed with respect to 1880–1909, the earliest 30-year period when data are available from all observational products. The E3SMv1 historical ensemble mean is shown in red and the ensemble minimum and maximum in orange shading. While E3SMv1 captures the bulk of the observed warming between the 1850s and 2010s, the trajectory of the warming is at times inconsistent with observations. The ensemble overlaps with observations until the 1950s, but in the subsequent decades, E3SMv1 departs from observations, first remaining too cold for several decades before warming up too rapidly starting around year 2000. The low anomalies in E3SMv1 before 1960 result from a compensation between a downward trend in the Northern Hemisphere and a positive trend in the Southern Hemisphere (not shown).

Details are in the caption following the image
Time evolution of annual global mean surface air temperature anomalies (with respect to 1880–1909). Comparison between observations from National Oceanic and Atmospheric Administration (NOAA) National Climatic Data Center (NCDC; blue), National Aeronautics and Space Administration (NASA) GISTEMP (green), HadCRUT4 (gray) and E3SMv1 ensemble mean and range (red and orange).

We turn to a regional analysis to help elucidate this inconsistency. In Figure 24 we decompose the SST anomalies into two regions: the Northern and Southern Hemispheres. The E3SMv1 ensemble range is shaded (blue) and the CESMv1 Large Ensemble (CESM-LE; Kay et al., 2015) mean and range is also plotted (gray). CESM-LE can be regarded as a proxy for E3SMv0. Model results are compared to observations from the Hadley-NOAA/OI merged data product (Hurrell et al., 2008; red). Note here that, unlike in Figure 23, the anomalies are computed relative to the 1920–1950 period as the CESM data begins at 1920. In the southern basins, E3SMv1 and CESM represent the evolution of observed SST well, where the E3SMv1 SST is slightly closer to data. However, in the Northern Hemisphere, the E3SMv1 SST anomalies decrease in the 1950s and warm quickly after the 1990s, similar to what is observed in the surface air temperature (Figure 29). Further, the warming effect is strongest in the North Atlantic. This pattern is not seen in the data. We discuss possible causes in the next section.

Details are in the caption following the image
Regionally averaged sea surface temperature anomalies from 1850–2014 for E3SMv1 historical ensemble (blue), Hadley-National Oceanic and Atmospheric Administration/OI merged sea surface temperature product (red), and the CESMv1 large ensemble (Kay et al., 2015; gray). For the ensemble data, the ensemble maximum and minimum are shown. All anomalies are relative to (1920–1950), which is the beginning of the CESMv1 record. (a) Northern Hemisphere and (b) Southern Hemisphere.

6 Radiative Forcings and Sensitivity

We aim to better understand E3SMv1 simulated warming over the historical record by analyzing the model radiative forcings and sensitivity.

6.1 ERF

ERF is the change in net downward TOA radiation due to observed changes in forcing agents (such as aerosol and greenhouse gases, GHGs) but not SST (allowing for adjustments in atmospheric temperatures, water vapor, and clouds; Intergovernmental Panel on Climate Change, IPCC, 2013, p. 665). A common approach to estimate ERF is by differencing net TOA fluxes in a pair of atmosphere-only simulations with identical SSTs and sea ice concentrations but different radiative forcings (e.g., Hansen et al., 2005). Alternatively, ERF can also be estimated from regression-based approaches (Gregory et al., 2004). Forster et al. (2016) contrast several methodologies to compute ERF and generally recommend fixed SST methodologies.

Total ERF measures the combined effects of GHGs, short-lived gases, aerosols (including interactions with clouds), volcanoes, and land use and land cover changes. We estimate the transient total ERF over the historical record relative to 1850 from pairs of atmosphere-only simulations. We use the DECK AMIP simulations as reference (amip_An) and perform one additional type of simulation with identical sea surface boundary conditions but with all forcing agents held back at their 1850 values (amip_1850allF_An). This methodology is referred to as ERF_trans in the Forster et al. (2016) nomenclature. Separately, we also estimate aerosol-related ERF (ERFari+aci), which measures total anthropogenic aerosol effects (including aerosol-radiation interactions, aerosol-cloud interactions, and the effect of light-absorbing particles in snow/ice). This requires a third type of simulation in which only aerosols and their precursors are held back at their 1850 values (amip_1850aeroF_An). To reduce year-to-year noise in the ERF estimates, we rely on an ensemble of three members.

Results are shown in Figure 25. Aerosol-related ERF (blue) increases in magnitude from the 1920s to the 1970s as anthropogenic aerosol emissions increase. Improved emissions standards in the 1970s cause aerosol-related ERF to stabilize at a value of −1.65 W/m2 during the last 20 years (1995–2014). This is substantially larger in magnitude than the IPCC AR5 expert judgment best estimate of −0.9 W/m2 (Intergovernmental Panel on Climate Change, 2013. p. 620). It falls within the 5% to 95% uncertainty range of −1.9 to −0.1 W/m2 but outside the likely range of −1.5 to −0.4 W/m2. Strong negative aerosol-related forcing counterbalances warming from GHGs in E3SMv1, with the net effect that total ERF (orange) hovers near zero for the first 100 years (except during volcanic eruptions) and only starts to significantly become positive once aerosol emissions stabilize in the 1970s. Over the last 20 years, the total forcing averages +1.10 W/m2. The IPCC AR5 expert judgment best estimates of the total anthropogenic ERF between 1750 and 2011 is +2.3 W/m2 with an uncertainty range of 1.1 to 3.3 W/m2 (Intergovernmental Panel on Climate Change, 2013, p. 696).

Details are in the caption following the image
Time evolution of annual global mean total effective radiative forcing (ERF; orange) and aerosol-related ERF (light blue) derived from three ensemble members.

We note that the treatment of planetary boundary layer turbulence, shallow convection, and cloud macrophysics has been unified by a single parameterization (CLUBB) in E3SMv1. As a result, the aerosol-related ERF now includes a contribution from interactions between aerosols and shallow cumulus clouds. In contrast, models whose shallow cumulus is unresponsive to aerosols may be artificially setting part of the aerosol-related ERF to 0.

6.2 Sensitivity

The DECK simulations include two idealized CO2 forcing simulations designed to estimate climate sensitivity at different time horizons. The equilibrium climate sensitivity (ECS) is defined as the equilibrium surface temperature change resulting from a doubling in CO2 concentrations (e.g., Intergovernmental Panel on Climate Change, 2007). While it would take thousands of simulated years to run a general circulation model to equilibrium, ECS is typically approximated from much shorter simulations using the approach of Gregory et al. (2004). This approach takes advantage of the fact that while the responses of global mean surface temperature and TOA energy imbalance to abruptly quadrupling CO2 are nonlinear in time, the relationship between these variables is usually linear. As a result, ECS can be extrapolated as the surface temperature change associated with zero TOA energy imbalance. This is typically done using 150-year “abrupt-4xCO2” simulations as demonstrated for E3SMv1 in Figure 26. Because the surface temperature versus TOA energy imbalance slope weakens with time in most models (Andrews et al., 2015; Armour et al., 2013; Ceppi & Gregory, 2017), ECS computed this way is best described as “effective climate sensitivity.” Figure 26 shows that E3SMv1 response to abrupt CO2 quadrupling is relatively linear and produces an ECS of 5.30 K.

Details are in the caption following the image
Global annual mean of surface air temperature change versus top-of-atmosphere net radiation change for 150 years of abrupt-4xCO2 relative to the underlying control simulation. Linear regression is depicted with a dashed line. Its intersection with the horizontal axis is twice the equilibrium climate sensitivity (ECS).

Figure 27 illustrates the time evolution of annual average surface air temperature from the E3SMv1 abrupt4xCO2 simulation (red). Clearly, 150 years is insufficient to achieve equilibrium. This graphic also includes results from another idealized experiment where CO2 concentration increases by 1% per year (1pctCO2; blue). This second simulation is useful for computing a shorter–time scale measure of warming called the transient climate response (TCR). TCR is defined as the change in surface temperature averaged for a 20-year period around the time of CO2 doubling (approximately year 70; Intergovernmental Panel on Climate Change, 2007, p. 629) from a 1pctCO2 simulation. As such, it depends on both climate sensitivity and ocean heat uptake rate. For E3SMv1, TCR is 2.93 K from 1pctCO2.

Details are in the caption following the image
Time evolution of annual global mean air surface temperature anomalies for the idealized CO2 forcing simulations abrupt-4xCO2 (red), 1pctCO2 (blue), and the control simulation (piControl; green). Solid lines are fits obtained with a two-layer energy balance model (discussed in section 6.3). Also depicted are estimates of equilibrium climate sensitivity (ECS) and transient climate response (TCR).

Both TCR and ECS are on the high side of a compilation of published values (Knutti et al., 2017). In particular, IPCC AR5 WG1 estimates that ECS is likely (>66% probability) between 1.5 and 4.5 K, while TCR is likely between 1 and 2.5 K Intergovernmental Panel on Climate Change, 2013 (2013, p. 871). E3SMv1 ECS and TCR are 17% larger than the likely upper bound from IPCC. While large, these values are below the extremely unlikely (<5%) upper bounds of 3 K for TCR and 6 K for ECS.

To better understand E3SMv1's high ECS relative to models that took part in CMIP5, we diagnose 2xCO2 effective radiative forcing (ERF2xCO2) and individual radiative feedbacks from the abrupt-4xCO2 simulation. In this particular case, ERF2xCO2 is derived by linear regression (a methodology referred to as ERF_reg in the Forster et al., 2016, nomenclature). In Figure 28, we show ERFs and radiative feedbacks from 28 CMIP5 models diagnosed in Caldwell et al. (2016), along with those diagnosed following the same procedure in E3SMv1. E3SMv1's ERF, along with its Planck, lapse rate, water vapor, combined lapse rate plus water vapor, and surface albedo feedbacks are all very close to the CMIP5 multimodel mean values. In contrast, E3SMv1's net cloud feedback is larger than in all but two CMIP5 models. Although its positive LW cloud feedback is slightly smaller than the CMIP5 average, its positive SW cloud feedback is larger than all CMIP5 models. Therefore, E3SMv1's high climate sensitivity is solely due to its large positive cloud feedback, which causes its net feedback parameter (which quantifies how strongly the 4xCO2 forcing is radiatively damped) to be less negative than all but two CMIP5 models (“Total” column of Figure 28). A more detailed diagnosis of the reasons for E3SMv1's large positive cloud feedback will be reported in a subsequent paper.

Details are in the caption following the image
Global and annual mean effective radiative forcings (ERFs) and radiative feedbacks derived as the y intercepts and slopes, respectively, of the regression line between top-of-atmosphere radiation anomalies and global surface temperature anomalies from 150-year abrupt-4xCO2 experiments. ERFs are divided by 2 to express them with respect to a doubling of CO2. The total radiative feedback (Total) is broken down into Planck, lapse rate (LR), water vapor (WV), combined lapse rate plus water vapor (LR + WV), surface albedo (Albedo), and net cloud (Cloud) components using radiative kernels of Soden et al. (2008). The cloud feedback is further broken down into its shortwave (CloudSW) and longwave (CloudLW) components. Individual Coupled Model Intercomparison Project Phase 5 (CMIP5) models are shown in gray circles, the CMIP5 multimodel mean (MMM) is shown as a black circle, and E3SMv1 is shown as a red cross. “Total” refers to the net radiative feedback computed directly from TOA fluxes. “Total-Σ' refers to the difference between the directly calculated net feedback and that estimated by summing kernel-derived components. This value is near zero for E3SMv1, indicating that errors in the overall radiative kernel decomposition of feedbacks are small.

6.3 Two-Layer Energy Balance Model

Having established that E3SMv1 is a high-sensitivity model with a strong aerosol forcing, we now explore the degree to which either the sensitivity or the aerosol forcing can explain the mismatch in the warming trajectory between E3SMv1 and observations (section 5.6 and Figure 23).

Stevens (2015) uses the historical record to constrain the aerosol forcing. He argues “that an aerosol forcing less than −1.0 W m−2 is very unlikely [because] a more negative aerosol forcing would imply that none of the roughly 0.3-K rise in Northern Hemisphere surface temperatures during the 100-yr period from 1850 to 1950 could be attributed to anthropogenic forcing, which seems implausible.”

Along similar lines of reasoning, Zhao et al. (2018) caution against the often seen “argument that the twentieth century warming does not strongly constrain either climate sensitivity or the strength of aerosol cooling because similar overall warming can result from relatively low sensitivity to CO2 and weak aerosol cooling, or by high sensitivity and strong aerosol cooling.” In their words, this argument holds “only to a limited extent, because of the likelihood of there having been a peak, or at least a plateau, in aerosol forcing in the 1980–1990s. As a result, in order to create the correct overall warming if climate sensitivity is high, one requires large enough aerosol forcing to cancel much of the warming prior to the 1980s, while after the aerosols peak the high sensitivity and reductions in aerosols combine to produce very rapid warming.”

Held et al. (2010) demonstrated that the time evolution of Geophysical Fluid Dynamics Laboratory CM2.1 global mean warming could be approximated quite realistically using a simple two-layer box model driven by a time evolving net radiative forcing. Similar approaches have also been used to estimate the climate sensitivity from global mean temperature observations (e.g. Aldrin et al., 2012; Padilla et al., 2011). Within the framework of a two-layer energy balance model (EBM), Geoffroy et al. (2013) derived analytical solutions for the evolution of the global surface temperature in response to idealized forcing scenarios abrupt-4xCO2 and 1pctCO2. Furthermore, using 16 AOGCMs from CMIP5, they demonstrated that EBMs calibrated exclusively with abrupt-4xCO2 data could accurately predict the temperature evolution in 1pctCO2 simulations.

The two-layer EBM is defined by the following system of equations:
urn:x-wiley:jame:media:jame20860:jame20860-math-0005(3)
urn:x-wiley:jame:media:jame20860:jame20860-math-0006(4)
Prognostic variables are the temperatures of the upper (T) and deep ocean layers (T0) with C and C0 their respective heat capacities. urn:x-wiley:jame:media:jame20860:jame20860-math-0007 is the total radiative forcing, λ the surface feedback parameter, and γ the heat exchange coefficient between the upper and deep ocean. The general solution for the upper (surface) temperature under a time varying forcing is given by Equation (B8) in Geoffroy et al. (2013):
urn:x-wiley:jame:media:jame20860:jame20860-math-0008(5)

The solution can be interpreted as the sum of a fast and slow convolution of the forcing with exponential decay functions. Table 1 in Geoffroy et al. (2013) lists the relationships between the weights (af, as), time scales (τf, τs), and the parameters characterizing the model (equations 3 and 4).

We calibrate the EBM for E3SMv1 using abrupt-4xCO2 data following the procedure outlined in Geoffroy et al. (2013), except for a small deviation in the second step (p. 1846) when we average over 20 instead of 10 years to estimate the fast time scale as we found that it provides a better fit to abrupt-4xCO2. Table 3 lists the parameter values, and the fits to abrupt-4xCO2 and 1pctCO2 are shown with solid black lines in Figure 27. The fits clearly demonstrate that the EBM calibrated with abrupt-4xCO2 can accurately predict the behavior of 1pctCO2. Furthermore, the TCR from the fit (3.07 K) is within 5% of its true value (2.93 K), indicating that for E3SMv1, both ECS and TCR could be estimated from abrupt-4xCO2 alone.

Table 3. Two-Layer EBM Parameters Calibrated From abrupt-4xCO2 Simulation
Parameter Value Unit
urn:x-wiley:jame:media:jame20860:jame20860-math-0014 6.671 W/m2
λ 0.629 W·m−2·K−1
af 0.558 unitless
τf 7.263 year
as 0.442 unitless
τs 160.093 year

We now explore the EBM's ability to reproduce the predicted E3SMv1 warming over the historical record following the approach of Held et al. (2010). The time varying forcing urn:x-wiley:jame:media:jame20860:jame20860-math-0009 in equation 5 is simply the total ERF from Figure 25 (orange line). The resulting temperature is shown with a thick gray line in Figure 29a. The simple model predictions agree much better with modeled rather than observed values, lying within the envelope of model ensemble values most of the time. Furthermore, the EBM reproduces—albeit in an exaggerated fashion—the E3SMv1 behavior of a lack of warming during the 1960–1990s followed by an excessive warming trend.

Details are in the caption following the image
Time evolution of annual global mean surface air temperature anomalies. (a): Observations (HadCRUT4; black), E3SMv1 ensemble mean and range (red and orange), two-layer energy balance model (gray). (b): EBM with reduced aerosol forcing (0% to 75% in 15% increments; blue with increasing line thicknesses). (c): EBM with reduced sensitivity (0% to 75%; green with increasing line thicknesses). (d): EBM with 50% reduction in both aerosol forcing and sensitivity (brown).
With the credibility of the EBM established, the EBM can also be applied to hypothetical scenarios in order to explore the effect of the forcing and sensitivity. New hypothetical total forcing with weaker aerosol forcing can be constructed by linear combinations of the original forcings:
urn:x-wiley:jame:media:jame20860:jame20860-math-0010(6)
where urn:x-wiley:jame:media:jame20860:jame20860-math-0011 is the orange line and urn:x-wiley:jame:media:jame20860:jame20860-math-0012 the blue line in Figure 25. Figure 29b explores reductions in aerosol forcing up to 75% in 15% increments (αaero from 0.0 to 0.75). A 50% reduction corresponds approximately to the median IPCC AR5 value of −0.9 W/m2.
Similarly, we can keep the forcing at its original value and reduce the sensitivity
urn:x-wiley:jame:media:jame20860:jame20860-math-0013(7)
as illustrated in Figure 29c for a reduction up to 75% (αsensitivity from 1.0 to 0.25) with no changes in the fast and slow time scales (τf, τs) and their corresponding weights (af, as).

Careful visual inspection of Figures 29b and 29c reveals that neither a reduction in aerosol forcing nor a reduction in sensitivity alone is sufficient to improve the match with the historical temperature record. Reducing aerosol forcing alone can improve the match up to the 1980s but not afterward when the aerosol forcing reaches a plateau and the high sensitivity causes an excessive warming trend (confirming the argument of Zhao et al., 2018). Instead, a substantial reduction in both is needed. For example, a 50% reduction in aerosol forcing and sensitivity (Figure 29d; brown) matches observations (black) much better than the original EBM calibrated with E3SMv1 (gray). To the extent that the EBM is a good proxy for the behavior of the full model, we conclude that improving the trajectory of the historical warming of E3SMv1 would require a substantial reduction in the magnitude of both aerosol forcing and sensitivity.

7 Conclusion

In this paper, we have described the new E3SMv1 fully coupled physical model in its standard resolution configuration. This model is designed to serve as a tool to address DOE mission-relevant water cycle questions. We have examined the E3SMv1 simulated climate with a set of experiments from the CMIP6 DECK. Key behaviors and biases are recapitulated below:
  1. Over the course of the long preindustrial control simulation, the coupled system has very little model drift, as evidenced in the net TOA flux, global mean surface air temperature, and seasonal range in sea ice area (Figure 3).
  2. The present-day climate simulated by an ensemble of historical simulations reveals that the atmosphere is credible compared to an ensemble of CMIP5 models (Figure 9) but also subject to biases common to many models: for example,

    1. stratocumulus coverage (Figure 5) and

    2. double Intertropical Convergence Zone (Figure 6).

  3. Positively, the atmosphere has a much improved representation of the MJO, in regard to its strength and propagation characteristics (Figure 22).
  4. The ocean also shows biases consistent with lower resolution ocean models: for example,

    1. Gulf stream separation (evident in Figure 10) and

    2. a shallow MLD bias in the SH (Figure 12).

  5. AMOC is weak (Figure 13), and there is large fresh water bias in the North Atlantic (Figure 11) and accompanying shallow MLD biases (Figure 12). These biases are certainly related and the subject of ongoing research with E3SMv1.
  6. The simulated ENSO variability is realistic. It is closer to observations than CESM1 (Figure 20).
  7. E3SM well simulates the spatial pattern associated with ENSO events and is closer to observations than CESM1 (Figure 21)
  8. Sea ice concentrations are too high in the Labrador Sea (Figure 14), and the seasonal growth of ice is delayed and too rapid, relative to observations.
  9. Streamflow simulated by E3SMv1 is consistent with observations in magnitude and timing; however, the seasonality is too large in a number of regions (Figure 19).
  10. E3SMv1's aerosol-related effective radiative forcing (ERFari+aci = −1.65 W/m2; Figure 25), equilibrium climate sensitivity (ECS = 5.3 K; Figure 26) and transient climate response (TCR = 2.93 K; Figure 27) are larger in magnitude than most CMIP5 models but fall within previously published uncertainty bounds. Predictions of large future warming are due to unusually large positive SW cloud feedback (Figure 28).
  11. The coupled climate in the historical ensemble does not warm as quickly as observations between 1960 and 1990 but warms more rapidly thereafter, with an end result that the E3SMv1 ensemble approaches observations by 2014 (Figure 23).
  12. An analysis with a simple energy balance model reveals that this mismatch is due to the combination of E3SMv1's strong aerosol-related forcing and high climate sensitivity (Figure 29).

The climate simulated by E3SMv1 has biases broadly consistent with other climate class models and also has improvements in certain regimes (e.g., tropical variability). These simulations and analysis help to establish the scientific credibility of this new model and set the stage for future additional analysis of these existing simulations as well as new simulations with E3SMv1 (e.g., high resolution coupled, future projections, and regionally refined).

Acknowledgments

This research was supported as part of the Energy Exascale Earth System Model (E3SM) project, funded by the U.S. Department of Energy, Office of Science, Office of Biological and Environmental Research with partial support from the Climate Model Development and Validation activity funded by the Office of Biological and Environmental Research in the US Department of Energy Office of Science. S. A. K. and M. D. Z. acknowledge support from the Regional and Global Modeling and Analysis Program of the U. S. Department of Energy, Office of Science, Office of Biological and Environmental Research. J. L. M. was supported by DE-SC0012778. J.-H. Yoon was partially supported by the National Research Foundation grant: NRF_2017R1A2b4007480. We thank the following for their assistance adapting the input4MIPS CMIP6 data sets to the E3SM model: Michael Prather and Juno Hsu (Appendix B.3), Beiping Luo (Appendix B.4), and George Hurtt and Ritvik Sahajpal (Appendix B.6). Simulations described in this work and most developmental simulations leading up to them relied on computational resources provided by the National Energy Research Scientific Computing Center (NERSC), a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under contract DE-AC02-05CH11231. The remaining developmental simulations used a high-performance computing cluster provided by the BER Earth System Modeling program and operated by the Laboratory Computing Resource Center at Argonne National Laboratory as well as resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under contract DE-AC05-00OR22725. Work at LLNL was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract DE-AC52-07NA27344. The Pacific Northwest National Laboratory (PNNL) is operated for DOE by Battelle Memorial Institute under contract DE-AC05-76RLO1830. Some material is based upon work supported by the National Center for Atmospheric Research, which is a major facility sponsored by the National Science Foundation under Cooperative Agreement No. 1852977. The E3SM project, code, simulation configurations, model output, and tools to work with the output are described at the website (https://e3sm.org). Instructions on how to get started running E3SM are available at the website (https://e3sm.org/model/running-e3sm/e3sm-quick-start). All model codes may be accessed on the GitHub repository (https://github.com/E3SM-Project/E3SM). Model output data are accessible directly on NERSC or through the DOE Earth System Grid Federation (https://esgf-node.llnl.gov/projects/e3sm).

    Appendix A: Energy Correction Term

    The definition of energy is slightly inconsistent between components of E3SMv1, in particular with respect to the treatment of internal water energy. MPAS-Ocean and sea ice properly account for energy changes due to water temperature changes, but other components currently do not. This inconsistency creates a small spurious energy imbalance between the atmosphere and ocean due to the fact that water evaporates from the ocean surface at a certain temperature and returns to the ocean as precipitation at a different (lower) temperature. Globally averaged, the imbalance is less than 0.5 W/m2.

    Kirchhoff's equation (Glickman, 2000, p. 432) relates the variation with temperature of the latent heat of a phase change to the difference between the specific heats of the two phases. For water, this is written as follows:
    urn:x-wiley:jame:media:jame20860:jame20860-math-0015(A1)
    where Lv is the latent heat of vaporization, cpv is the specific heat at constant pressure of water vapor, and cw is the specific heat of liquid water. Like other Earth system models (I. Held, personal communication, Feb 2017), the E3SMv1 atmosphere component violates Kirchhoff's equation by neglecting variation of Lv with temperature, while at the same time assuming cpv ≠ cw.

    The ocean experiences a net cooling because it supplies heat to bring the liquid precipitation to the temperature of the sea surface, while solid precipitation is first melted (using heat from the ocean) and then brought to the temperature of the sea surface. However, there is no corresponding warming term in the atmosphere. The net impact is an offset between long-term trends in net TOA energy flux and ocean heat content (the largest heat reservoir of the coupled system).

    We correct for that imbalance by adding back the missing warming term in the atmosphere with an ad hoc correction term. To mimic what is done in the ocean, an energy flux term (IEFLX) is introduced:
    urn:x-wiley:jame:media:jame20860:jame20860-math-0016(A2)
    where i denotes the grid column, cpsw the heat capacity of sea water, QFLX the surface moisture flux, PRECT the precipitation flux, and Tsurf the surface (skin) temperature. IEFLX is first calculated for each grid box and then globally averaged and later applied as a uniform adjustment to the sensible heat flux at each grid box (Figure A1):
    urn:x-wiley:jame:media:jame20860:jame20860-math-0017(A3)
    where I denotes the total number of grid columns on the cubed sphere mesh, Ai the grid cell area for column i, and SHFLX the sensible heat flux. The calculated global annual mean IEFLX is about +0.4 W/m2. Sensitivity simulations show that using the IEFLX correction improves energy conservation consistency between net TOA and ocean heat content with minimal impact on the simulated climate.
    Details are in the caption following the image
    Flowchart of the IEFLX calculation in E3SMv1. IEFLX shown in this figure is a global mean quantity. TPHYSBC and TPHYSAC indicate the model physics parameterizations before (BC) and after (AC) the coupler calculation. SHFLX indicates the sensible heat flux.

    Appendix B: Input Data

    The E3SMv1 DECK simulations generally followed the input4MIPS data sets, which are described by Durack et al. (2018). The details of the versions used, and any deviations from the input4MIPS data are specified in the sections below.

    B.1 GHG Concentrations

    GHG concentrations were specified using v1.2.0 of input4MIPS GHG historical concentrations (Meinshausen & Vogel, 2016; Meinshausen et al., 2017) for CO2, CH4, N2O, CFC-12, and CFC-11eq (i.e., CFC-11 plus all other major halocarbon species converted to give an equivalent amount of CFC-11 forcing). The data were specified annually. The model assumed that the concentration applies to 1 January and interpolated linearly during the year to the next year's value. To enable time interpolation until the end of 2014, concentrations for 2015 and 2016 were added by linear extrapolation from 2013 and 2014. The concentrations were assumed to be uniform throughout the atmosphere (i.e., well mixed).

    Ozone concentrations were as specified in sections B.2 and B.3.

    B.2 Tropospheric Aerosols and Related Data Sets

    Aerosol concentrations, sizes, and optical properties were provided by the four-mode Modal Aerosol Module (MAM4; Liu et al., 2016) but modified to include marine organic aerosols (Burrows et al., 2018), with the coarse mode extended to include carbonaceous aerosols (i.e., BC, primary organic matter [POM], marine organic aerosols, and second organic aerosol [SOA]) to treat the resuspension of aerosol particles from evaporated raindrops more appropriately.

    Monthly anthropogenic emissions of aerosols and precursor gases for MAM4 were specified as follows. SO2, sulfate, BC, POM, and SOA gases were obtained from the CMIP6 (Coupled Model Intercomparison Project Phase 6) emission data sets, as described by Hoesly et al. (2017, 2018). Open fire emissions of SO2, BC, POM, and SOA were from the biomass burning data sets (van Marle et al., 2016, 2017) developed for CMIP6. All the CMIP6 emissions had annual data. Biogenic emissions for SOA precursor sources (e.g., isoprene and monoterpenes) were obtained from the standard MOZART emissions as described by Emmons et al. (2010). The vertical distribution of SOA precursor sources was prescribed to mimic the explicit treatment of gas- and particle-phase chemical oxidation of SOA (Shrivastava et al., 2015). The inject height of fire emissions, as well as industrial and power plant emissions, followed the AeroCom (Aerosol Comparisons between Observations and Models) protocols (Dentener et al., 2006).

    The oxidants necessary to calculate secondary aerosol production were read in from a file, which included O3, OH, H2O2, HO2, and NO3. The data were provided for each month for 1 year in each decade (1849, 1855, 1865, …, 2015). The ozone values were derived from the input4MIPS Ozone data set v1.0 (Hegglin et al., 2016). Concentrations for the other species were not provided by input4MIPS, so the preexisting specified-oxidant data used by MAM4 were used, for which the provenance is not entirely known, but was probably from a CAM4 based CAM-CHEM transient simulation for IPCC AR5, with an extension to 2015 by copying 2000 values.

    B.3 Stratospheric Ozone

    We used a prognostic linearized ozone chemistry scheme to calculate stratospheric ozone using a single tracer (Linoz v2, Hsu & Prather, 2009). The linearized chemistry coefficients were calculated using the GHG concentrations from v1.2.0 of the input4MIPS GHG historical concentrations with the assistance of J. Hsu and M. Prather (private communication, Oct-Nov 2017), with a 3-year lag as simple way to account for the time surface concentrations take to mix into the stratosphere. The input data were generated for every month in years spaced 5 years apart (1845, 1850, …, 2015).

    B.4 Stratospheric Aerosols

    We modified E3SMv1 to pass the stratospheric aerosol optical properties (extinction, single scattering albedo, and asymmetry factor) from input4MIPS directly to the radiation routine (after regridding from the input file to the model grid). We used version 3 of the data set, which was created to exactly match the E3SMv1 radiation wavelength ranges by one of the data set creators, B. Luo (private communication, June-Sept 2017).

    B.5 SST and Sea Ice

    For our AMIP simulations, we used v1.1.3 of the PCMDI SST and sea ice fractions (Durack & Taylor, 2017; Taylor et al., 2000), which cover the years 1870 to 2016. E3SM interpolated ocean temperatures and ice fractions between the midmonth values; hence, we used the diddled version of the data set in which the values were tweaked so that the monthly mean values in the model matched the monthly mean values in the original data set. Since the input data set used the Gregorian calendar and E3SM uses a 365-day calendar, the model time interpolated on the Gregorian calendar, which caused 29 Feb data to be skipped and will lead to a small discontinuity at the start of 1 March during leap years.

    B.6 Land Use

    Land use, and land use change, files were regridded to the E3SM ne30 grid by G. Hurtt and R. Sahajpal (private communication, March 2017 - Jan 2018) using v2.1h of the input4MIPS land use data (Hurtt et al., 2017).

    B.7 Nitrogen Deposition

    Nitrogen deposition was not used by E3SM for the DECK simulations.

    B.8 Solar Input

    Solar irradiances came from v3.2 of the solar irradiance data set from input4MIPS (Matthes, Funke, Kruschke, et al., 2017; Matthes, Funke, Andersson, et al., 2017). The data were specified monthly.

    B.9 Orbital Parameters

    Earth's orbital parameters were inadvertently fixed to 1990 values for all DECK simulations.

    Appendix C: Analysis Tools

    The analysis of this paper was largely enabled through a suite of diagnostic software packages developed in tandem with the model. These packages were designed for the E3SM development and analysis community to be usable, extensible, and with shareable results, each with a different scientific focus and goal.

    New and improved netCDF Operators (Zender, 2008, 2018) were developed and customized for E3SM analysis, and verified to work on CESM output. These include a climatology generator and a time series splitter accessed through the new ncclimo operator and a regridder accessed through the new ncremap operator. Each is a parallelized tool suitable for serial or background-parallel mode execution on personal laptops and workstations and background-parallel and MPI-parallel operation on high-performance computing nodes. These tools are embedded or used as preprocessing steps for E3SM_Diags, MPAS-Analysis, and A-PRIME. Their full documentation is at the website (http://nco.sf.net/nco.html).

    E3SM_Diags is a modern, Python-based diagnostics package developed to facilitate evaluating Earth system models. The package includes a set of comprehensive toolkits and updated analysis data sets. This software is designed in a flexible, modular, and object-oriented fashion, enabling users to manipulate different processes in a diagnostics workflow. Numerous configuration options for metrics computation (i.e., regridding options) and visualization (i.e., graphical backend, color map, and contour levels) are customizable. Built-in functions to generate derived variables and to select diagnostics regions are supported and can be easily expanded to accommodate earth system models with output conventions that are CMIP compliant. Modern computer technologies, such as multiprocessing and containerization, are applied in the software development, which enhance the performance and stability of the software. Detailed documentation can be found from the website (https://e3sm.org/resources/tools/diagnostic-tools/e3sm-diagnostics).

    MPAS-Analysis (https://github.com/MPAS-Dev/MPAS-Analysis) is a Python-based tool for performing postprocessed analysis and plotting of output from E3SM's ocean and sea ice components (MPAS-Ocean and MPAS-Seaice, respectively). MPAS-Analysis uses the netCDF Operators to compute climatologies, extract time series, and remap data sets to common reference grids. Comparisons between simulation results and a wide variety of observational data sets are supported on both latitude/longitude and Antarctic stereographic grids. MPAS-Analysis also supports comparisons between E3SM simulations, allowing users to examine the influence of changing meshes, resolution, parameters, model physics, and so on. Parallelism has been introduced into MPAS-Analysis by breaking each analysis task into modular subtasks that can be run in parallel to efficiently produce hundreds of plots. MPAS-Analysis is aware of E3SM namelists, meaning that tasks are automatically disabled for runs where the necessary output was not produced. The end result of running MPAS-Analysis is a website with image galleries of all plots, sorted by component and analysis type, as well as a set of NetCDF files containing the postprocessed data, available for further analysis.

    A-PRIME is a priority metrics package that is designed to provide a quick, broad overview of coupled model behavior (Evans et al., 2018). The target user would execute A-PRIME on model data as a run is progressing to determine whether the model is on track to produce global level expected behavior. It provides a suite of averaged and time series behavior of the most common variables that drive radiation, dynamical, and hydrological balance. When there are sufficient simulation years available, it also provides ENSO metrics. The top-level directory of the software provides a generic script that targets execution on DOE supercomputers, where E3SM simulations are currently executed and/or postprocessed. These scripts point to Python postprocessing and visualization modules for multiple components in the coupled model. The ocean and sea ice modules load portions of the MPAS-Analysis diagnostics as a submodule.