Historical simulations of models participating in the sixth phase of the Coupled Model Intercomparison Project (CMIP6) are evaluated over 10 Australian regions for their performance in simulating extreme temperatures, among which three models with initial-condition large ensembles (LEs) are used to estimate the effects of internal variability. Based on two observational data sets, the Australian Water Availability Project (AWAP) and the Berkeley Earth Surface Temperatures (BEST), we first analyze the models' abilities in simulating the probability distributions of daily maximum and minimum temperature (TX and TN), followed by the spatial patterns and temporal variations of the extreme indices, as defined by the Expert Team on Climate Change Detection and Indices (ETCCDI). Overall, the CMIP6 models are comparable to CMIP5, with modest improvements shown in CMIP6. Compared to CMIP5, the CMIP6 ensemble tends to have narrower interquartile model ranges for some cold extremes, as well as narrower ensemble ranges in temporal trends for most indices. Over southeast, tropical, and southern regions, both CMIP ensembles generally exhibit relatively large deficiencies in simulating temperature extremes. We also confirm that internal variability can affect the trends of the extremes and there is uncertainty in representing the irreducible variability among different LEs in CMIP6. Furthermore, the evaluation based on Perkins' skill score (PSS) and root-mean-square error (RMSE) in the three LEs does not directly correlate with the ranges of the trends for extreme temperatures. The findings of this study are useful in informing and interpreting future projections of temperature-related extremes over Australia.

Key Points

The assessment on the probability distributions of daily maximum and minimum temperature makes the evaluation of extremes more robust
Temperature extremes over Australia are broadly similar in CMIP5 and CMIP6
There are differences in estimating internal variability across multiple CMIP6 models

1 Introduction

Extreme temperatures pose severe threats to human society and the natural environment, such as human health, energy consumption, agriculture, and ecosystems (Intergovernmental Panel on Climate Change [IPCC], 2012). During recent decades, distinct warming trends have been documented (e.g., Donat et al., 2013; Perkins-Kirkpatrick & Lewis, 2020) and attributed to anthropogenic influence (e.g., Diffenbaugh et al., 2017; Fischer & Knutti, 2015; Min et al., 2011), which may further change the severity of these impacts (IPCC, 2013). In Australia, observations also show clear warming trends in extreme temperatures, which are represented by most global climate models (GCMs) relatively well (e.g., Alexander & Arblaster, 2009, 2017). However, the Australian climate is highly variable (e.g., Herold et al., 2018; Westra et al., 2016), making the analysis of extremes over different subregions more complex. The climate over Australia is related to a variety of physical mechanisms and teleconnections to modes of internal climate variability. For example, the frequency of heatwaves over southern and northern parts of Australia can be influenced by the El Niño-Southern Oscillation (ENSO); and for southeastern Australia, there is a positive correlation between the Southern Annular Mode (SAM) and heatwave frequency (Perkins et al., 2015). In this study, we utilize state-of-art climate models to investigate if there are improvements in simulating extreme temperatures and the effects of internal variability over different Australian regions.

To understand how extreme temperatures evolve in the past, present, and future climate, GCMs are the main tools available. The GCMs in the sixth phase of the Coupled Model Intercomparison Project (CMIP6; Eyring et al., 2016), organized by the Working Group on Coupled Modelling (WGCM) of the World Climate Research Programme (WCRP), recently became available and will contribute to the Intergovernmental Panel on Climate Change (IPCC) sixth Assessment Report (AR6). Compared to the previous phase, CMIP5 (Taylor et al., 2012), the models in CMIP6 generally have finer model resolution and improved physical processes (Eyring et al., 2016; Stouffer et al., 2017). However, the improvements in model configuration may not always lead to better simulations. For example, recent studies (e.g., Meehl et al., 2020; Tokarska et al., 2020; Zelinka et al., 2020) have shown that equilibrium climate sensitivity (ECS), a quantity of how global surface temperature changes once equilibrium is reached in response to an instantaneous doubling of CO₂, has a greater range in CMIP6 (1.8°C–5.6°C), which is likely due to the representation of cloud feedbacks and cloud-aerosol interactions.

Extreme temperature can be measured in many ways. The Expert Team on Climate Change Detection and Indices (ETCCDI), organized by the joint World Meteorological Organization (WMO) Commission on Climatology/WCRP project on Climate Variability and Predictability/Joint Technical Commission for Oceanography and Marine Meteorology, defines 16 core indices (Zhang et al., 2011), which are based on daily scale data and usually describe extremes on an annual basis. Compared to other indices or methods that describe temperature extremes, such as extreme value theory (e.g., Coles, 2001; Kharin et al., 2007; Kharin et al., 2013; Perkins et al., 2014; Zwiers et al., 2011) and the frequency of record-breaking high or low monthly temperatures (Meehl et al., 2009), the ETCCDI indices are consistent, widely used and easy to interpret (e.g., Alexander & Arblaster, 2017; Kim et al., 2020; Klein Tank et al., 2009; Sillmann et al., 2013; Zhang et al., 2011). In a global study, using these extreme indices, Sillmann et al. (2013) found that the inter-model spread in CMIP5 decreases for extreme temperatures, compared to CMIP3. As an updated analysis of Sillmann et al. (2013), Kim et al. (2020) concluded that there are limited improvements for CMIP6 models in simulating temperature extremes, both globally and regionally; however, some systematic biases (e.g., the cold bias in cold extremes over high-latitude regions) still exist. In Australia, there are distinct warming trends in CMIP5 models for most locations, but cold extremes are generally overestimated, and warm extremes underestimated (Alexander & Arblaster, 2017). CMIP6 has not been analyzed in terms of the ETCCDI indices over subregions for Australia, nor have the CMIP5 or CMIP6 indices been compared as yet.

Furthermore, to get reliable information on regional climate, it is critical to determine the effects of internal variability and its influence on the evaluation of extreme temperatures over different Australian regions. Internal variability emerges from the chaotic nature of the climate system, contributing to the observed fluctuations in temporal variations of temperature or hydro-meteorological variables (Deser, Knutti, et al., 2012; Deser et al., 2020; Deser, Phillips, et al., 2012; Lehner et al., 2020; Sippel et al., 2020; Xie et al., 2015). Although internal variability can be predicted for limited times (e.g., the predictability of ENSO), it is fundamentally unpredictable and irreducible (e.g., Bengtsson & Hodges, 2019; Deser, Knutti, et al., 2012; Deser et al., 2020; Deser, Phillips, et al., 2012; Lehner et al., 2020; Lorenz, 1963; Shepherd, 2014; Xie et al., 2015). Previous studies on regional climate have shown that the inherently irreducible variability can contribute to climate changes and mask the forced responses over multidecadal timescales (e.g., Dai & Bloecker, 2019; Deser, Knutti, et al., 2012; Deser et al., 2020; Deser, Phillips, et al., 2012; Hawkins & Sutton, 2009; Hegerl et al., 2015; Hu & Deser, 2013; Perkins & Fischer, 2013; Perkins-Kirkpatrick et al., 2017). For example, Perkins and Fischer (2013) concluded that internal variability could result in larger or smaller trends in heatwaves than observed over Australia. Neglecting the effects of the irreducible variability may make the evaluations on regional climate misleading (Dai & Bloecker, 2019).

In addition to internal variability, there are another two major sources of uncertainties, which are sampled in the multimodel ensembles in the CMIP5 and CMIP6. The first is the uncertainty from radiative forcing, including natural external variability (e.g., solar variability and volcanic sulfur emissions) and anthropogenic influences (e.g., greenhouse gas [GHG] emissions and land-use change), which exists because the effective radiative forcing can differ among the models within each CMIP ensemble; the second is the climate response uncertainty (previously termed “model uncertainty”), which occurs as different model structures may exhibit diverse climate responses to the same radiative forcing (e.g., Lehner et al., 2020; McKinnon & Deser, 2018; Stouffer et al., 2017). Consequently, a multimodel ensemble is not ideal to isolate and quantify the effects of internal variability (Lehner et al., 2020).

In contrast, a single-model initial-condition large ensemble (SMILE; hereafter LE) is convenient to evaluate the influences of internal variability, as external forcing and model structure are identical among the members. The spread across the members within an LE, resulting from slightly different initial conditions, is considered to be induced by internal variability; and the multi-member average represents the forced response (e.g., Dai & Bloecker, 2019; Deser, Knutti, et al., 2012; Deser et al., 2020; Deser, Phillips, et al., 2012; Lehner et al., 2020; Maher et al., 2020; Schlunegger et al., 2020). However, recent studies show that different LEs, most of which are built upon the CMIP5-class models, can produce different estimates of internal variability, differing the relative importance of internal variability compared to other sources of uncertainties (e.g., Lehner et al., 2020; Maher et al., 2020; Schlunegger et al., 2020). To date, estimating the influence of internal variability on regional temperature extremes, based upon CMIP6-class models, has not been documented.

Since CMIP6 has not been analyzed in terms of the ETCCDI indices over Australian regions and the effects of internal variability on extreme temperatures need to be taken into consideration, the aim of this study is to compare the performance of CMIP6 models with CMIP5 in simulating temperature extremes over Australian regions, as well as an analysis investigating how internal variability influences the evaluation of extreme temperatures. The study is organized as follows: Section 2 introduces the observed and model data. The methods are summarized in Section 3. Section 4 describes the results and the discussion and conclusions are presented in Section 5.

2 Data

In this study, we use daily maximum and minimum temperatures (TX and TN) in the historical simulations from 31 CMIP6 (Table S1) and 26 CMIP5 models (Table S2). Only one ensemble member (typically the first member) from each model is considered, as using all members would overemphasize some models with a large number of simulations (Seneviratne & Hauser, 2020). We also use three LEs in CMIP6 to analyze the internal variability of the trends for extreme temperatures:

CanESM5-LE: 25 members used, r(1–25)i1p1f1.
CNRM-CM6-1-LE: 30 members used, r(1–30)i1p1f2.
MIROC6-LE: 50 members used, r(1–50)i1p1f1.

As suggested by previous studies (Alexander & Arblaster, 2017; Sillmann et al., 2013; Srivastava et al., 2020), there are large differences between observational data sets. To robustly validate the simulated results produced by the models from CMIP5 and CMIP6, the Australian Water Availability Project (AWAP; Jones et al., 2009) and the Berkeley Earth Surface Temperatures (BEST; Rohde, Muller, Jacobsen, Muller, et al., 2013; Rohde, Muller, Jacobsen, Perlmutter, et al., 2013) are employed here.

AWAP is generated by the Bureau of Meteorology (BOM), the Bureau of Rural Sciences, and the Commonwealth Scientific and Industrial Research Organization, which aims to understand the terrestrial water balance of Australia and the responses of land surface changes to climate variability and change (Jones et al., 2009). The gridded data set includes rainfall, temperature, vapor pressure, solar exposure, and the normalized difference vegetation index (NDVI) at the horizontal resolution of 0.05° × 0.05° (approximately 5 × 5 km²) over the period 1911–present. Although the analyses over data-sparse regions (e.g., central Western Australia) should be taken with caution as the station network is changed over time (e.g., station coming in and out of use) (Alexander & Arblaster, 2017; King et al., 2013), AWAP is a high-quality observed data set over Australia (King et al., 2017), which in this study is the primary reference data set.

As a globally observed data set, BEST is also analyzed in this study, which provides daily high and low temperatures from 1880 to the present (Rohde, Muller, Jacobsen, Muller, et al., 2013; Rohde, Muller, Jacobsen, Perlmutter, et al., 2013). Compared to other global observational data sets (e.g., Global Precipitation Climatology Project [GPCP], Global Historical Climatology Network [GHCN]), the resolution of the Berkeley data is 1° × 1°, which is relatively higher and covers a longer period. Moreover, more records (around 37,000) are incorporated into the data set, compared to 5,000–7,000 records for other global data sets. Although Berkeley Earth claims to address some major concerns (e.g., data selection, data adjustment, poor station quality, and the urban heat island effect) systematically and objectively, there are still some issues. Through an analysis over Canada, Way et al. (2017) found that monthly minimum temperatures in BEST show larger biases and a systematic underestimation of warming, compared to monthly maximum temperatures, which suggests that some inhomogeneities in the raw data are not taken into account by the Berkeley’s algorithm. For this study, it is also an opportunity to check its validity in measuring temperature extremes over Australia.

3 Methods and Data Processing

3.1 Perkins' Skill Score

As a measure of how well each model can capture the probability distributions of weather variables in the observations, Perkins' skill score (PSS; Perkins et al., 2007) is defined as follows:

$urn:x-wiley:23284277:media:eft2847:eft2847-math-0001$

where n is the number of bins used to calculate the probability distribution, $urn:x-wiley:23284277:media:eft2847:eft2847-math-0002$ is the frequency of the observed values, and $urn:x-wiley:23284277:media:eft2847:eft2847-math-0003$ is the frequency of simulated values in a given bin. A score of 0 indicates no overlapping area between the simulated and observed data, and a score of 100% means the two distributions are identical.

In this study, since the definitions of the ETCCDI indices are based on TX and TN, it is necessary to examine the models' ability in simulating the distributions of TX and TN before applying the metrics to conduct research. Consequently, we utilized PSS to assess the overall similarity between the observed and simulated data (e.g., Kumar et al., 2014; Lewis, 2018; Perkins et al., 2007).

3.2 Extreme Temperature Indices

The ETCCDI indices used in this study are outlined in Table S3. The indices defined in Zhang et al. (2011) can be classified into four categories: absolute indices (e.g., hottest day [TXx]), threshold-based indices (e.g., frost days [FD]), percentile indices (e.g., cold days [TX10p]), and duration indices (e.g., cold spell duration index [CSDI]). Since the definitions of growing season length (GSL) and ice days (ID) are not suitable over most of Australia (Alexander & Arblaster, 2017), they are excluded here. Furthermore, compared to previous studies (e.g., Alexander & Arblaster, 2017; Sillmann et al., 2013), the bootstrap resampling procedure proposed by Zhang et al. (2005) is also applied to the calculations of warm spell duration index (WSDI) and CSDI, and the spells crossing year boundaries are taken into consideration.

We use 30-year climatologies to investigate spatial patterns. For temporal variations, the trends for the time series of the ETCCDI indices are estimated by Theil-Sen estimator and Mann-Kendall nonparametric test is used as the significance test (e.g., Alexander & Arblaster, 2009; Dey et al., 2019).

3.3 Model Performance Metric

Following Sillmann et al. (2013), the evaluation of model performance is based on root-mean-square error (RMSE), which is calculated as:

$urn:x-wiley:23284277:media:eft2847:eft2847-math-0004$

where X is the model climatology of an ETCCDI indicator, Y represents the corresponding climatology in the observed data, and the angular brackets denote spatial averaging over a particular domain. It is a quantity that measures the degree of agreement between the observed and simulated data.

3.4 Data Processing

To investigate the extreme temperatures over Australia in more detail, Australia is divided into nine subregions shown in Table S4 and Figure S1, which is based on a study by Perkins et al. (2014) and the BOM (http://www.bom.gov.au/climate/change/about/temp_timeseries.shtml). Ten regions were determined according to climatological and geographical conditions, abbreviated AUS (Australia), NA (Northern Australia), SA (Southern Australia), SEA (South East Australia), MEA (Middle Eastern Australia), TA (Tropical Australia), SWA (South West Australia), SSA (Southern South Australia), CAU (Central Australia), and MWA (Mid-Western Australia). Since there has been an increase in in-situ observations since 1950, the analysis is carried out from 1950, and the base period is from 1961 to 1990, which is commonly used and allows for a standardized quantification.

The observed and simulated data sets of TX and TN are first regridded to 1° × 1° resolution using bilinear interpolation; the calculations of the extreme indices are then performed. It is noted that reversing the order of operation may have significant effects on the resulting gridded values (e.g., Avila et al., 2015; Chen & Knutson, 2008; Herold et al., 2017; Zhang et al., 2011). For example, indices sensitive to resolution choice (e.g., maximum 1-day precipitation amount) are substantially altered when the order of operation is changed (Herold et al., 2017). In addition, following the practice in King et al. (2015), grid boxes containing less than 75% land are masked out. To calculate the trends, we first average each ETCCDI index over the regions; then Theil-Sen estimator and Mann-Kendall nonparametric test are applied to the time series of the extreme indices.

In Section 4, each model is first evaluated for TX and TN using the PSS, which is based on the probability distributions between observations and simulations over the regions for the period 1950–2005. Second, the ETCCDI indices are analyzed in terms of the spatial patterns during the base period, which are evaluated by RMSE. The temporal variations for the extreme indices in CMIP5 (1950–2005) and CMIP6 (1950–2014) are then investigated (the boxplots of the trends for the observed and simulated data from 1950 to 2005 are indicated in Figure S11). Finally, based on the three LEs in CMIP6, the influences of internal variability on the evaluation of extreme temperatures in the single-member multimodel ensembles are estimated.

4 Results

4.1 Probability Distributions and PSS

Figures 1-4 show the probability distributions of TX and TN and their PSSs over the Australian regions during the period 1950–2005 for AWAP, BEST, CMIP6, and CMIP5 models. Bin sizes of 0.5°C were used. For the probability distributions of TX (Figure 1), the two observations are generally comparable over the regions, though there are slight differences between AWAP and BEST over the regions SWA and SSA. In contrast, the probability distributions of TN (Figure 2) in the two observed data sets show larger differences over most regions (except NA). Overall, for TN, BEST tends to have right-shifted distributions (warmer-side tails), with higher peaks over the northern regions and lower peaks over the southern regions, compared to AWAP.

Details are in the caption following the image — **Figure 1**
Open in figure viewer PowerPoint

Probability distributions of daily maximum temperature (TX) during the period 1950–2005 over Australian regions for AWAP (black), BEST (yellow), the multimodel medians in CMIP6 (red), and CMIP5 (blue). Shading denotes the full range across the models in each CMIP ensemble.

For both TX and TN, the multimodel medians in CMIP6 and CMIP5 are generally similar over all regions (Figures 1 and 2). Compared to AWAP, the medians of the two CMIP ensembles in the probability distributions of TX tend to overestimate the lower tails and slightly underestimate the upper tails in Figure 1. For TN (Figure 2), the lower tails are underestimated and the upper tails overestimated. Furthermore, the medians in CMIP6 and CMIP5 are more analogous to AWAP than BEST, and the TX medians fit to observations are much better than for TN. The model spread, as measured by the full range of the multimodel ensemble in each CMIP, tends to be larger in the upper tails and narrower in the lower tails for CMIP6 when compared to CMIP5 (Figure 1). This suggests that more models in CMIP6 tend to show warmer patterns. In particular, for the probability distributions of TX in CMIP6 models, the larger spread in the upper tail is mainly caused by the three models CanESM5, MIROC6, and MRI-ESM2 (not shown).

In Figures 3 and 4, compared to both observations, the multimodel means of PSSs in CMIP6 and CMIP5 models are generally around 90%, which implies that both CMIP ensembles simulate the daily scale extreme temperatures similarly and relatively well. The lower multimodel mean PSSs are found for TX over TA (∼83%) and TN over SEA (∼82%), TA (∼84%), and SSA (∼84%). Also, over most regions shown in Figure 4 (e.g., AUS, NA, and MEA), higher scores for AWAP do not mean the same in BEST, suggesting that the two observed data sets are significantly different and the TN in BEST may be biased. It further implies that AWAP is more appropriate for evaluation of TN and cold extremes over Australian regions and analyzing cold extremes based on TN in BEST in the following parts should be taken cautiously. For the model spreads of PSSs, the full ranges of the probability distributions for TX and TN in CMIP6 are commonly wider than CMIP5 over the regions. This could be due to the fact that several models in CMIP6, such as MIROC6 and NorCPM1, show relatively lower scores. It is also noted that the models with higher resolution (e.g., MRI-ESM2-0) do not generally show higher scores than those with relatively coarse resolution (e.g., FGOALS-g3; Figures 3 and 4). As the change of temperature may be more related to large-scale meteorological patterns (Grotjahn et al., 2016), the relatively lower PSSs in some models with higher resolution may result from the generation of unrealistic local details (e.g., soil moisture) in simulations (Lau & Nath, 2012).

In general, models in CMIP6 and CMIP5 can perform quite differently based on AWAP or BEST, especially for TN; and the multimodel means and spreads of PSSs over most regions in CMIP6 are comparable to that in CMIP5, though the multimodel means are typically slightly lower in CMIP6 for both TX and TN over most regions (Figures 3 and 4). This is because some models in CMIP6, which usually produce lower scores, collectively reduce the ensemble mean. Compared to AWAP (Figure 3), MIROC6, NorCPM1, IPSL-CM6A-LR, and CanESM5 usually have lower scores. Of those, NorCPM1 and IPSL-CM6A-LR have cold shifts while warm shifts occur for MIROC6 and CanESM5 (not shown). In contrast, the PSSs of MIROC-ES2L, MIROC6, MPI-ESM1-2-HR, and NorESM2-LM for the probability distributions of TN are usually lower over Australian regions (Figure 4), which all have warm shifts (not shown). It is interesting to note that the model MIROC-ES2L typically has lower PSSs in Figure 4 but relatively higher scores in Figure 3, implying that MIROC-ES2L tends to simulate higher temperatures over Australia. The results based on PSSs suggest that when using historical simulations from the above models to calculate extremes, the results should be interpreted with caution.

4.2 Spatial Patterns of Climatologies

Examining the extreme temperature indices averaged over the period 1961–1990 helps us to determine the magnitude and spatial distributions of model bias. The 30-year climatologies of TXx, TNn (coldest night), and diurnal temperature range (DTR) for the observations and the medians from CMIP6 and CMIP5 models are shown in Figures 5-7, as well as their differences. The climatological patterns of other indices, including coldest day (TXn), warmest night (TNx), WSDI, CSDI, summer days (SU), tropical nights (TR), and FD, are shown in Figures S2–S8. Except for DTR (Figure 7), AWAP and BEST exhibit similar patterns for other temperature indices (Figures 5, 6, and S2–S8). Overall, compared to AWAP, the magnitude in BEST for most indices is higher over most parts of Australia, although the absolute values of TXx (Figure 5) and FD (Figure S8) in BEST are generally lower. The negligible variations of DTR in BEST (Figure 7b) should be treated with caution, which can result from the biased TN shown in Figures 2 and 4. This may be also related to the minimization process in the Berkeley’s homogenization algorithm, minimizing the mean square of the local weather term and suppresses regional differences to some extent. This issue needs further investigation; however, it is beyond the scope of our study (Rohde, Muller, Jacobsen, Muller, et al., 2013; Rohde, Muller, Jacobsen, Perlmutter, et al., 2013) In Figures 5, 6 and 7 the differences between AWAP and BEST are largest over southwest parts for TXx, north regions for TNn, and central Australia for DTR, which suggests that the evaluation of the models can be quite different compared to different observed data. Compared to the biases of the models, the observation uncertainty can be of the same magnitude (e.g., CSDI) or even larger (e.g., TNx in Figure S3 and TR in Figure S7).

The observed climatological indices are reasonably well represented by the models from CMIP6 and CMIP5. However, similar to CMIP5, systematic errors still exist in the CMIP6 multimodel medians. As shown in Figures 5-7 and S2–S8, the distinct differences are usually located over the eastern part of tropical Australia, southeast and western Australia. For example, for TXx, there are cold biases over southwest Australia and warm biases over southeast Australia (Figure 5h). In general, relative to AWAP, the multimodel medians of CMIP6 appear to show improvements for some indices (e.g., TXx, TXn, CSDI, and SU), compared to CMIP5.

To investigate the regional performance of CMIP6 models, box-and-whisker plots are employed to show the ETCCDI indices over Australian regions (Figure 8). The boxes indicate the interquartile model spreads (range between the 25th and 75th percentiles), the black lines within the boxes are the multimodel medians, the whiskers extend to the edges of 1.5 × interquartile ranges, and “outlier points” that fall outside of the whiskers are denoted by diamonds. Except for DTR, BEST exhibits broadly higher values than AWAP over most regions (Figures 8b–8i). However, for TXx (Figure 8a) and FD (Figure 8j), the magnitudes of indices in BEST are generally lower than AWAP. Moreover, the differences between the observational data sets can be comparable to the interquartile range of the models from CMIP6 and CMIP5 over most regions for many indices (except TXx, TXn, SU, and FD). This implies that based on different observational data, the model evaluation results may differ, which is consistent with previous studies (e.g., Kim et al., 2020; Sillmann et al., 2013; Srivastava et al., 2020). For future studies over Australian regions, we advocate the use of AWAP over BEST, as both TX and TN are more realistic in AWAP.

Compared to AWAP, the multimodel medians of CMIP6 tend to overestimate the duration indices (i.e., WSDI and CSDI) over all Australian regions. Among the absolute and threshold indices, TXx, TXn, DTR, SU, and FD are commonly underestimated by CMIP6 over most regions (except TXx, TXn, and SU over SEA); while the medians in CMIP6 models overestimate TNx, TNn, and TR. There are relatively higher biases between AWAP and the medians in the CMIP6 models over some regions such as SEA, MEA, TA, and CAU, compared to other regions.

For the comparison between CMIP6 and CMIP5 models, the multimodel medians and interquartile model ranges are analyzed and shown to be broadly comparable. The distinct differences for the medians are among the absolute indices. For TXx and TNx, the medians in CMIP6 models are higher than CMIP5. In contrast, for TXn and TNn, CMIP6 shows lower values over most regions (except for TXn over the regions CAU and MWA). The interquartile model ranges in CMIP6 tend to be lower than CMIP5 for TNn, WSDI, and CSDI over most regions, which indicates that the model uncertainty in CMIP6 may be reduced. However, over some regions such as NA, TA, and MEA, the interquartile range tends to be larger for some indices, compared to other regions, suggesting that models simulating the extremes over these regions may have more uncertainty.

4.3 Metric Evaluation

With respect to AWAP, the RMSEs for the CMIP6 and CMIP5 models are used to assess the models' overall performance in simulating extreme temperature indices averaged for the base period 1961–1990 over Australian regions (Figure 9; RMSEs based on BEST is shown in Figure S9). The medians of the RMSEs in the two ensembles commonly have higher values for the indices over tropical and eastern Australia (Figure 9). Similar to previous studies (e.g., Srivastava et al., 2020), the models do not perform consistently well over the regions (not shown). This demonstrates that there is large variability for the performance of the models in simulating different indices over different regions, suggesting that different indices or regions are influenced by different processes. For example, in CMIP6, the model MIROC-ES2L has higher RMSEs across all regions for TNn, while its performance in simulating TXn is relatively better than other models (lower RMSEs). Overall, the RMSEs in CMIP6 indicate that the models need further improvement over the regions MEA, TA, CAU, and MWA. The models HadGEM3-GC31-MM, HadGEM3-GC31-LL, and GFDL-CM4 are commonly among the best performers, while NorCPM1, NorESM2-LM, and MIROC6 tend to show higher RMSEs (not shown). However, among the best performers, the evaluation of the model “NorESM2-LM” needs to be cautiously interpreted, since its performance estimated by PSSs is broadly modest among the CMIP6 models.

Compared to the RMSEs in CMIP5 models, there are some improvements shown in CMIP6. Usually, for some cold extremes (e.g., TNn, warm nights [TN90p], cold nights [TN10p], CSDI, and FD), the interquartile model ranges are commonly narrower in CMIP6. For TXx and SU, the means and medians of RMSEs in CMIP6 are generally lower than CMIP5.

4.4 Temporal Variations

The time series of the anomalies and the actual values for extreme temperature indices averaged over Australia (10°S–45°S, 110°E–155°E) are shown in Figure 10 and Figure S10, respectively. Furthermore, the boxplots representing trends over Australian regions are displayed in Figure 11 and S11, and the number of models that show trends of the ETCCDI indices significant at the 95% confidence level is summarized in Table 1.

Table 1. Percentage (Unit: %) of Models in CMIP6 and CMIP5 That Show Trends of the ETCCDI Indices Significant at the 95% Confidence Level

Region	CMIP Phase	TXx	TXn	TNx	TNn	DTR	TX90p	TX10p	TN90p	TN10p	WSDI	CSDI	SU	TR	FD
AUS	CMIP6	58.1	38.7	83.9	80.6	16.1	77.4	80.6	100.0	100.0	77.4	96.8	74.2	96.8	71.0
AUS	CMIP5	53.8	15.4	61.5	61.5	11.5	65.4	46.2	92.3	96.2	57.7	76.9	46.2	84.6	42.3
NA	CMIP6	61.3	25.8	87.1	67.7	22.6	80.6	71.0	96.8	100.0	80.6	96.8	64.5	100.0	41.9
NA	CMIP5	46.2	7.7	61.5	50.0	7.7	65.4	42.3	92.3	92.3	61.5	73.1	46.2	84.6	30.8
SA	CMIP6	41.9	41.9	58.1	80.6	12.9	64.5	74.2	96.8	100.0	51.6	80.6	54.8	80.6	67.7
SA	CMIP5	34.6	34.6	42.3	57.7	15.4	42.3	38.5	80.8	88.5	30.8	73.1	38.5	69.2	34.6
SEA	CMIP6	12.9	51.6	38.7	58.1	16.1	54.8	80.6	93.5	90.3	32.3	67.7	45.2	67.7	64.5
SEA	CMIP5	19.2	38.5	19.2	38.5	19.2	38.5	57.7	73.1	84.6	15.4	34.6	26.9	42.3	34.6
MEA	CMIP6	25.8	35.5	41.9	71.0	9.7	61.3	48.4	90.3	90.3	51.6	80.6	51.6	80.6	61.3
MEA	CMIP5	30.8	26.9	34.6	38.5	19.2	42.3	42.3	80.8	80.8	42.3	61.5	50.0	73.1	38.5
TA	CMIP6	61.3	22.6	87.1	67.7	19.4	77.4	67.7	96.8	96.8	80.6	93.5	58.1	93.5	3.2
TA	CMIP5	34.6	11.5	61.5	30.8	7.7	65.4	50.0	80.8	84.6	61.5	57.7	38.5	76.9	3.8
SWA	CMIP6	51.6	35.5	41.9	67.7	6.5	74.2	71.0	96.8	100.0	45.2	74.2	41.9	74.2	35.5
SWA	CMIP5	50.0	38.5	38.5	50.0	19.2	42.3	42.3	61.5	84.6	19.2	46.2	23.1	53.8	15.4
SSA	CMIP6	45.2	41.9	29.0	54.8	16.1	54.8	74.2	87.1	90.3	35.5	58.1	41.9	64.5	29.0
SSA	CMIP5	26.9	30.8	26.9	34.6	15.4	38.5	38.5	69.2	80.8	15.4	46.2	26.9	38.5	26.9
CAU	CMIP6	58.1	19.4	74.2	64.5	16.1	67.7	51.6	93.5	90.3	71.0	77.4	54.8	83.9	32.3
CAU	CMIP5	38.5	7.7	61.5	53.8	7.7	53.8	26.9	80.8	84.6	46.2	50.0	38.5	76.9	34.6
MWA	CMIP6	74.2	25.8	87.1	67.7	9.7	77.4	54.8	96.8	100.0	67.7	80.6	58.1	90.3	22.6
MWA	CMIP5	61.5	19.2	53.8	50.0	7.7	57.7	26.9	84.6	80.8	57.7	61.5	38.5	65.4	11.5

Abbreviations: CMIP5/CMIP6, fifth/sixth phase of the Coupled Model Intercomparison Project; ETCCDI, Expert Team on Climate Change Detection and Indices.

As shown in Figure 10, the temporal variations of the two observations for the extremes are quite similar and they are reasonably well captured by both the CMIP ensembles. However, for some indices, differences between AWAP and BEST are substantial. For example, the differences between the two observations for TR (Figure S10m) can be as large as the total inter-model range, further indicating the observational uncertainty can be quite large, likely due to the aforementioned erroneous TN data in BEST. Consistent with Alexander and Arblaster (2017), the temporal variations of TNx, TNn, and TR in AWAP are close to the lower end of the model spread in CMIP6 and CMIP5, while the observed TXn, DTR, and FD tend to be at the upper end (Figure S10). In terms of the model spread, some outliers shown in CMIP5 are corrected in CMIP6 (e.g., the outliers shown in TN10p and CSDI produced by the model GFDL-ESM2G in 1964).

The trends of the temperature indices over each region in the observed and simulated data are displayed in Figure 11, and Tables S5 and S6 additionally show the trends and significance for the indices in the observations. For all the temperature indices, the warming trends of BEST are generally higher than AWAP over most regions, with the lower warming trends in BEST usually located over SSA, CAU, and MWA, which are data-sparse regions. Again, the differences between the observations can be as large as the interquartile model range (e.g., TN10p). Compared to the medians of CMIP5 models, the medians in CMIP6 are commonly closer to AWAP (e.g., TXx, warm days [TX90p], and SU). Moreover, both the spreads and interquartile model ranges tend to be narrower in CMIP6, and there is a larger portion of models in CMIP6 that show the trends significant as compared to CMIP5 (Table 1). This implies that the model uncertainty in CMIP6 may be somewhat reduced, or that both the single-member multimodel ensembles in CMIP5 and CMIP6 do not sample the three kinds of uncertainties reasonably. Next, in Section 4.5, we use three LEs only to investigate the effects of internal variability as they are unlikely to be enough to cover the spread of model uncertainty. For the trends calculated over the period 1950–2005 (Figure S11), the interquartile ranges in CMIP6 are still broadly narrower, while the multimodel medians in CMIP6 indicate lower warming rates than CMIP5. Additionally, the interquartile model ranges in CMIP6 and CMIP5 are usually larger over NA, TA, and MWA for some indices (e.g., TNn, TX90p, TN90p, WSDI, and CSDI), compared to other regions (Figures 11 and S11).

4.5 Internal Variability

The ranges of probability distributions and PSSs for TX and TN in the CanESM5-LE, CNRM-CM6-1-LE, and MIROC6-LE are shown in Figures S12–S15. For TX and TN, there are quite small ranges in the probability distributions and PSSs among the members in each of the three LEs (Figures S12–S15). This is likely due to the large amount of data, making the influences of internal variability indistinct. The ranges of PSSs in the three LEs are generally broader for TX (Figure S14) than TN (Figure S15), in which the effects of internal variability in MIROC6-LE are generally larger than the other LEs (Figure S14). Over NA, MEA, TA, CAU, and MWA, the ranges in PSSs (based on both observations) for MIROC6-LE are slightly more than 5%, relatively larger compared to other regions. For CNRM-CM6-1-LE, the PSSs of TX span larger ranges over NA and TA, while CanESM5-LE over CAU and MWA. Figures S16–S19 exhibit the 30-year climatologies and RMSEs for TXx and TNn in each LE. Since the 30-year averaging also reduces the magnitude of internal variability, the situation is analogous to that shown in Figures S14 and S15. It demonstrates that the evaluation of extreme temperatures over Australia in the single-member multimodel ensembles of CMIP5 and CMIP6, based on PSS and RMSE, is reasonable, which is slightly influenced by the internal variability.

In Figure 12, although the multi-member means of the TXx trends (i.e., the forced response) increase at a similar rate of ∼0.2°C per decade for all the LEs over the regions, the effects of internal variability can alter the regional forced responses, which varies for each region and LE. Generally, the TXx trends in all LEs span larger ranges over SEA, MEA, and SSA; while the ranges of the trends are relatively narrower over TA and SWA. For different LEs, the ranges in MIROC6-LE are much wider, which are larger than CNRM-CM6-1-LE by a factor of ∼3 or more across Australian regions. For example, over SSA, the range of TXx trends in MIROC6-LE is ∼0.7°C per decade while ∼0.2°C per decade in CNRM-CM6-1-LE. It is interesting to note that the ranges in MIROC6-LE can be larger than the inter-model spread in CMIP6 (Figure 11) over certain regions (e.g., SEA and MEA), which indicates that the inter-model spreads in both CMIP ensembles do not fully represent the range of major uncertainties. For TNn (Figure 13), all the LEs show relatively comparable ranges over the regions, although the differences of the forced responses among the LEs are slightly larger than that in TXx.

Consistent with recent studies (e.g., Lehner et al., 2020; Maher et al., 2020; Schlunegger et al., 2020), the estimates of internal variability among CanESM5-LE, CNRM-CM6-1-LE, and MIROC6-LE are different. Additionally, the evaluation based on PSS and RMSE in the three LEs does not directly correlate with the ranges of the trends for TXx and TNn. For example, despite the high RMSEs and low PSSs in MIROC6-LE (relative to AWAP), the TXx trends exhibit much broader ranges while the trends of TNn are more comparable over the regions, compared to other LEs. In Figures S20 and S21, we plot the TXx and TNn trends for the first 25 members in the LEs, which show similar ranges demonstrated in Figures 12 and 13. As suggested by recent studies (e.g., Sippel et al., 2020; Thompson et al., 2015; Xie et al., 2015), it should be the model errors that lead to the differences in the estimates of internal variability among the LEs, which needs further research. Although the model uncertainty in internal variability highlights that the representation of internal variability itself varies considerably across the three LEs, internal variability is still important in contributing to the inter-model spread for the trends in the two CMIPs.

5 Discussion and Conclusions

This study examines the performance of the newly released CMIP6 models in simulating the 30-year climatologies and time series of extreme temperature indices over Australian regions, with an analysis estimating the influences of internal variability on the evaluation of the extremes based on the single-member multimodel ensembles. Using two observational data sets, AWAP and BEST, as the verification, the historical simulations from 31 CMIP6 models are compared with 26 models from CMIP5. Since extreme temperatures are defined based on TX and TN, we also use PSS to evaluate the models' abilities in simulating the probability distributions of TX and TN, for which we expect more robust conclusions to be obtained. Among the CMIP6 models, three LEs are employed to estimate the effects of internal variability.

Similar to previous studies, the observational uncertainty (e.g., the spatial pattern of DTR and the time series of TNn shown between AWAP and BEST) can be substantial. This demonstrates that multiple observations or reanalysis data sets should be employed for the evaluation studies on climate models (e.g., Alexander & Arblaster, 2017; Herold et al., 2017; Kim et al., 2020; Sillmann et al., 2013; Srivastava et al., 2020), which also benefits the identification of the biases and improvements of the data. For example, compared to AWAP, the spatial pattern of DTR shown in BEST is smoother, which can be influenced by the erroneous TN. It may also due to the process that minimizes the square of the local weather term in the algorithm. Moreover, while AWAP and BEST use a comparable amount of Australian stations in their calculations (Jones et al., 2009; Rohde, Muller, Jacobsen, Muller, et al., 2013; Rohde, Muller, Jacobsen, Perlmutter, et al., 2013), the interpolation procedures in BEST are more complex (Rohde, Muller, Jacobsen, Muller, et al., 2013; Rohde, Muller, Jacobsen, Perlmutter, et al., 2013). Thus, due to different underpinning methods, it is not surprising that these observational products yield different ETCCDI values, which highlights why multiple data sets should always be used when evaluating climate models. The study indicates that TN in BEST is biased and not an optimal choice for evaluation over Australia.

Although the performance of CMIP6 and CMIP5 models in simulating extreme temperatures are comparable, there are some improvements in CMIP6. For TXx and SU, the multimodel means and medians of RMSEs in CMIP6 are generally lower. In terms of model ranges in CMIP6, the interquartile model ranges of RMSEs, for some cold extremes (e.g., TNn, TN90p, TN10p, CSDI, and FD), are usually narrower; and there are narrower interquartile model ranges for the temporal trends as well. In addition, with the results from PSS, the RMSEs for some individual models need to be interpreted with caution. For example, as the model MIROC-ES2L is much better at simulating TX than TN, the relatively lower RMSEs of some cold extremes for MIROC-ES2L are doubtful. Moreover, the lower PSSs and the higher RMSEs for the model NorCPM1confirm that its performance in simulating extreme heat is among the worst performers.

This study confirms that there are model differences in estimating internal variability (e.g., Lehner et al., 2020; Maher et al., 2020; Schlunegger et al., 2020) and further shows that the range for the trends may not be related to the performance evaluated by PSS and RMSE (e.g., the ranges of the TNn trends among the three LEs). Although the ideal ensemble sizes for the LEs need to be determined to sufficiently quantify the magnitude of internal variability, it should be model biases that lead to the different estimates, which requires more attention to address the issue. Also, since the thermodynamic response to external forcing is better understood than the dynamics (the factor can primarily regulate the atmospheric circulation; Shepherd, 2014), it may be related to the different relative ranges of the TXx and TNn trends in MIROC6-LE. Another potential way to derive reasonable representations of internal variability is using observationally-based LEs, which are combined with LEs (McKinnon & Deser, 2018; McKinnon et al., 2017). In addition, as suggested by Xie et al. (2015), the process-based evaluation methods should be developed, which may be helpful to better understand internal variability in different models.

Over the regions SEA, TA, and SSA, both CMIP ensembles usually show relatively large deficiencies in simulating temperature extremes. As documented in previous studies, over southeast Australia, the SAM and the Madden Julian Oscillation are two important factors related to extremes (Parker et al., 2014; Perkins et al., 2015); in southern Australia, it is generally assumed that there exists a positive relationship between the Indian Ocean Dipole (IOD) and extreme events (White et al., 2014), and TA can be influenced by the South Pacific convergence zone, tropical cyclones and ENSO (Perkins et al., 2015; Vincent et al., 2011). Compared to other regions, the narrower ranges of the TXx trends over TA indicated by the three LEs, needs further investigation. As suggested by Meehl et al. (2020), the range of model results may not represent the uncertainty, for which the small ranges may be due to the fact that all LEs miss the same processes or feedbacks. With the improvements in understanding physical processes and reducing model biases, the models' performance in simulating extreme temperatures and estimating the uncertainty can be further improved over Australian regions. Despite the different estimates among the LEs and over the regions, internal variability can still have important roles in contributing to the inter-model spread. Even after the model errors are further reduced, estimating the effects of internal variability is essential for adaptation planning on regional scales.

This study provides an assessment of the CMIP6 models' ability in simulating extremes, first analyzing the probability distributions of daily scale weather variables and then calculating the extreme indices. However, it should be recognized that with more CMIP6 models available, the conclusions may be changed to some extent. Also, in the future, remote sensing data may be assimilated into the observations, so that robust conclusions over the data-sparse regions like western Australia can be obtained.

Acknowledgments

The authors would like to thank two anonymous reviewers for their constructive comments, which have improved the quality of this manuscript. The authors thank Lisa V. Alexander for feedback and comments and Zeke Hausfather for discussions about Berkeley Earth surface temperature data sets. This research/project was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government. The authors further acknowledge the World Climate Research Programme’s Working Group on Coupled Modelling, which is responsible for CMIP and coordinated CMIP5 and CMIP6. The authors thank the climate modeling groups for producing and making available their model output, the Earth System Grid Federation (ESGF) for archiving the data and providing access, and the multiple funding agencies who support CMIP and ESGF. The authors also thank the Bureau of Meteorology, the Bureau of Rural Sciences and the Commonwealth Scientific and Industrial Research Organization for providing the Australian Water Availability Project (AWAP) data. Sarah E. Perkins-Kirkpatrick is supported by ARC (Grant number FT170100106) and CLEX (Grant number CE170100023).

Conflict of Interest

The authors declare no financial or other conflicts of interests that could have appeared to influence the work reported in this paper.

Open Research

Data Availability Statement

The AWAP data (Jones et al., 2009) are available from the BOM (http://www.bom.gov.au/metadata/catalogue/19115/ANZCW0503900567#distribution-information). The BEST data set is obtained from http://berkeleyearth.org/data, and the methodological details are provided in the references: Rohde, Muller, Jacobsen, Muller, et al. (2013) and Rohde, Muller, Jacobsen, Perlmutter, et al. (2013). The CMIP6 and CMIP5 outputs can be downloaded from the Earth System Grid Federation (https://esgf-node.llnl.gov/search/cmip6/ and https://esgf-node.llnl.gov/search/cmip5/). Code to reproduce the ETCCDI indices is available at https://doi.org/10.5281/zenodo.4903200.

Supporting Information

References

Alexander, L. V., & Arblaster, J. M. (2009). Assessing trends in observed and modelled climate extremes over Australia in relation to future projections. International Journal of Climatology, 29(3), 417–435. https://doi.org/10.1002/joc.1730
Alexander, L. V., & Arblaster, J. M. (2017). Historical and projected trends in temperature and precipitation extremes in Australia in observations and CMIP5. Weather and Climate Extremes, 15, 34–56. https://doi.org/10.1016/j.wace.2017.02.001
Avila, F. B., Dong, S., Menang, K. P., Rajczak, J., Renom, M., Donat, M. G., & Alexander, L. V. (2015). Systematic investigation of gridding-related scaling effects on annual statistics of daily temperature and precipitation maxima: A case study for south-east Australia. Weather and Climate Extremes, 9, 6–16. https://doi.org/10.1016/j.wace.2015.06.003
Bengtsson, L., & Hodges, K. I. (2019). Can an ensemble climate simulation be used to separate climate change signals from internal unforced variability? Climate Dynamics, 52(5–6), 3553–3573. https://doi.org/10.1007/s00382-018-4343-8
Chen, C. T., & Knutson, T. (2008). On the verification and comparison of extreme rainfall indices from climate models. Journal of Climate, 21(7), 1605–1621. https://doi.org/10.1175/2007jcli1494.1
Coles, S. (2001). An introduction to statistical modeling of extreme values. Springer.
Dai, A., & Bloecker, C. E. (2019). Impacts of internal variability on temperature and precipitation trends in large ensemble simulations by two climate models. Climate Dynamics, 52(1–2), 289–306. https://doi.org/10.1007/s00382-018-4132-4
Deser, C., Knutti, R., Solomon, S., & Phillips, A. S. (2012). Communication of the role of natural variability in future North American climate. Nature Climate Change, 2(11), 775–779. https://doi.org/10.1038/nclimate1562
Deser, C., Lehner, F., Rodgers, K. B., Ault, T., Delworth, T. L., DiNezio, P. N., et al. (2020). Insights from Earth system model initial-condition large ensembles and future prospects. Nature Climate Change, 10(4), 277–286. https://doi.org/10.1038/s41558-020-0731-2
Deser, C., Phillips, A., Bourdette, V., & Teng, H. Y. (2012). Uncertainty in climate change projections: The role of internal variability. Climate Dynamics, 38(3–4), 527–546. https://doi.org/10.1007/s00382-010-0977-x
Dey, R., Lewis, S. C., & Abram, N. J. (2019). Investigating observed northwest Australian rainfall trends in Coupled Model Intercomparison Project phase 5 detection and attribution experiments. International Journal of Climatology, 39(1), 112–127. https://doi.org/10.1002/joc.5788
Diffenbaugh, N. S., Singh, D., Mankin, J. S., Horton, D. E., Swain, D. L., Touma, D., et al. (2017). Quantifying the influence of global warming on unprecedented extreme climate events. Proceedings of the National Academy of Sciences of the United States of America, 114(19), 4881–4886. https://doi.org/10.1073/pnas.1618082114
Donat, M. G., Alexander, L. V., Yang, H., Durre, I., Vose, R., Dunn, R. J. H., et al. (2013). Updated analyses of temperature and precipitation extreme indices since the beginning of the twentieth century: The HadEX2 dataset. Journal of Geophysical Research: Atmospheres, 118(5), 2098–2118. https://doi.org/10.1002/jgrd.50150
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., & Taylor, K. E. (2016). Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geoscientific Model Development, 9(5), 1937–1958. https://doi.org/10.5194/gmd-9-1937-2016
Fischer, E. M., & Knutti, R. (2015). Anthropogenic contribution to global occurrence of heavy-precipitation and high-temperature extremes. Nature Climate Change, 5(6), 560–564. https://doi.org/10.1038/nclimate2617
Grotjahn, R., Black, R., Leung, R., Wehner, M. F., Barlow, M., Bosilovich, M., et al. (2016). North American extreme temperature events and related large scale meteorological patterns: A review of statistical methods, dynamics, modeling, and trends. Climate Dynamics, 46(3–4), 1151–1184. https://doi.org/10.1007/s00382-015-2638-6
Hawkins, E., & Sutton, R. (2009). The potential to narrow uncertainty in regional climate predictions. Bulletin of the American Meteorological Society, 90(8), 1095–1108. https://doi.org/10.1175/2009bams2607.1
Hegerl, G. C., Black, E., Allan, R. P., Ingram, W. J., Polson, D., Trenberth, K. E., et al. (2015). Challenges in quantifying changes in the global water cycle. Bulletin of the American Meteorological Society, 96(7), 1097–1115. https://doi.org/10.1175/bams-d-13-00212.1
Herold, N., Behrangi, A., & Alexander, L. V. (2017). Large uncertainties in observed daily precipitation extremes over land. Journal of Geophysical Research: Atmospheres, 122(2), 668–681. https://doi.org/10.1002/2016jd025842
Herold, N., Ekström, M., Kala, J., Goldie, J., & Evans, J. P. (2018). Australian climate extremes in the 21st century according to a regional climate model ensemble: Implications for health and agriculture. Weather and Climate Extremes, 20, 54–68. https://doi.org/10.1016/j.wace.2018.01.001
Hu, A., & Deser, C. (2013). Uncertainty in future regional sea level rise due to internal climate variability. Geophysical Research Letters, 40(11), 2768–2772. https://doi.org/10.1002/grl.50531
Intergovernmental Panel on Climate Change. (2012). In C. B. Field, V. Barros, T. F. Stocker, D. Qin, D. J. Dokken, K. L. Ebi, et al. (Eds.), Managing the risks of extreme events and disasters to advance climate change adaptation. A special report of Working Groups I and II of the Intergovernmental Panel on Climate Change. Cambridge University Press.
Intergovernmental Panel on Climate Change. (2013). In T. F. Stocker, D. Qin, G.-K. Plattner, M. Tignor, S. K. Allen, J. Boschung, et al. (Eds.), Climate change 2013: The physical science basis. Contribution of Working Group I to the fifth assessment report of the Intergovernmental Panel on Climate Change. Cambridge University Press.
Jones, D. A., Wang, W., & Fawcett, R. (2009). High-quality spatial climate data-sets for Australia. Australian Meteorological and Oceanographic Journal, 58(4), 233–248. https://doi.org/10.22499/2.5804.003
Kharin, V. V., Zwiers, F. W., Zhang, X., & Hegerl, G. C. (2007). Changes in temperature and precipitation extremes in the IPCC ensemble of global coupled model simulations. Journal of Climate, 20(8), 1419–1444. https://doi.org/10.1175/jcli4066.1
Kharin, V. V., Zwiers, F. W., Zhang, X., & Wehner, M. (2013). Changes in temperature and precipitation extremes in the CMIP5 ensemble. Climatic Change, 119(2), 345–357. https://doi.org/10.1007/s10584-013-0705-8
Kim, Y. H., Min, S. K., Zhang, X., Sillmann, J., & Sandstad, M. (2020). Evaluation of the CMIP6 multi-model ensemble for climate extreme indices. Weather and Climate Extremes, 29, 15. https://doi.org/10.1016/j.wace.2020.100269
King, A. D., Alexander, L. V., & Donat, M. G. (2013). The efficacy of using gridded data to examine extreme rainfall characteristics: A case study for Australia. International Journal of Climatology, 33(10), 2376–2387. https://doi.org/10.1002/joc.3588
King, A. D., Karoly, D. J., & Henley, B. J. (2017). Australian climate extremes at 1.5 °C and 2 °C of global warming. Nature Climate Change, 7(6), 412–416. https://doi.org/10.1038/nclimate3296
King, A. D., van Oldenborgh, G. J., Karoly, D. J., Lewis, S. C., & Cullen, H. (2015). Attribution of the record high Central England temperature of 2014 to anthropogenic influences. Environmental Research Letters, 10(5), 054002. https://doi.org/10.1088/1748-9326/10/5/054002
Klein Tank, A. M. G., Zwiers, F. W., & Zhang, X. (2009). Guidelines on analysis of extremes in a changing climate in support of informed decisions for adaptation. World Meteorological Organization.
Kumar, D., Kodra, E., & Ganguly, A. R. (2014). Regional and seasonal intercomparison of CMIP3 and CMIP5 climate model ensembles for temperature and precipitation. Climate Dynamics, 43(9–10), 2491–2518. https://doi.org/10.1007/s00382-014-2070-3
Lau, N. C., & Nath, M. J. (2012). A model study of heat waves over North America: Meteorological aspects and projections for the twenty-first century. Journal of Climate, 25(14), 4761–4784. https://doi.org/10.1175/jcli-d-11-00575.1
Lehner, F., Deser, C., Maher, N., Marotzke, J., Fischer, E. M., Brunner, L., et al. (2020). Partitioning climate projection uncertainty with multiple large ensembles and CMIP5/6. Earth System Dynamics, 11(2), 491–508. https://doi.org/10.5194/esd-11-491-2020
Lewis, S. C. (2018). Assessing the stationarity of Australian precipitation extremes in forced and unforced CMIP5 simulations. Journal of Climate, 31(1), 131–145. https://doi.org/10.1175/jcli-d-17-0393.1
Lorenz, E. N. (1963). Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20(2), 130–141. https://doi.org/10.1175/1520-0469(1963)020<0130:Dnf>2.0.Co;2
Maher, N., Lehner, F., & Marotzke, J. (2020). Quantifying the role of internal variability in the temperature we expect to observe in the coming decades. Environmental Research Letters, 15(5), 054014. https://doi.org/10.1088/1748-9326/ab7d02
McKinnon, K. A., & Deser, C. (2018). Internal variability and regional climate trends in an observational large ensemble. Journal of Climate, 31(17), 6783–6802. https://doi.org/10.1175/jcli-d-17-0901.1
McKinnon, K. A., Poppick, A., Dunn-Sigouin, E., & Deser, C. (2017). An “observational large ensemble” to compare observed and modeled temperature trend uncertainty due to internal variability. Journal of Climate, 30(19), 7585–7598. https://doi.org/10.1175/jcli-d-16-0905.1
Meehl, G. A., Senior, C. A., Eyring, V., Flato, G., Lamarque, J. F., Stouffer, R. J., et al. (2020). Context for interpreting equilibrium climate sensitivity and transient climate response from the CMIP6 Earth system models. Science Advances, 6(26), eaba1981. https://doi.org/10.1126/sciadv.aba1981
Meehl, G. A., Tebaldi, C., Walton, G., Easterling, D., & McDaniel, L. (2009). Relative increase of record high maximum temperatures compared to record low minimum temperatures in the U.S. Geophysical Research Letters, 36(23), L23701. https://doi.org/10.1029/2009gl040736
Min, S. K., Zhang, X., Zwiers, F. W., & Hegerl, G. C. (2011). Human contribution to more-intense precipitation extremes. Nature, 470(7334), 378–381. https://doi.org/10.1038/nature09763
Parker, T. J., Berry, G. J., Reeder, M. J., & Nicholls, N. (2014). Modes of climate variability and heat waves in Victoria, southeastern Australia. Geophysical Research Letters, 41(19), 6926–6934. https://doi.org/10.1002/2014gl061736
Perkins, S. E., Argüeso, D., & White, C. J. (2015). Relationships between climate variability, soil moisture, and Australian heatwaves. Journal of Geophysical Research: Atmospheres, 120(16), 8144–8164. https://doi.org/10.1002/2015jd023592
Perkins, S. E., & Fischer, E. M. (2013). The usefulness of different realizations for the model evaluation of regional trends in heat waves. Geophysical Research Letters, 40(21), 5793–5797. https://doi.org/10.1002/2013gl057833
Perkins, S. E., Moise, A., Whetton, P., & Katzfey, J. (2014). Regional changes of climate extremes over Australia – A comparison of regional dynamical downscaling and global climate model simulations. International Journal of Climatology, 34(12), 3456–3478. https://doi.org/10.1002/joc.3927
Perkins, S. E., Pitman, A. J., Holbrook, N. J., & McAneney, J. (2007). Evaluation of the AR4 climate models' simulated daily maximum temperature, minimum temperature, and precipitation over Australia using probability density functions. Journal of Climate, 20(17), 4356–4376. https://doi.org/10.1175/jcli4253.1
Perkins-Kirkpatrick, S. E., Fischer, E. M., Angélil, O., & Gibson, P. B. (2017). The influence of internal climate variability on heatwave frequency trends. Environmental Research Letters, 12(4), 044005. https://doi.org/10.1088/1748-9326/aa63fe
Perkins-Kirkpatrick, S. E., & Lewis, S. C. (2020). Increasing trends in regional heatwaves. Nature Communications, 11(1), 1–8. https://doi.org/10.1038/s41467-020-16970-7
Rohde, R., Muller, R., Jacobsen, R., Muller, E., Perlmutter, S., Rosenfeld, A., et al. (2013). A new estimate of the average Earth surface land temperature spanning 1753 to 2011. Geoinformatics and Geostatistics: An Overview, 1(1). https://doi.org/10.4172/2327-4581.1000101
Rohde, R., Muller, R., Jacobsen, R., Perlmutter, S., Rosenfeld, A., Wurtele, J., et al. (2013). Berkeley Earth temperature averaging process. Geoinformatics and Geostatistics: An Overview, 1(2). https://doi.org/10.4172/2327-4581.1000103
Schlunegger, S., Rodgers, K. B., Sarmiento, J. L., Ilyina, T., Dunne, J. P., Takano, Y., et al. (2020). Time of emergence and large ensemble intercomparison for ocean biogeochemical trends. Global Biogeochemical Cycles, 34(8), e2019GB006453. https://doi.org/10.1029/2019GB006453
Seneviratne, S. I., & Hauser, M. (2020). Regional climate sensitivity of climate extremes in CMIP6 versus CMIP5 multi-model ensembles. Earth's Future, 8(9), e2019EF001474. https://doi.org/10.1029/2019EF001474
Shepherd, T. G. (2014). Atmospheric circulation as a source of uncertainty in climate change projections. Nature Geoscience, 7(10), 703–708. https://doi.org/10.1038/ngeo2253
Sillmann, J., Kharin, V. V., Zhang, X., Zwiers, F. W., & Bronaugh, D. (2013). Climate extremes indices in the CMIP5 multimodel ensemble: Part 1. Model evaluation in the present climate. Journal of Geophysical Research: Atmospheres, 118(4), 1716–1733. https://doi.org/10.1002/jgrd.50203
Sippel, S., Meinshausen, N., Fischer, E. M., Székely, E., & Knutti, R. (2020). Climate change now detectable from any single day of weather at global scale. Nature Climate Change, 10(1), 35–41. https://doi.org/10.1038/s41558-019-0666-7
Srivastava, A., Grotjahn, R., & Ullrich, P. A. (2020). Evaluation of historical CMIP6 model simulations of extreme precipitation over contiguous US regions. Weather and Climate Extremes, 29, 100268. https://doi.org/10.1016/j.wace.2020.100268
Stouffer, R. J., Eyring, V., Meehl, G. A., Bony, S., Senior, C., Stevens, B., & Taylor, K. E. (2017). CMIP5 scientific gaps and recommendations for CMIP6. Bulletin of the American Meteorological Society, 98(1), 95–105. https://doi.org/10.1175/bams-d-15-00013.1
Taylor, K. E., Stouffer, R. J., & Meehl, G. A. (2012). An overview of CMIP5 and the experiment design. Bulletin of the American Meteorological Society, 93(4), 485–498. https://doi.org/10.1175/bams-d-11-00094.1
Thompson, D. W. J., Barnes, E. A., Deser, C., Foust, W. E., & Phillips, A. S. (2015). Quantifying the role of internal climate variability in future climate trends. Journal of Climate, 28(16), 6443–6456. https://doi.org/10.1175/jcli-d-14-00830.1
Tokarska, K. B., Stolpe, M. B., Sippel, S., Fischer, E. M., Smith, C. J., Lehner, F., & Knutti, R. (2020). Past warming trend constrains future warming in CMIP6 models. Science Advances, 6(12), eaaz9549. https://doi.org/10.1126/sciadv.aaz9549
Vincent, E. M., Lengaigne, M., Menkes, C. E., Jourdain, N. C., Marchesiello, P., & Madec, G. (2011). Interannual variability of the South Pacific Convergence Zone and implications for tropical cyclone genesis. Climate Dynamics, 36(9–10), 1881–1896. https://doi.org/10.1007/s00382-009-0716-3
Way, R. G., Oliva, F., & Viau, A. E. (2017). Underestimated warming of northern Canada in the Berkeley Earth temperature product. International Journal of Climatology, 37(4), 1746–1757. https://doi.org/10.1002/joc.4808
Westra, S., White, C. J., & Kiem, A. S. (2016). Introduction to the special issue: Historical and projected climatic changes to Australian natural hazards. Climatic Change, 139(1), 1–19. https://doi.org/10.1007/s10584-016-1826-7
White, C. J., Hudson, D., & Alves, O. (2014). ENSO, the IOD and the intraseasonal prediction of heat extremes across Australia using POAMA-2. Climate Dynamics, 43(7–8), 1791–1810. https://doi.org/10.1007/s00382-013-2007-2
Xie, S. P., Deser, C., Vecchi, G. A., Collins, M., Delworth, T. L., Hall, A., et al. (2015). Towards predictive understanding of regional climate change. Nature Climate Change, 5(10), 921–930. https://doi.org/10.1038/nclimate2689
Zelinka, M. D., Myers, T. A., McCoy, D. T., Po-Chedley, S., Caldwell, P. M., Ceppi, P., et al. (2020). Causes of higher climate sensitivity in CMIP6 models. Geophysical Research Letters, 47(1), e2019GL085782. https://doi.org/10.1029/2019gl085782
Zhang, X., Alexander, L., Hegerl, G. C., Jones, P., Tank, A. K., Peterson, T. C., et al. (2011). Indices for monitoring changes in extremes based on daily temperature and precipitation data. Wiley Interdisciplinary Reviews: Climate Change, 2(6), 851–870. https://doi.org/10.1002/wcc.147
Zhang, X., Hegerl, G., Zwiers, F. W., & Kenyon, J. (2005). Avoiding inhomogeneity in percentile-based indices of temperature extremes. Journal of Climate, 18(11), 1641–1651. https://doi.org/10.1175/jcli3366.1
Zwiers, F. W., Zhang, X., & Feng, Y. (2011). Anthropogenic influence on long return period daily temperature extremes at regional scales. Journal of Climate, 24(3), 881–892. https://doi.org/10.1175/2010jcli3908.1

Citing Literature

Volume9, Issue7

July 2021

e2020EF001902

This article also appears in:

CMIP6: Trends, Interactions, and Impacts

Evaluation of Extreme Temperatures Over Australia in the Historical Simulations of CMIP5 and CMIP6 Models

Abstract

Key Points

1 Introduction

2 Data