Volume 122, Issue 6 p. 5910-5930
Research Article
Free Access

A new observational solar irradiance composite

Margit Haberreiter

Corresponding Author

Margit Haberreiter

Physikalisch-Meteorologisches Observatorium and World Radiation Center, Davos Dorf, Switzerland

Correspondence to: M. Haberreiter,

[email protected]

Search for more papers by this author
Micha Schöll

Micha Schöll

Physikalisch-Meteorologisches Observatorium and World Radiation Center, Davos Dorf, Switzerland

LPC2E/CNRS and Université d'Orléans, Orléans, France

Search for more papers by this author
Thierry Dudok de Wit

Thierry Dudok de Wit

LPC2E/CNRS and Université d'Orléans, Orléans, France

Search for more papers by this author
Matthieu Kretzschmar

Matthieu Kretzschmar

LPC2E/CNRS and Université d'Orléans, Orléans, France

Search for more papers by this author
Stergios Misios

Stergios Misios

LAP, Aristotle University of Thessaloniki, Thessaloniki, Greece

Atmospheric, Oceanic and Planetary Physics, Oxford University, Oxford, UK

Search for more papers by this author
Kleareti Tourpali

Kleareti Tourpali

LAP, Aristotle University of Thessaloniki, Thessaloniki, Greece

Search for more papers by this author
Werner Schmutz

Werner Schmutz

Physikalisch-Meteorologisches Observatorium and World Radiation Center, Davos Dorf, Switzerland

Search for more papers by this author
First published: 24 April 2017
Citations: 47

Abstract

Variations in the solar spectral irradiance (SSI) are an important driver of the chemistry, temperature, and dynamics of the Earth's atmosphere and ultimately the Earth's climate. To investigate the detailed response of the Earth's atmosphere to SSI variations, a reliable SSI data set is needed. We present an observational SSI composite data set that is based on 20 instruments and has been built by using probabilistic approach that takes into account the scale-dependent uncertainty of each available SSI observation. We compare the variability of this new composite with available SSI reconstructions and discuss the respective modeled responses in the Earth's atmosphere. As the composite is based on purely statistical means, we consider it as a valuable independent data set.

Key Points

  • An SSI composite is presented based on a probabilistic approach that accounts for the scale-dependent uncertainty of each SSI data set
  • The present composite is solely based on original (unadjusted) observations
  • We have introduced a novel framework for merging different SSI observations into one single homogeneous composite

Plain Language Summary

The changing solar spectrum incident at Earth has important impact on the composition and dynamics of the Earth's atmosphere as well as the climate. In order to quantify the processes at play, it is crucial to have a reliable solar spectral irradiance data set, including the uncertainties, at hand. We present a new composite data set that includes observations from 20 different space instruments and is based on a probabilistic approach. This new composite is compared with available reconstructions, and its response in the Earth's atmosphere is discussed. We consider this new composite as a valuable independent data set.

1 Introduction

The impact of solar variability on the Earth's climate has a long history as a research field and has also become a hotly debated topic [Haigh, 1994; Haigh et al., 2010; Ermolli et al., 2013; Ball et al., 2016]. This impact is mediated by several mechanisms including direct solar heating of the Earth's surface, changes in stratospheric ozone, and the impact of energetic particles [Haigh, 2007; Gray et al., 2010; Ermolli et al., 2013; Solanki et al., 2013].

Although there is strong evidence for this natural forcing to be weak in comparison to that of man-made greenhouse gases, large uncertainties remain regarding the magnitude of the impacts and the mechanisms involved [Stocker and Qin, 2014]. Two key mechanisms have been identified and investigated in a number of modeling studies: The first one, known as the bottom-up mechanism, involves direct changes in the troposphere, caused by the variability primarily in the visible part of the solar spectrum, directly penetrating to the lower troposphere, warming the surface, and modulating the atmosphere-ocean interactions [e.g., Meehl et al., 2009]. The second one, named top-down mechanism, involves the direct influence of solar ultraviolet (UV) variations—in particular below 200 nm—on the upper atmosphere, causing an increase in stratospheric ozone and related warming, indirect dynamical effects at lower stratospheric levels, and finally influencing surface climate through stratosphere-troposphere coupling [e.g., Haigh, 1994; Matthes et al., 2006].

For the top-down mechanisms the key input variable is the solar UV radiation, on which this paper will concentrate. Direct observations of the spectrally resolved solar irradiance (or SSI, for solar spectral irradiance) started in the 1970s. These observations, however, are fragmented in time and in wavelength. Only in 2003—with the launch of the SORCE spacecraft—did the continuous monitoring of the solar spectrum really start. As a consequence, our present knowledge of the solar variability in different parts of the spectrum is limited. However, a continuous SSI data set is crucially needed to assess decadal and multidecadal timescales that are most relevant for climate studies. This paucity of observations is further complicated by numerous problems; making radiometrically accurate SSI observations is notoriously challenging, as instruments degrade in time, have a limited lifetime, and rarely allow in-flight calibration.

To overcome these problems, and reconstruct the SSI over longer periods, several SSI models have been developed and their output, the reconstructed SSI time series, has been widely used to represent solar spectral variations in climate and chemistry-climate modeling studies [Matthes et al., 2006; Austin et al., 2008; Ermolli et al., 2013; Solanki et al., 2013; Hood et al., 2015; Mitchell et al., 2015; Misios et al., 2015].

However, and while most of them correctly reproduce recent SSI observations, large uncertainties remain regarding their ability to reconstruct past variations on multidecadal timescales. More importantly, all these models are eventually trained and tested on the few available SSI observations. This situation has led to the international collaborative project SOLID (First European Comprehensive Solar Irradiance Data Exploitation, http://projects.pmodwrc.ch/solid/) with the aim of collecting and merging all available SSI observations into one single composite that will finally offer better conditions for training SSI models, understanding solar variability, and assessing its impact on climate. Such homogeneous data sets are of particular importance for initiatives such as the Climate Model Intercomparison Project (CMIP, http://www-pcmdi.llnl.gov/projects/cmip/), which aims at quantifying the contributions of various forcings, including the solar forcing.

Although the SOLID composite addresses the full solar spectrum, including the visible and near-infrared, in this paper we focus on the UV only, from 100 nm to 400 nm because of its key role in Sun-climate studies. Indeed, the visible and near-infrared bands mostly lead to direct heating of the surface and oceans only, and their forcing can be conveniently summarized in the spectrally integrated SSI, named total solar irradiance (TSI). The TSI has received considerable attention, and its reconstruction efforts have a long history [Fröhlich and Lean, 1998; Kopp and Lean, 2011]. In comparison, the SSI has received much less attention, while its variability also raises many questions. Recent observations by the SORCE satellite, for example, have revealed SSI variations in the UV band that depart from past observations and could potentially lead to a different climate response to the solar cycle [Haigh et al., 2010]. These variations are still hotly debated [Lean and DeLand, 2012] and provide yet another motivation for building a single composite that would allow different periods to be compared. Recently, Ball et al. [2016] investigated the response of the Earth's atmosphere to different SSI data sets and concluded that the simulated ozone response driven by the SORCE SSI observations cannot be reconciled with ozone observations.

The first attempt to merge different SSI observations in a single UV data set was made by DeLand and Cebula [2008] who created a data set with daily spectra covering the 120–400 nm wavelength range from 1978 to 2005. In the presence of multiple simultaneous observations, one single instrument was selected. We go further by including all available data sets and covering the period from 8 November 1978 to 31 December 2014. In addition, we provide a systematic framework for merging overlapping observations, filling in periods of missing data with solar proxies, taking into account uncertainties, and finally providing uncertainties for the composite, too. The first stage, which consists in collecting all the data sets and preprocessing them to enable their merging, has already been described by Schöll et al. [2016], henceforth referred to as Paper I.

In the current paper, we concentrate on the making of the SSI composite, including a time-dependent uncertainty of the data sets, and the properties of that composite. Two major assets of the mathematical procedure we propose are its traceability and the decoupling of the statistical problem (merging SSI data sets) from the scientific one (correcting SSI data sets). Our method can be readily extended to other spectral bands, or to combinations of observations. At this stage we merge the data sets without attempting to correct them. While the help of additional physical constraints and SSI models should lead to better results, we consider the making of a purely observational composite as an important and mandatory first step toward more elaborate composites.

The paper is structured as follows. First, in section 2, we briefly discuss the data sets used for the observational UV composite presented here. Then, in section 4, we present in detail the tools to derive the composite. Third, in section 12, we present an interim version of the SSI composite, labeled SOLID.beta, and compare it with available SSI reconstruction models. In section 13, we discuss remaining issues, which are solved in section 14, where we present the final version of the observational composite. Then, in section 15, the response of atmospheric heating to the SOLID composite is evaluated and compared against other SSI products. Finally, in section 18, we summarize our findings and list the specific actions that remain to be taken before the composite can be truly considered as being representative of the solar UV variation.

2 SSI Observations

Our composite is based on 20 instruments, which are listed in Table 1. In addition, we also took into consideration 9 reference spectra, of which—due to the unknown uncertainty of the other data sets —we only used one, i.e., the ATLAS3 spectrum, and 6 solar proxy data sets, as listed in Table 2. In Paper I, we describe how they are all converted into a unified format, together with independent uncertainty estimates. Several instruments were not included in our composite. ENVISAT/SCIAMACHY, for example, was excluded at this stage because it is not degradation corrected. The broadband filter radiometers SOHO/VIRGO-SPM, PROBA2/LYRA, and PICARD/PREMOS were also omitted because of their coarse spectral resolution. However, these instruments are valuable for validating the SSI composite, as discussed in section 13.

Table 1. List of the Instruments Used for Making the Composite and the Version of the Data Set, if Available, as Used in the SOLID Compositea
Wavelength Range Observation Period
Instrument Version (nm) (mm/yyyy) Reference
GOES13/EUVS n.a. 11.7–123.2 07/2006 to 10/2014 Evans et al. [2010]
GOES14/EUVS n.a. 11.7–123.2 07/2009 to 11/2012 Eparvier et al. [2009]
GOES15/EUVS n.a. 11.7–123.2 04/2010 to 10/2014 Eparvier et al. [2009]
ISS/SolACES n.a. 16.5–57.5 01/2011 to 03/2014 Schmidtke et al. [2014]
NIMBUS7/SBUV n.a. 170.0–399.0 11/1978 to 10/1986 DeLand and Cebula [2001]
NOAA9/SBUV2 n.a. 170.0–399.0 03/1985 to 05/1997 DeLand et al. [2004]
NOAA11/SBUV2 n.a. 170.0–399.0 12/1988 to 10/1994 Cebula et al. [1998]
SDO/EVE 5 5.8–106.2 04/2010 to 10/2014 Woods et al. [2012]
SME/UV n.a. 115.5–302.5 10/1981 to 04/1989 Rottman et al. [1982]
SNOE/SXP n.a. 4.5 03/1998 to 09/2000 Bailey et al. [2000]
SOHO/CDS 3.1 31.4–62.0 04/1998 to 06/2010 Harrison et al. [1995]
SOHO/SEM n.a. 25.0–30.0 01/1996 to 06/2014 Wieman et al. [2014]
SORCE/SIM 21 240.0–2412.3 04/2003 to 05/2015 Rottman et al. [2005]
SORCE/SOLSTICE 13 115.0–309.0 04/2003 to 05/2015 Rottman [2005]
SORCE/XPS 10 0.5–39.5 04/2003 to 05/2015 Woods and Rottman [2005]
TIMED/SEE-EGS 11 27.1–189.8 02/2002 to 02/2013 Woods et al. [1994]
TIMED/SEE-XPS 11 1.0–9.0 01/2002 to 11/2014 Woods et al. [1999]
UARS/SOLSTICE 11 119.5–419.5 10/1991 to 09/2001 Rottman et al. [1993]
UARS/SUSIM 22 115.5–410.5 10/1991 to 08/2005 Brueckner et al. [1993]
  • a See Paper I for more details. The observation period corresponds to the data as they are used in the SOLID composite. The datasets, for which no version was available, are labelled with “n.a.”.
Table 2. Proxies Used in Addition to the Original SSI Data in Order to Fill in Data Gaps
Origin Relevant
Name of proxy (Observatory) for
30.0 cm radio flux Nobeyama/Toyokawa UV
15.0 cm radio flux Nobeyama/Toyokawa UV
10.7 cm radio flux Penticton/Ottawa EUV/UV
8.0 cm radio flux Nobeyama/Toyokawa EUV/UV
3.2 cm radio flux Nobeyama/Toyokawa EUV/UV
Sunspot darkening Greenwich/SOON network VIS

Let us stress that none of the original SSI data sets and proxy data sets has been corrected or adjusted as our prime objective is to build a composite that is as objective as possible, before considering what changes will be needed to enhance its value for climate studies. For the same reason, we deliberately excluded inputs coming from model reconstructions of the SSI. While this questionable choice inevitably leads to discrepancies between some of the instruments, it is also an asset, in the sense that it provides us with a fully independent reference data set for testing models and checking individual instruments. The time-dependent uncertainties are furthermore evaluated by independent means, thus ensuring that all the instruments can be meaningfully compared. This is important for what follows because these uncertainties determine the contribution of each instrument to the composite.

In addition to SSI observations, we consider six solar proxies. While they do not strictly correspond to SSI observations, they are known to reproduce SSI variations in given bands and for specific timescales. The 10.7 cm radio flux (or F10.7 index), for example, has been widely used as a proxy for the EUV band, whereas the 30 cm flux is better suited for reproducing the solar rotational variability at longer UV wavelengths [Dudok de Wit et al., 2014]. The sunspot darkening is a proxy that has been successfully used to reproduce the variability in TSI [Fröhlich and Lean, 2004].

This list does not include the MgII index, which is a widely used proxy for solar variability in the UV [Viereck et al., 2001]. Indeed, while we concentrate here on the space age only (from 7 November 1978 onward), the full SOLID data set actually starts in 1950, thus requiring proxies that cover the full period. Most radio fluxes are available for that period, whereas measurements of the MgII index started only in November 1978. This is the reason that we did not include the Mg II index in the SOLID composite. Incidentally, because the MgII index is highly correlated with radio fluxes at 15 and 30 cm, most of its variability can be adequately reproduced with a combination of radio fluxes at these two wavelengths [Dudok de Wit et al., 2014].

Resorting to proxies is also a questionable choice, but it is the only solution for filling in the numerous data gaps in the composite, while relying on observations that have been shown to be good substitutes for the SSI. Of course, one should interpret such proxy reconstructions with great care when considering timescales in excess of several years, for which they may not properly capture the true solar variability.

2.1 TSI Data Set

TSI observations are formally part of the SSI. However, they have received considerably more attention so far, and expectations are high regarding a potential long-term variability, whose assessment requires extra care. For that reason, we processed the TSI independently and built its composite without resorting to proxy data or TSI models. We also used slightly different approach, which is detailed in a companion paper [Dudok de Wit et al., 2017].

3 Methodology

The approach we advocate for this is a probabilistic one, in which, at each time step, we consider a weighted average of the different observations. These weights are set by the uncertainty of the observations, for which we thus need a metric that can be consistently applied to all the observations.

An important and novel feature of our approach is its multiscale character. Simultaneous observations may agree on one particular timescale and diverge on another one. Among the most debated examples is the difference between the SORCE/SOLSTICE and SORCE/SIM instruments at 240 nm [Lean and DeLand, 2012]: both show the same solar rotational variability but disagree on the presence of a long-term trend. In such a case, direct averaging of the two will mix instrumental artifacts with the solar signal, rather than keep the best of each record. Likewise, it is important to take into account the disparity in the quality of the different data sets by assigning to each of them a time-dependent (and also timescale-dependent) uncertainty. Another motivation for working scale by scale is the occurrence of artificial jumps in the composite whenever different data sets with data gaps are directly averaged.

As we shall show below, the uncertainty of the data sets is a crucial ingredient of our composite because it allows us to weight cotemporal observations at the same wavelength. Ideally, such uncertainties should be provided by the instrument teams. In practice, they are often absent or lack a crisp definition and therefore cannot be meaningfully compared. Since we need to rely on the same definition for all data sets, we decided to use instead an independent estimation of the uncertainty (see section 4) and use it to assign weights.

Our uncertainty is inherently scale dependent; its high-frequency part (called precision) is the easiest to estimate. We also require a long-term uncertainty and call it stability (section 5). The multiscale approach allows mixing precision and stability at each scale in a natural way. The heuristic idea is as follows: at a specific timescale, every sample incorporates information over a time interval that is proportional to its scale. As such, the uncertainty at this scale includes both the precision over this time interval as well as the stability over the same time interval weighted by the timescale. As a consequence, the contribution of the precision to the uncertainty decreases with larger timescales, while the contribution of the stability increases in proportion to the scale.

The creation of the composite involves a multiscale decomposition of the records, followed by a weighted average, and finally the recombination of the different scales. In practice, this procedure requires five steps, which we detail below.

The first direct spectral observations available are from the NIMBUS7/SBUV instrument, which started to provide data on 8 November 1978. Therefore, all data sets are extended back in time to that date and cover the time up to 31 December 2014.

3.1 Precision

In this work we use the term precision to quantify in each SSI data set the high-frequency noise component that is routinely associated with random instrumental fluctuations. The associated timescales are 3–4 days and below. As described in Paper I, we determine the precision by using an adapted version of the wavelet noise estimator by Donoho and Johnstone [1995]. Let us note, however, that this estimator cannot properly quantify low frequency noise (i.e., on timescales of several days and beyond) and therefore is not suited for observations that are contaminated, for example, by red noise. For that purpose, we consider a second and different estimator, i.e., the long-term uncertainty estimate as discussed in the following.

3.2 Long-Term Uncertainty

Beyond the uncertainty caused by the short-term noise, referred to as precision here, an additional uncertainty of the measured irradiance comes from the evolution of the instrument's response in time. The instrument's response can change because of the instrument's (detector or filter) degradation, also called aging, but also when changes in the operations of the spacecraft or instrument affect the measurement. When these effects are identified and/or understood, they can be corrected. However, it is not rare that SSI data sets have residual instrumental effects that remain in the signal as part of the real SSI time evolution. The uncertainty on the SSI data set caused by these effects can be referred to as long-term precision or stability. We use here the term of stability.

It is out of reach to identify and correct a posteriori the instrumental effects affecting the time evolution of the observed SSI. Here we estimate the uncertainty on SSI time series (i.e., the stability) by comparing the latter to a two-timescale proxy model and identifying patterns that cannot be reproduced by that very admissible model. More precisely, for each instrumental record I(λ,t) we determine the high-frequency and low-frequency coefficients, urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0001 and urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0002, respectively, of the following model
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0003(1)
where the summation is over different solar proxies Pi and the superscripts LF and HF represent the low- and high-frequency components, respectively. These components are determined using a running average of 108 days, i.e., four solar rotations, a duration that is similar to the lifetime of active regions on the Sun. Using the more traditional 81 days to distinguish short and long timescale does not affect the results, and we found that, qualitatively, 108 days allows to better isolate the long-term variations that can include instrumental trends.

It is known that using a two-timescale reconstruction model generally improves the fitting of observed time series [Woods et al., 2000]. To build this admissible proxy model, we use six proxies: the Daily Sunspot Area (DSA) [Wilson and Hathaway, 2006], the Mg II index [Skupin et al., 2005], and the radio fluxes at 3.2 cm, 10.7 cm, 15 cm, and 30 cm [Dudok de Wit et al., 2014]. Clearly, the larger the number of simultaneous proxies, the larger the number of degrees of freedom in the model, and the more the model might reproduce nonsolar variations. One can therefore decide to be more or less allowable in asking the true SSI to be reproducible by proxies.

Once the observed time series is fitted with this proxy model over the whole record, we estimate its stability empirically by
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0004(2)
where the slope is computed over a chosen time interval τ. The stability is therefore time dependent, and it is given in Wm−2 yr−1 (or, alternatively, in % yr−1). This stability represents the ability of a proxy model, adjusted at each time series, to reproduce the observed slope computed over a dedicated time interval τ, which thus also sets the timescale at which the stability is considered. Here we estimate it for a timescale of τ = 3 years in order to focus on trends.

3.3 Temporal Extension of SSI Data Sets

The multiscale decomposition can only be performed on regularly sampled time series with no missing data. Following the preprocessing stage that is described in Paper I, our input data are SSI time series with daily values, with one record for each 1 nm wavelength bin.

Prior to decomposing the data, we extend each data set over the interval going from 8 November 1978 to 31 December 2014 in order to make all records of equal length. This preparatory step has not been covered in Paper I. We fill in the data gaps by expectation-maximization [Dudok de Wit, 2011]. This technique exploits the correlation between specific bands of the SSI and the above mentioned solar proxies to replace missing values by a linear combination of the latter, after prior decomposition into two different timescales with a cutoff at 81 days. Two separate timescales are needed to properly handle the possibility for the same solar features, such as sunspots, to cause darkening (in the visible) and brightening (in the UV). The 81 day cutoff is commonly used to separate fast variations from slow ones [Woods et al., 2000].

After extending the SSI data sets, we end up with one large table that has 13,204 time steps and as many columns as there are individual wavelengths multiplied by the number of instruments, i.e., typically 107 records. Let us stress that this temporal extension is required only to decompose the data into different timescales and as such is a technical necessity; the interpolated or extrapolated values enter the composite with a lowered weight, as they are assigned a higher uncertainty.

3.4 Multiscale Decomposition of the SSI Records

The natural tool for decomposing time series into different timescales is the wavelet transform, which decomposes each time series into J + 1 different records of identical length. Their characteristic scales a = 2j are numbered by their level j = 0,1,2,…,J. The largest level J is the largest integer such that aJ=2Jd, where d is the duration (in days) of the record.

This wavelet decomposition can be performed in different ways. We need an undecimated (or redundant) transform, so that a precise time stamp can be assigned to each wavelet coefficient. The transform should also be orthogonal in scale, so that operations performed on the wavelet transform at one particular scale do not affect neighboring scales. The solution we advocate here, which is widely used in astrophysics, is based on the one-dimensional à trous wavelet transform [Starck et al., 2010].

We decompose the time series dλ(t) at wavelength λ and time t into multiple orthogonal time series urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0005 by convolving it with a Gaussian kernel urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0006 of width (scale) a = 2j, as given in equation 3:
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0007(3)
For the lowest level a = 1 we just take urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0008 as the identity function on t, so that
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0009(4)
Boundaries are handled by mirroring the data on each side. The wavelet transform at level j now reads
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0010(5)
One advantage of this method is the easy recovery of the original time series dλ simply by adding up its wavelet transforms dj,λ. Indeed,
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0011(6)
At each level j, both precision and stability contribute to the combined uncertainty. While the precision decreases with larger scales due to the noise averaging out over a larger time period, the uncertainty associated with the stability increases toward larger scales. What would be really needed here is an estimate of the uncertainty versus timescale. For the TSI Dudok de Wit et al. [2017] find the power spectral density of the uncertainty to be proportional to the inverse of the frequency (with frequency =1/a), as with so-called 1/f noise, or shot noise. This scaling may not apply to the SSI, whose instruments have different noise characteristics. For that reason, we rely here on the precision and an empirical stability estimate to obtain the uncertainty at all scales.
We estimate the uncertainty ελ,j(t) at level j by using a weighted spectrum of the squared precision urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0012, together with a weighted and scaled sum of the stability sλ(t), where the weights are the Gaussian kernel for both precision and stability and the stability is scaled proportional to the kernel width a.
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0013(7)
As an example of the decomposition of the SSI data records Figure 1 shows the SSI time series and their respective uncertainties for Levels 0, 1, 2 (top left), Levels 3, 4, 5 (top right), Levels 6, 7, 8 (bottom left), and Levels 9, 10, 11 (bottom right) of the extended SORCE/SOLSTICE time series (the result after applying the steps as described in section 6), at 220 nm.
Details are in the caption following the image
Illustration of the multiscale decomposition of the extended SORCE/SOLSTICE data set at 220.5 nm for (top left) Levels 0, 1, and 2; (top right) Levels 3, 4, and 5; (bottom left) Levels 6, 7, and 8; and (bottom right) Levels 9, 10, and 11. Level 12 with a scale of a12=4096 days is not shown as it represents a quasi-constant offset.

In the following we keep the uncertainties for each timescale constant in time, although the method can easily handle time-varying uncertainties. In particular, we can incorporate information from the instrument teams and, for example, increase the uncertainty during times when the spacecraft has off-pointing problems. Such adjustments can also be made at specific timescales if the observations are known to be affected by band-limited noise. In future version of the data set, this will be considered, for example, for observations by ENVISAT/SCIAMACHY. This instrument adequately reproduces solar variability at short timescales (days to months), but its long-term variability suffers from several interruptions that occurred during its lifetime. When several cotemporal observations with differing variations are averaged, it is eventually their uncertainty which determines how each of them will be weighted. Therefore, it is of great importance to properly estimate such uncertainties by injecting physical information in them. In doing so, we decouple the scientific problem (What prior information goes into the composite?) from the statistical one (What is the best way of making the composite?). Clearly, the first problem is open-ended, and here we focus on the second one.

3.5 Averaging the Wavelet Transformed Data

Once we have performed the multiscale decomposition, dj,λ, of the SSI for each level j (and instrument I, not indicated here), we determine the composite by maximizing its likelihood, which is equivalent to making a weighted average of the values of dj,λ; the weights are given by the inverse variance of each record. This is done scale-wise. The uncertainty of the composite can be obtained by classical error propagation. Thanks to the orthogonality of the wavelet transform, uncertainties at different scales can be processed separately and added as if they were independent.

The equations to derive the scale-wise components, urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0014, of the composite for each level j and its respective uncertainties, urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0015, again for each level j, are
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0016(8)
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0017(9)
where
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0018(10)
are the weights for level j and instrument i for the wavelength of interest, and εi,j is the corresponding uncertainty at level j. As described in Paper I, we assume purely uncorrelated noise. Therefore, for interpolated and proxy-extended data, after flagging them, we increase the uncertainty to the 1σ standard variation of the actual variation of the SSI data of that data set scaled with the urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0019, with N being the number of missing days determined from the onset and end of the data gap. This results in a substantially lower weight of the interpolated and extrapolated data values in the composite.

3.6 Combining the Reconstructed Scales

To obtain the final composite, the components of the composite are summed up scalewise (i.e., over all levels), as given in equation 6. The uncertainty, εcomp, of the final composite is obtained by error propagation
urn:x-wiley:jgra:media:jgra53489:jgra53489-math-0020(11)

3.7 Absolute Scale of Composite

The above procedure results in a time series that combines all available instruments while taking into account the uncertainty of the individual data sets, with both precision and stability. However, the absolute scale also needs to be set. To do so, we recalibrate the composite to the ATLAS 3 reference spectrum by Thuillier et al. [2004] and, at the same time, constrain it to the TSI composite by Dudok de Wit et al. [2017] for that particular day when the reference spectrum was measured. We note that this recalibration is in line with the nominal TSI value recommended by the IAU 2016 Resolution B3 [Prša et al., 2016] based on the latest TSI absolute measurements by Kopp and Lean [2011] and Schmutz et al. [2013].

First, we rescale the ATLAS 3 reference spectrum to make its integrated spectrum match the value of the TSI composite for the day of the ATLAS 3 observation. For this, we first use a black-body model to the unobserved infrared tail of the spectrum and successively scale the integrated spectrum to the TSI composite. Second, we use this modified ATLAS 3 spectrum to rescale the value of the SOLID composite such that it matches the reference spectrum for its date of the observation.

4 Preliminary Observational Composite

In the following, we present the first preliminary observational SSI composite for different wavelengths and daily resolution along with the UV data sets that went into the production of the composite. For practical reasons we set the spectral resolution to 1 nm, but at the longer wavelength ranges (starting at 623 nm) we have a coarser resolution of 2 nm and higher. Figure 2 shows the observational composite and the available observations it is made of for 121.5 nm (top) and 200.5 nm (bottom). As expected, the composite closely corresponds to a weighted average of the individual records. Note that most records agree well on their short-term variability and less so on the long term, hence the importance of merging them scalewise. Thanks to this multiscale approach, and to the use of solar proxies for filling in gaps, the stability of the composite is better than what would have been obtained by simply daisy-chaining the records. Most observations and the composite agree better at short wavelengths, where the larger relative variability results in a more favorable signal-to-noise ratio. There are some exceptions, however. At 121.5 nm, for example, the solar cycle variability of SME exceeds that of the other instruments, which suggests the need for a correction, as already reported by Woods et al. [2000]. One side effect of the use of proxies is their small but significant contribution to the composite. This is evident, for example, in the variation of the Lyman α line: the large excursion observed in mid-1992 is smaller in the composite than it is in the original observations from UARS/SUSIM. One could easily reduce this contribution from proxies and even set it to 0 if there are cotemporal observations available, as we did for making the TSI composite [Dudok de Wit et al., 2017]. However, for spectral bands that suffer from fragmented observations or from a low signal-to-noise ratio (especially above 300 nm), proxies help constrain the composite.

Details are in the caption following the image
SOLID composite along with the individual irradiance data sets that are used to produce it, shown at (top) 121.5 nm and (bottom) 200.5 nm.

Figure 3 compares the observational composite with various existing reconstructions and the SORCE Solar Spectral Irradiance data set for the following spectral bins: 120–122 nm, 122–200 nm, 200–242 nm, 242–310 nm, 300–350 nm, 350–400 nm, 400–500 nm, 500–600 nm, 600–800 nm, and 800–2000 nm. The reconstructions we consider are from the semiempirical SATIRE-S model [Yeo et al., 2014], the empirical NRLSSI1 model [Lean, 2000] and its more recent version NRLSSI2 [Coddington et al., 2016], and direct observations from SORCE/SOLSTICE and SORCE/SIM as part of the SORCE Solar Irradiance data set. In this data set, SIM observations are used from 310 nm onward.

Details are in the caption following the image
Illustration of the observational SOLID composite (black line), model reconstructions from NRLSSI1 (green line) and SATIRE-S (blue line), and since 2003, the observations from the SORCE mission (red line), with SIM being used from 310 nm onward. These data are integrated over the following spectral intervals: (first row) 120–122 nm (left) and 122–200 nm (right); (second row) 200–242 nm (left) and 242–310 nm (right); (third row) 310–350 nm (left) and 350–400 nm (right); (fourth row) 400–500 nm (left) and 500–600 nm (right); and (fifth row) 600–800 nm (left) and 800–2000 nm (right). The numbers in the legend given after the name of each data set indicate the value for the absolute scaling of each of the data sets to the SOLID composite. The gray-shaded area illustrates the cumulated uncertainty of the SOLID composite starting from 1 January 1996.

The first message coming from this figure is the good agreement between the composite and the three models for the spectral ranges from 120 to 310 nm (Figure 3, first and second rows), for none of which it significantly deviates from, given its confidence intervals. This highlights the difficulty in assessing differences between these models when they are as poorly constrained by the observations. This also illustrates the need for a sound assessment of the confidence intervals. Clearly, future improvements of the composite should not depend on the individual data sets alone but also on refinements that will be made in the precision and stability estimates.

The second message is the unusual solar cycle variability of the composite after 2003, when SORCE/SIM data start being used. As long as there are alternatives to these observations (i.e., for 200–400 nm, and at lower wavelengths), the composite remains in good agreement with the models. At longer wavelengths, when SORCE/SIM is the only observing instrument, the discrepancy with the models and with former solar cycles becomes striking, which is in line with previous studies by Cahalan et al. [2010] and Ermolli et al. [2013]. As the observations with the SIM instrument are the only available data set to cover the wavelengths longward of 410 nm, we consider that particular data set further in sections 13 and 14.

Figure 3 also illustrates the precision and stability of the composite as derived in Paper I and section 5 by showing how it increases in time when moving away from a date that has been arbitrarily chosen to be in the center of the interval. The figure thereby reveals how weak the solar cycle variability becomes, as compared to the uncertainty, when moving from the UV to the visible. Note that this representation may actually overestimate the true long-term stability, which does not increase that fast in time. Indeed, for the TSI, Dudok de Wit et al. [2017] find that the stability does not increase linearly with time and so these commonly used fan-shaped structures may need to be revised in future work.

A different representation of the composite is given in Figure 4, which shows the annual mean of the SSI variation relative to the TSI variation for the spectral intervals 120–122 nm, 122–200 nm, 200–242 nm, 242–310 nm, 310–400 nm, 400–500 nm, 500–600 nm, 600–800 nm, and 800–2000 nm. We note that the variability of the high-resolution spectrum shows much more detail [see Fontenla et al., 2015, Figure 14; Ermolli et al., 2013, Figure 6]. To better illustrate the variation of the SSI, Figure 4 shows the change in SSI relative to the change in TSI for the declining phase of solar cycle 23 (annual means from 2003 to 2008, Figure 4, top) and solar cycle 22 (annual means from 1989 to 1996, Figure 4, bottom). For better visibility, the 120–122 nm and 122–200 nm spectral bins are scaled with a factor of 100 and 10, respectively.

Details are in the caption following the image
(top) Change of the annual mean of SSI in different spectral bins from 2003 to 2008 with respect to the variation in TSI for the same time interval. Shown are the relative changes for SOLID.beta (black), SATIRE-S (blue), NRLSSI1 and NRLSSI2 (dark and light green), and the SORCE composite (red) with SIM being used from 310 nm onward. For better illustration, the first and second spectral bins, i.e., for 121–122 nm and 130–200 nm, are multiplied by a factor of 100 and 10, respectively, additionally shown in partly transparent color. (bottom) Same as Figure 4 (top) but for the annual means of 1989 and 1996, for which no SORCE data are available.

For both time intervals, the variation in the Lyman α line (i.e., the 120–122 nm band) is lowest for the SOLID composite and highest for SATIRE-S, with NRLSSI2 and NRLSSI1 being in between. The reason for this is the absence of rescaling of the SME data, whose values are approximately 1.5 times smaller than its expected level [Woods et al., 2000]. The absence of correction explains the lower solar cycle variability of the SOLID composite as compared to the Lyman α composite by Woods et al. [2000]. We stress again that we took the approach to not change the original observations, with all the consequences this might have.

For the 122–200 nm and 200–242 nm wavelength ranges the variation of all data sets is very close, with a slightly higher variation for the SORCE data set. Moreover, for the wavelength range 242–310 nm and both time intervals the SOLID composite gives the highest variation, while SATIRE-S, NRLSSI2, and NRLSSI1 give a lower variation.

Then, for the 310–350 nm and 350–400 nm wavelength ranges and for the 2003–2008 time interval the SOLID composite again gives the highest variation, and the variation is successively lower for SATIRE-S, NRLSSI2, and NRLSSI1. In this case, for both spectral ranges, the SORCE variation gives a considerably larger variation than all other data sets. For the 1989–1996 time interval the variation of the SOLID composite is between the SATIRE-S and NRLSSI2.

And finally, for the 2003–2008 time interval, and the 400–500 and 500–600 nm range, the SOLID and SORCE composite give a negative solar cycle variation, while SATIRE-S, NRLSSI2, and NRLSSI1 show a variation in phase with the solar cycle. The reason is that for this time frame and wavelength range the only data set available is SORCE/SIM. Therefore, the SOLID composite follows the overall variation as observed by the SORCE/SIM instrument. Whether the unusual variation reported by SIM is real or not is still an open question.

5 Validation of SSI Composite

Let us now concentrate on recent observations when SORCE was operating. For that period, the SOLID composite compares well with the observations made by the PREcision Monitoring Sensor [Schmutz et al., 2009] on board the PICARD mission [Thuillier et al., 2006]. Figure 5 shows the SOLID composite convolved with the broadband filters of the PREMOS instrument and the PREMOS observations [Cessateur et al., 2016] centered at 210 nm (left) and 215 nm (right). The agreement between both time series for the two wavelength channels is excellent. In particular, the temporal evolution of the rotational variability is in very good agreement. Some deviation after, however, can be seen in the 210 nm channel after May 2012 and in the 215 nm channel from early 2013 onward. At 210 nm Cessateur et al. [2016] find a similar deviation when comparing the PREMOS data with SORCE/SOLSTICE (see their Figure 13). We emphasize here that for the comparison shown in Figure 5 both data sets are in absolute values, and both the PREMOS channels and the SOLID composite are not detrended; i.e., they include the long-term trend. Due to the limited spectral coverage of the PREMOS data, this can only be considered as a limited validation.

Details are in the caption following the image
Comparison of the SOLID composite (red line) with the PREMOS observations (black line) for the (left) 210 nm channel and (right) 215 nm channel. The absolute scale has been set to the SOLID data set.

Clearly, an issue with the composite is the variability in the 400–500 nm and 500–600 nm spectral bins. The only observations available for these spectral bins are from the SIM instrument, and these data have already been discussed controversially [see, e.g., Haigh et al., 2010]. While Merkel et al. [2011] conclude from their analysis of observations from the Microwave Limb Sounder (MLS) and Sounding of the Atmosphere using Broadband Emission Radiometry (SABER) and Global climate modeling that a higher UV variability is consistent with the different ozone solar cycle trends across the stratopause, Ball et al. [2016] argue that a higher UV variability is incompatible with the Stratospheric Water and Ozone Satellite Homogenized (SWOOSH) and the Global OZone Chemistry And Related trace gas Data records for the Stratosphere (GOZCARDS) data sets.

From Figures 2 and 3 it becomes clear that the overall trend in the SIM data cannot be reproduced by the multiscale analysis using the proxy data, as discussed above. On the one hand, it could be argued that the approach to extend the observational data sets, as described in section 4, has limitations. In addition, the reason might be that the degradation correction of the SIM data is incomplete. As pointed out by Ball et al. [2016] the observed SSI trend of the SIM instrument is inconsistent with observed ozone trends. This highlights the need to add additional constraints to future versions of the SOLID composite.

In order to investigate this further, we use an additional independent data set that is not employed in the composite itself. First, we use data taken with the VIRGO-SPM instrument [Fröhlich et al., 1995] on board SOHO. Wehrli et al. [2013] investigated the correlation of the relative annual variability of VIRGO-SPM channels and TSI over the time frame of 2002 to 2012. Interestingly, the relative variation as observed with the VIRGO-SPM channel at 500 nm gave a statistically significant positive correlation with the relative variation of TSI, with a regression factor of 1.65 and a correlation coefficient of r2=0.72.

Figure 6 shows the relative variation of the TSI scaled with the factor 1.65 (black) along with the relative variation at 500 nm channel of VIRGO-SPM and the relative variation at 500 nm as observed with SIM (red). The discrepancy between the observations from SIM and the scaled TSI variation suggests that—at least for the early phase of the SORCE mission until approximately 2010—a strong part of the variation seen by SIM may be instrumental. To further illustrate this, Figure 6 (right) shows the corresponding scatterplot of the relative scaled TSI (black line in Figure 6, left) and the relative SIM observation at 500 nm (red line in Figure 6, left) for the years from 2003 to 2012, for which SIM observations are available. The color code illustrates the different years of the time series. This plot indicates that the behavior of SIM changed around 2010. While its SSI observations continuously increase, the TSI first decreases from 2003 until the solar minimum in 2008 and then increases with the onset of solar cycle 24. From Wehrli et al. [2013] we know that the SSI at 500 nm is—with high statistical significance—in phase with the TSI solar cycle variability. Therefore, we conclude that the SIM data before 2010 are likely to be unreliable and disregard them for our purposes. SIM seems to have recovered after 2010 and provides observations that are in phase with the TSI variability. So, for the remainder of this paper we shall only use SIM data from 1 January 2010 onward.

Details are in the caption following the image
(left) Comparison of the scaled TSI (black), the VIRGO-SPM 500 nm channel (blue), and SORCE/SIM at 500 nm (red). Following Wehrli et al. [2013], the relative TSI variability has been multiplied by the regression coefficient 1.65. (right) Scatterplot of the relative scaled TSI variation (black line in Figure 6, left) and the scaled SIM observations (red curve in Figure 6, left). The color code indicates the different years when the observations were taken. The dotted line is the line with slope equal to 1, i.e., the expected gradient following Wehrli et al. [2013]. The data points for the years 2010 to 2012 (blue to black) that are employed in the revised SSI composite are additionally overlayed with grey triangles.

6 Final SSI Composite

Based on the discussions in section 13, we now provide the final version of the observational composite by only using SIM observations after 1 January 2010. All other observational data sets are kept as described above. Figures 7 and 8 show the relative contribution of the SSI in specific spectral ranges to the TSI. The figures are the same as Figures 3 and 4 but for the final composite. Not surprisingly, the SOLID composite now also shows a variation from 400 nm onward that is in phase with the solar cycle. In the spectral range between 200 and 242 nm the SOLID composite is slightly higher than the models but lower than the SORCE composite. For the spectral bins from 310–350 nm and 350–400 nm, the SOLID composite gives the lowest variation of all four (positively correlated) data sets. For 400–500 nm the SOLID composite is slightly lower than the three models. The SORCE data set shows a negative SSI contribution to TSI in this and all other spectral bins longward of that. For 500–600 nm the SOLID composite is almost identical to NRLSS1, while both SATIRE and NRLSSI2 give a slightly lower SSI variation. Then, for 600–800 nm the SOLID composite leads to the highest SSI contribution to the TSI variation. And finally, from 800–2000 nm SOLID is slightly higher than SATIRE, while NRLSSI2 largely exceeds the other data sets, and NRLSSI1 gives the lowest SSI contribution. For this spectral range we point out that the SIM data reveal a discontinuity at approximately 950 nm, where the SSI solar cycle variation changes from in phase to out of phase with solar activity. Based on our method to build the composite, this discontinuity is also present in the SOLID composite. While in our view the out-of-phase variability in the IR is correct, we would expect a more gradual transition than what is currently present in the SIM data and, as a consequence of that, also in the SOLID composite. Therefore, we raise some care with respect to this characteristic of the SSI composite.

Details are in the caption following the image
Same as Figure 3 but for the final SOLID composite.
Details are in the caption following the image
Same as Figure 4 but for the final SOLID composite.

Overall, the SOLID composite is an extremely valuable data set to understand how to combine disparate observations, and test SSI models, as well as solar proxies. However, this data set should be considered as the first step, with multiple possible improvements.

While the observational composite is consistent from a statistical point of view, future versions require additional physical knowledge to further reduce the impact of uncorrected instrumental artifacts. One obvious way of moving in that direction would be to build a hybrid data set that combines observational and modeled SSI. Another important step is to correct some of the observational data sets for instrumental artifacts.

A thorough evaluation of the results and comparisons to observations will help not only to assess differences, weaknesses, and strengths of the models but also to advance knowledge and further understanding of the physical mechanisms that govern the influence of the Sun to the Earth's atmosphere and climate.

7 Sensitivity of Atmospheric Heating to the SSI Data Set

7.1 Radiation Scheme

To demonstrate the effects of the different spectra on the atmospheric energy balance, we performed calculations with the LibRadtran v.2.0 high resolution model [Emde et al., 2016], which is a library of radiative transfer equation solvers widely used for UV and heating rate calculations (for a full list of relevant publications see http://www.libradtran.org). LibRadtran was configured with the pseudospherical approximation of DISORT solver, which accounts for the sphericity of the atmosphere, running in a six-stream mode. Calculations pertain to a cloud- and aerosol-free tropical atmosphere (0.56°N), the surface reflectivity is set to 0.1, and the effects of Rayleigh scattering are enabled. The atmosphere is portioned into 80 layers extending from the surface to 80 km. Here we show signals above 23 km. The model output is daily averages of spectral heating rates from 120 nm to 700 nm with a 1 nm spectral resolution for 15 January, calculated according to the recommendations for the Radiation Intercomparison of the Chemistry-Climate Model Validation Activity (CCMVal)(http://homepages.see.leeds.ac.uk/∼earpmf/ccmvalrad.shtml). Calculations were performed for NRLSSI2, SATIRE-S, SORCE, and SOLID for solar maximum (2003) and solar minimum (2008) years using the resolved solar spectra. That is, in the 200–310 nm spectral band, which is important for ozone formation and absorption, NRLSSI2, SORCE, and SOLID come in 1 nm bins, whereas the resolution of SATIRE-S is 1 nm up to 289.5 and 2 nm thereafter. For this study the parts of the spectra that come with a coarser spectral resolution than 1 nm are interpolated linearly. As pointed out by Deland and Cebula [2012] a different resolution than the SORCE native resolution could introduce additional effects in the heating rate calculation for that data set. The integrated difference between the solar maximum and solar minimum conditions is shown in Figure 4a for specific spectral intervals. Since we are primarily concerned here with the direct effect of atmospheric heating, the ozone feedback is not considered and ozone mixing ratios are kept constant to the climatology of a standard atmosphere in all calculations. The ozone response to solar cycle variations in SSI and the impacts of the use of NRLSSI and SORCE spectral irradiance as forcing functions has been reported by Swartz et al. [2012], who provide a detailed evaluation of the individual mechanisms of atmospheric response to solar cycle variations in SSI through direct solar heating and photolysis.

7.2 Effects in the Earth's Atmosphere

Figure 9a compares solar cycle anomalies of total spectral heating rates (K/day), integrated from 120 to 700 nm, for all SSI data sets. The solar cycle signature in heating rates is characterized by two distinctive maxima in the stratopause (50 km) and upper mesosphere (above 70 km), resulting from the larger solar fluxes available for absorption by O3 and O2. The SOLID data set introduces a solar cycle related heating of about 0.12 K/day at the stratopause, which is considerably stronger than NRLSSI2 but weaker than SORCE. The difference in heating rates between SOLID and SATIRE-S is not statistically significant if the uncertainty in the SOLID irradiances is taken into account (gray shading in Figure 9a). The respective uncertainty of heating rate anomalies ranges from 0.09 to 0.14 K/day. It should be noted that since UARS/SUSIM and UARS/SOLSTICE were taking measurements in 2003, their smaller amplitude for solar cycle variations below 310 nm moderates the SOLID forcing term relative to the SORCE-only forcing. This leads to weaker heating in the stratosphere between SOLID and SOLID.beta versions.

Details are in the caption following the image
Comparison of solar heating rate differences (K/day) between solar minimum (2008) and maximum (2003) for the SOLID (black solid), SOLID.beta (black dashed) composites, NRLSSI2 (green), SATIRE-S (blue), and SORCE (red). (a) The integrated (120–700 nm) heating rate anomalies; the relative contribution (%) of the (b) 130–200 nm, (c) 200–242 nm, (d) 242–310 nm, (e) 310–400 nm, and (f) 400–600 nm intervals to the integral. Grey shading indicates the uncertainty in the total heating rate anomalies from the SOLID SSI uncertainty (corresponding to the gray-shaded areas in Figure 3).

The solar cycle heating in the mesosphere is related to absorption in the Lyman α and the Schumann-Runge bands (175 to 200 nm) (Figures 9b and 10). More specifically, irradiance changes in wavelengths between 130 and 200 nm contribute about 30–50% of the total solar heating above 70 km in SOLID, with weaker contribution at higher altitudes as the strong heating in Lyman α is not a part of the integral in Figure 9b. NRLSSI2 shows a slightly stronger contribution in this region, whereas SOLID.beta shows the weakest. Figure 9 indicates that anomalous heating in the stratosphere occurs primarily from 220 nm to 310 nm, as expected from the strong ozone absorption in the Hertzberg continuum (200 to 242 nm) and Hartley bands (242 to 310 nm). The heating rate anomalies in the stratosphere in the Hartley bands are lower in SOLID compared to NRLSSI2 and SATIRE-S, whereas SORCE shows the strongest anomalies at all heights and spectral bins except from about 290 to 310 nm (Figure 10). Nevertheless, SSI changes in this spectral region contributes almost the same to the total heating in all data sets (Figures 9c and 9d).

Details are in the caption following the image
(a) Solar spectral heating rate anomalies (K/day/nm) between solar minimum (2008) and maximum (2003) for the SOLID composite. The integral gives the total heating anomalies shown in Figure 9a. Differences in the solar cycle heating rates (K/day/nm) between SOLID and (b) NRLSSI2, (c) SATIRE-S, and (d) SORCE.

A notable difference of SOLID compared to the other spectra is the strong positive contribution between 400 and 600 nm (Chappuis bands, Figure 9f) to the total heating of the lower tropical stratosphere. Solar fluxes in this spectral region increase by about 20-40% from solar minimum to maximum. On the other hand, SOLID.beta shows an opposite contribution with negative effect below 40 km. As described in section 12, the reason is that the SOLID.beta variability in this spectral range is essentially determined by the SIM observations, which in a similar fashion show the same negative SSI change. The assimilation of SIM observations after 2010 in SOLID now leads to a positive contribution, much stronger than NRLSSI2 and SATIRE, to balance the almost zero effect between 310 and 400 nm. Although, the line-by-line calculations do not take into account solar fluxes in wavelengths above 700 nm, the out-of-phase variability in SOLID.beta and SORCE is expected to introduce negative heating rate anomalies in the lower stratosphere and troposphere, as highlighted in other studies [Cahalan et al., 2010]. Without taking into account the early part of SORCE/SIM observations, we find with SOLID a considerable positive influence in the heating of the lower stratosphere stemming from the Chappuis bands, where, nevertheless, the total heating is negligible. Yet this strong change in the visible may cause significant changes on the surface climate and should be examined with climate model simulations.

The heating rate calculations of Oberländer et al. [2012] demonstrated a stronger solar cycle signal in the stratosphere for SATIRE-S compared to the first version of the NRL model. The SSI variability in NRLSSI2 still shows the weakest solar cycle signal as the heating rate anomalies at 50 km calculated with libRadtran are about 32% and 45% weaker than for SATIRE-S and SOLID, respectively (Figure 9a). As mentioned above, our interest lies in comparing the direct effects of SSI variability on atmospheric heating between the different SSI data sets; thus, we have deliberately neglected the ozone feedback. This of course results to the underestimation of the total (radiative and chemistry) solar forcing effect on atmospheric heating but allows for a more straightforward comparison, since the ozone response and the relevant combined effects strongly depend on the relative contribution of wavelengths that photolyze molecular oxygen (<240 nm) and ozone (<310 nm), respectively [e.g., Haigh et al., 2010; Swartz et al., 2012; Ball et al., 2014]. Gray et al. [2009] estimated that the ozone feedback amplifies the direct radiative heating in the stratopause by more than 50%, but it is less important in the middle stratosphere. On the other hand, the ozone effect was found dominant in the lower tropical stratosphere, contributing about 70–80% of the total solar cycle heating. The calculations of Swartz et al. [2012] suggested for even a stronger contribution of ozone feedback on the total stratospheric heating, which adds up linearly to the direct heating throughout the middle and upper stratosphere. A comprehensive assessment of the combined radiative and chemistry contributions shall be made in a future study using a chemistry-climate model.

8 Conclusion

In this study, we have introduced a novel framework for merging different SSI observations into one single homogeneous and continuous composite data set. Here we combined observations from 20 instruments into a single composite with daily values from 8 November 1978 to 31 December 2014, running from 0.5 to 1991.5  nm, with most bins having a 1 nm spectral resolution. Three important assets of this composite are (1) its reliance on uncertainty estimates of the instrumental data, which are derived in an independent way and are used to weigh their individual contribution to the composite; (2) the estimation of the uncertainty of the composite itself, which is essential for testing it against model reconstructions; and (3) the objectivity and traceability of the method, in the sense that the contribution of each instrument, and at specific timescales, can be controlled and adapted if necessary.

The present composite is solely based on original (unadjusted) observations, as we consider this to be a mandatory first step toward more elaborate products that will require additional physical information in order to enhance their scientific value. The list of obvious improvements includes (1) improve the uncertainty estimates of the instrumental records at different timescales and, in particular, at long ones (i.e., stability); (2) include model reconstructions to fill data gaps and to complement observations when these suffer from large uncertainties; (3) include physical constraints, such as the equality between the spectrally integrated SSI and the TSI, which is measured independently; and (4) additionally, exploit the high coherence of the SSI variability to exclude incoherent instrumental noise and thereby improve its signal-to-noise ratio. This approach, which has already been explored by Cessateur et al. [2011], can be incorporated as an intermediate step of the composite building, before the records are averaged. Since, however, it entails a decrease of the uncertainty, the latter must be carefully reevaluated.

We recently applied a similar approach to the TSI [Dudok de Wit et al., 2017] and to the sunspot number. Both have challenges on their own. With the TSI, no contributions from solar proxies are allowed. With sunspot numbers, the absence of absolute scale requires an additional normalization step. Although the maximum likelihood framework, which we are using here, is well suited for such purposes, eventually, a Bayesian approach will be required to properly specify the prior information.

In order to understand the role of the Sun as one of the natural forcing mechanisms, it is crucial to have a realistic estimate of the solar irradiance variability at hand. With the work presented we are confident we achieved a considerable step closer toward this goal.

Acknowledgments

The authors acknowledge that the research leading to these results has received funding from the European Community Seventh Framework Programme (FP7 2012) under grant agreement 313188 (SOLID, http://projects.pmodwrc.ch/solid). Moreover, this work was supported by the COST ActionES1005 TOSCA (Toward a more complete assessment of the impact of solar variability on the Earth's climate). The authors thank Stephan Nyeki for carefully proofreading the manuscript. This research has made use of NASA's Astrophysics Data System. We gratefully acknowledge all the instrumental teams for making their data freely available.