Comparison of Three Methodologies for Removal of RandomNoiseInduced Biases From SecondOrder Statistical Parameters of Lidar and Radar Measurements
Abstract
Randomnoiseinduced biases are inherent issues to the accurate derivation of secondorder statistical parameters (e.g., variances, fluxes, energy densities, and power spectra) from lidar and radar measurements. We demonstrate here for the first time an altitudeinterleaved method for eliminating such biases, following the original proposals by Gardner and Chu (2020, https://doi.org/10.1364/ao.400375) who demonstrated a timeinterleaved method. Interleaving in altitude bins provides two statistically independent samples over the same time period and nearly the same altitude range, thus enabling the replacement of variances that include the noiseinduced biases with covariances that are intrinsically free of such biases. Comparing the interleaved method with previous variance subtraction (VS) and spectral proportion (SP) methods using gravity wave potential energy density calculated from Antarctic lidar data and from a forward model, this study finds the accuracy and precision of each method differing in various conditions, each with its own strengths and weakness. VS performs well in highSNR, yet its accuracy fails at lowerSNR as it often yields negative values. SP is accurate and precise under highSNR, remaining accurate in worse conditions than VS would, yet develops a positive bias under lowSNR. The interleaved method is accurate in all SNRs but requires a large number of samples to drive randomnoise terms in covariances toward zero and to compensate for the reduced precision due to the splitting of return signals. Therefore, selecting the proper bias removal/elimination method for actual signal and sample conditions is crucial in utilizing lidar/radar data, as neglecting this can conceal trends or overstate atmospheric variability.
Key Points

We demonstrate for the first time an altitudeinterleaved method for eliminating randomnoiseinduced biases from secondorder statistics

The interleaved method is compared to the spectral proportion and variance subtraction methods to reveal strengths and limitations of each

Lidar data from Antarctica & a forward model are used to assess accuracies and precisions of the three methods under different conditions
Plain Language Summary
Secondorder statistics like atmospheric waveinduced variances or fluxes of physical parameters calculated from lidar or radar data would have positive biases if detectionnoiseinduced variances or fluxes were not properly eliminated. If such biases remained, not only would atmospheric variances be overestimated, implying unrealistic wave activity and growth rates, but certain phenomena could be concealed, presenting misleading pictures of the atmospheric observations. While various methods have been developed to account for this, all have limitations to their use. This study summarizes the development, theory, and application procedures for three different methods, namely conventional variance subtraction, the spectral proportion method, and the interleaved method, which can be used to correct such biases. The newest of these methods is adapted from operating on the data timewise into an altitudewise approach, improving its applicability. The performances of the three methods are compared by calculating the variancebased gravity wave potential energy density derived from lidar data taken at McMurdo, Antarctica. Their accuracies are assessed by using a forward model. This study aims to guide future research by providing information on how and when to apply each of these methods in order to enhance the outcomes of existing and future lidar/radar systems and datasets.
1 Introduction
Lidar and radar systems provide unrivaled monitoring of the middle and upper atmosphere, allowing for hightemporal/spatial resolution measurements of atmospheric parameters and constituents, which in turn enable the quantification of complex processes like atmospheric fluxes, constituent transport, turbulence, and gravity waves (e.g., Gardner & Liu, 2010; Hocking, 1996; Lu et al., 2015). These systems can observe a wide range of altitudes by taking advantage of various signalreturn mechanisms (e.g., Baumgarten, 2010; Chu et al., 2020; Chu, Yu, et al., 2011; Kaifler & Kaifler, 2021), which allows detailed studies into vertical coupling, analyzing how atmospheric processes develop as they travel over a wide range of altitudes. Such sophisticated systems have led to decades of impressive remote sensing campaigns (e.g., Li et al., 2020; She et al., 2019; Stober et al., 2021), yet to make full use of the data, advances must be made in data handling. Advances in science inevitably require the use of secondorder statistics such as variances and fluxes, which are inherently biased by randomnoise in the data generated during the detection processes (e.g., Chu et al., 2018; Gardner & Liu, 2014; Whiteway & Carswell, 1995). These biases grow increasingly problematic when analyzing the higher or wider reaches of lidar and radar data. To deal with these biases, various correction methods have been developed over the years (e.g., Chu et al., 2018; Gardner & Chu, 2020; Whiteway & Carswell, 1995), each with its own advantages and disadvantages. These methods have not yet been compared sidebyside to assess their effectiveness under various conditions, which are therefore the subject of this work.
On the righthand side of Equation 2, the second term is the noisevariance and the third term is the crossterm between the waveinduced and noiseinduced perturbations. Because and are independent and is random and possesses a zeromean, the expectation of the cross term , that is, this crossterm will vanish when averaging over a large number of samples. However, the noisevariance term will remain and contribute a positive bias. Such a bias must be corrected to accurately estimate the waveinduced variance. It is important to note that all the overbars above represent the sample average over time, not ensemble average (e.g., Gubner, 2006). Statistics performed in this kind of geophysical studies have assumed that the sampled perturbations induced by atmospheric waves arise from a stationary, ergodic random process. Therefore, the sample average can be used to approximate the ensemble average (Gardner & Chu, 2020), and in this case, the variance computed as the sample average like above can approximate the ensemble expectation of the variance, where “variance” is treated as a signal representing the wave strength, in the limit of very large number of samples. However, the limited number of samples not only differentiates computed variance from the theoretical expectation of signals but also introduces another consequence—the crossterm may not approach zero enough, adding additional uncertainties to the estimate of waveinduced variance.
The most direct way to isolate is to estimate at each altitude and then subtract it from the total variance. As the crossterm approaches zero due to noncorrelation after averaging over sufficient samples, only the waveinduced variance will remain. This approach is named the variance subtraction method (e.g., Duck et al., 2001; Whiteway & Carswell, 1995). Estimating this term becomes problematic in lowSNR conditions, as large uncertainties in the estimation often result in the estimated variancebias being greater than the geophysical variance itself, yielding a physicallyimpossible “negative variance” when the noise term is subtracted.
To overcome this issue, Chu et al. (2018) developed a solution called the spectral proportion method, where Monte Carlo simulations based on parameter uncertainties are used to estimate the waveoccupied proportion in the total at each altitude. Then the total variance is scaled down to estimate . By this method, there will be no negative variance induced by the waves. This technique may overestimate in highnoise scenarios as the uncertainty in determining increases substantially.
Gardner and Chu (2020) developed a new approach named the interleaved method. In this method, the return photon counts are split into two separate but interleaved samples so that two statistically independent samples probe the same air parcel over the same time period. Consequently, the variance in Equation 2 is substituted with a covariance between these two samples (see Section 2.3 for details) which no longer contains the noiseinduced bias once a statisticallysufficient sample size is used. This method improves the accuracy of the variance estimate by eliminating the noiseinduced bias yet decreases the precision both through increased uncertainty caused by photon count splitting and by any remaining terms containing the noiseinduced perturbations which have not approached zero under a limited sample size.
Since each approach has strengths and weaknesses, this paper compares these three methods in terms of their accuracy and precision using Antarctic lidar data as well as a forward model. The lidar data used here are the Rayleigh scattering signals collected with an Fe Boltzmann lidar from 2011 to 2020 at the Arrival Heights Observatory near McMurdo Station (Chu, Huang, et al., 2011; Chu et al., 2020; Chu, Yu, et al., 2011). Although these techniques are demonstrated on lidar measurements, they can be applied to radar data as both are similarly affected by noiseinduced biases in higherorder parameters. Additionally, this paper demonstrates an alternative way to apply the interleaved method by interleaving in altitude as opposed to timeinterleaving as initially demonstrated in Gardner and Chu (2020).
2 Three Methodologies
can be calculated either from temperature gradient or from atmospheric density gradient and speed of sound Here, is the background temperature, is the background density, and and are the specific heat at constant pressure and constant volume, respectively. We use the densitybased calculation for estimating from density, and the temperaturebased to calculate from temperature, though the two yielded nearly identical results.
In Equation 5, the last two terms are not waveinduced but introduced by noise, with the second introducing a positive bias and the third term introducing additional noise. Although is used as the demonstration here, atmospheric kinetic energy has similar issues with noiseinduced bias, as where and are the waveinduced wind perturbations, and therefore can be treated similarly to .
2.1 NoiseVariance Subtraction
The limitations of this method become obvious when attempting to use it on noisy data. Here, growing uncertainty in parameter error can cause to become so large that it exceeds , producing a negative value for , a nonphysical result (see Sections 3 and 4).
2.2 Spectral Proportion Method
These negative values led to the development of the spectral proportion method in Chu et al. (2018) which eliminates the possibility of negative . This method also uses the calculated total variance, and then scales it down to estimate the waveinduced energy. As demonstrated in Chu et al. (2018), we first perform a Monte Carlo simulation by constructing 1,000 copies of , adding normallydistributed noise onto them with a standard deviation equal to the measurement uncertainty at that altitude and time, and calculating the perturbations for each of these 1000fields individually. Then, for each altitude, the 1DFFT of the perturbations is calculated for each field and averaged over all 1,000 iterations. The noise floor level is then estimated from the averagedspectral plots shown in Figures 1a–1d by taking the average of all minima locations above frequency , ignoring the highest minima. We only use minima above to avoid the influence of any spectral filters applied during the background subtraction, and we ignore the highest minima to ensure that we do not include any troughs which are actually above the noise floor.
This method can, however, introduce positivebias under high noise, which is discussed further in Sections 3 and 4.
2.3 Interleaved Method
The common idea of the previous two methods was based on the total variance calculation that includes both wave and noiseinduced variances, and then each method employed some algorithm to either remove the estimated noisevariance or scale down the total variance to estimate the waveinduced variance. The interleaved method instead eliminates the noiseinduced bias altogether by calculating the covariance of simultaneous, collocated samples taken in a way such that the noiseterms are driven toward zero. Gardner and Chu (2020) point out that this bias elimination would optimally be done using two adjacent lidars, but such a setup would be complex, expensive, and uncommon. The interleaved method they propose instead introduces a practical way to achieve the same biaselimination using a single lidar (see diagrams in Figure 2). Gardner and Chu (2020) have demonstrated interleaving time bins (Figures 2a and 2c) for the covariance calculation but suggested that in many lidar systems it may make more sense to apply it to adjacent altitude bins. Here we describe the altitudeinterleaving method.
This flexibility in interleavingdirection allows for the interleaved method to be used on data taken by many different lidar systems. For example, the Na Doppler lidar used as an example in Gardner and Chu (2020) saves its raw data in time intervals of 4.5 s, which means the data can be very finely timeinterleaved, while the Fe Boltzmann lidar used for this study saves its data in time intervals of 1 min, likely too coarse to use a timeinterleaved approach. However, the Fe Boltzmann lidar saves its counts at a highvertical resolution of 48 m, allowing the interleaved method to be utilized altitudinally.
Moreover, as interleaving in derived atmospheric parameters (e.g., wind velocities ) is equivalent to interleaving in raw data (e.g., radar power or lidar photon counts). Therefore, when applying the interleaved method to radar measurements, one may choose to interleave highresolution in time or altitude domain if interleaving the raw return signals is challenging. By averaging together odd and even (in either time or altitude) wind values, respectively, to create two independent samples, for example, and , the covariance will be biasfree and can be used in place of variance. Researchers using radar techniques like coherent detection may be able to cleverly implement this interleaving idea in the raw data (more analogous to the photon count interleaving for lidars).
3 Error Analyses
Results obtained from these three methods are compared in Figure 3, where four cases are shown—winter and summer observations representing high and low SNRs, respectively, with both large and small samples sizes. By comparing these cases, one can get a sense for how the accuracy and precision of each method respond to increased sample size, as well as how they behave in varying noise levels. Before discussing these results in Section 4, it is necessary to introduce the analyses of uncertainty in precision and bias in accuracy.
3.1 Precision (Uncertainty) Analysis
We tabulate the uncertainty equations in Table 1, among which Equations 1820 consider the photonnoiseinduced uncertainty only. That is, only the error propagation from is considered while any statistical error (see below) is omitted. The uncertainty in the variance subtraction method is calculated using the noisevariance given by Equation 6, and the spectral proportion method's uncertainty is found using as in Chu et al. (2018). In the interleaved method, the uncertainty is increased because of the reduction in SNR caused by splitting the photon counts into two groups. That splitting manifests itself as a on thus there is a factor of two in Equation 20. This factor decreases the precision of the measurement, necessitating a larger sample size to reduce the overall uncertainty.
Randomnoiseinduced uncertainties^{a}  

Variance subtraction^{b} 

Spectral proportion  
Interleaved method 
Total uncertainties (including both statistical and random errors)  

Variance subtraction 

Spectral proportion 

Interleaved method 

Equations 1820 are suitable for estimating the randomnoiseinduced uncertainty for of individual measurements. If regarding each individual measurement of as a sample of the overall stationary ergodic signal, then the statistical error of sampling needs to be taken into account when taking the sample time average. Gardner and Chu (2020) have presented extensive analyses on such overall uncertainties of the estimated sample variances and covariances. Here, we apply their results of and tabulate the total equations as Equation 21 through Equation 23. The statistical error component, the first term of Equations 2123, relates the individually measured atmospheric variances to the presumed stationary signal and expresses uncertainty based on the observational length and the correlation time of the parameter that is, the number of total independent samples taken during the observations for the sample average. Driving this statistical error toward zero requires a large number of samples, so this statistical error introduces a persistent value to the calculated statistical uncertainty.
3.2 Accuracy Analysis With Forward Modeling
While the precision of each method can be assessed from Figure 3 and Table 1, the analysis in Section 3.1 does not give much insight into the accuracies of these methods. To address this issue, a forward model was developed to test their performance. As the modeled wave energy is known, it is possible to assess any potential systematic bias introduced by each of the three methods, in addition to the assessment of their uncertainties in precision.
represents the photon counts from these unwanted sources and is estimated by averaging the Fe Boltzmann lidar's photon counts from 150–180 km over many observations during the appropriate season. Poissondistributed noise is then added to by using the photon count at each altitudetime grid point as the rate parameter for a Poissondistributed sampling, resulting in , which is a matrix of noisy counts resembling those received by the photon counting system.
is then treated as if it were a real lidar observation and processed as usual using all three biasremoval/elimination methods. is processed identically to to generate a reference for the true, modeled in the field with no bias from photon noise. This simulation process was repeated with the same wave spectra many times, each iteration with distinct, random noise, and the results from each method were averaged to create Figure 5. We used the same spectra in generating the wavefield in each of these plots to satisfy the stationary signal requirement (discussed in Section 1) for averaging large amounts of data, which also serves to minimize any additional effects which could obscure the accuracy of the methods.
4 Comparison of Three Methods
We now analyze how the three methods perform under different conditions of SNR and number of samples, in terms of their accuracy and precision. We define accuracy as how close the results are to the true atmospheric and precision as how well the results are determined (with no reference to the true values) as well as how repeatable the results would be (Bevington & Robinson, 2003). In this dataset, the summer measurements have lower SNRs than winter due to the full sunlight under which the data was collected. Additionally, as this data is derived from Rayleigh scattering, higher altitudes correspond to lower SNRs due to the exponential decrease of atmospheric number density with increasing altitude. Both temperature and density can be derived from these Rayleigh scattering signals. For this study, we use density, but these same values can also be derived from temperature. This relation is explored further in Appendix A.
We first compare the precision of these methods under various conditions. The precision of each method is largely determined by SNR and is increased with the use of a greater sample size (as the uncertainty is decreased). This is evidenced by Equations 1823, as the term (which reflects SNR) scales directly with the uncertainty, and the term scales down this uncertainty as the number of observations increases. In Figures 3a and 3e, the variance subtraction and the interleaved method cannot be scientifically interpreted as shown due to their negative values. The spectral proportion method appears to show a realistic trend, even when using the small sample size. In Figures 3c and 3g, as more samples are incorporated, the variance removal and spectral proportion method continue to show similar, yet more precise trends. The interleaved method now yields nearlyentirely positive results which trend at a lower than the spectral proportion derived results.
We then assess the accuracy of the methods using the forward modeled results in Figure 5. The performance of the variance subtraction method in all conditions clearly shows that this method has low accuracy under highnoise due to its negative values in Figure 3 and its strong departure from the modeled seen in Figure 5. While the spectral proportion method yields a precise profile in all cases, Figures 5b and 5d show that it overestimates when the SNR becomes low, as evidenced by the departure from the modeled . While the interleaved method showed an especially noisy result under a small samplesize, Figure 5 shows that this noisy profile still generally centers around the modeled , with the same remaining true when using a largersample size.
The negativebias of variance subtraction method is due to the uncertainty in the estimation of the noisevariance. As the noisevariance is computed using the temperature or density error as in Equation 6, large uncertainties in these error values inevitably occur near the top of the measurement (where the SNR has significantly declined) which cause the noisevariance to increase dramatically. When subtracted from total variance via Equation 7 these often yield negative variances and thus, physicallyimpossible negative values. Alongside the positive bias caused by the second term of Equation 2, noise can be introduced by the third term if the samplesize was insufficient to drive it close to zero. While these tests show poor performance from the variance subtraction method at higheraltitudes, the plots still show that it is a valid approach under highSNRs. It has been successfully utilized many times before for loweraltitude studies (e.g., Chu et al., 2009; Duck et al., 2001; Yamashita et al., 2009).
The positive bias of spectral proportion method seen in Figures 3g and 5d is caused by highnoise in the initial sample contaminating the spectra at a given altitude. Looking at Figures 1a–1d, there is a regularly occurring peak near ≈ , yet Figure 1d shows additional peaks which are not present at the other altitudes. These could be due to a localized wave present at this altitude throughout the signal, but extensive testing and modeling has revealed that these peaks can be caused by strong noise present in the observation. As this noiseinduced peak disguises itself as a waveinduced peak, any noisefloor determination method which captures the energy in the wavepeaks will also capture the energy in the noiseinduced peak, causing an overestimation in the waveenergy calculated in the This systematic underestimation of the noise floor reflects how strong noise is able to cause a positive bias in the form of an overestimated when using the spectral proportion method, leading to an overestimated on noisy data.
The interleaved method does not suffer from either positive or negative bias and generally remains centered around the modeled . Its core drawback is the increased uncertainty due to the splitting of the photon counts into two subsamples and additional noise contributed by the final three terms in Equation 10 (if they have not been driven to zero). The bin splitting reduces the signal level in each coarse bin by half, increasing overall uncertainty by . This increase in uncertainty is significant, and, coupled with the last three terms in Equation 10, can often lead to the derived displaying negative values. Without enough samples to beat down the uncertainty to a certain level and zeroout the noise terms from Equation 10, singleobservation results from the interleaved method may not be scientifically meaningful. Additionally, remaining noiseterms which did not approach zero due to a small sample size will affect the resultant with the capability to strongly offset the value calculated by the covariance. The stronger the noiseterms, the more samples are needed to drive the noise terms toward zero and remove their influence. Figure 6 further illustrates these trends by showing the behavior of an entire month of summer observations which is overlayed with its mean. For many of these individual runs, negative values may occur at many altitudes, but given a large enough samples size, the mean value becomes positive. However, even with a large number of samples as shown in Figure 3g, the uppermost bin is still negative, and obviously deviates from the trend established by the bins below. This result reinforces that lowerSNR increasingly necessitates the use of a larger sample size to compensate for the binsplittinginduced uncertainty increase as well as to facilitate the driving of the noise terms toward zero. Appendix B further elaborates on the development of this precision increase with the increasing sample size.
5 Comparison to Previous Results
We now compare the performance of the interleaved method to that of the spectral proportion in Figure 7 by replicating results from a previously published study. In Chu et al. (2018), lidar data from 2011 to 2015 taken by the McMurdo Fe Boltzmann lidar was processed with the spectral proportion method to yield . One of the major findings from that study was the strong asymmetry between summer and winter . That study found significantly stronger in winter as opposed to summer, with this pattern repeating annually.
The comparison in Figure 7 has demonstrated the repeatability of the seasonal asymmetry observations. The interleaved method generally agreed with the trends attained from the spectral proportion method. While some of these interleaved values are negative, it is now known from Section 4 that we must average over many samples (or apply a fit, as we do here) to reveal the true trend. Not only are the observational results replicated by the interleaved method, but since the interleaved method is more accurate in the highnoise summer and does not overestimate like the spectral proportion method, the summertime derived using the interleaved method yields lower summer minimums, and in the winters, a slightly higher maximum. This result means that the major conclusions of Chu et al. (2018) are strengthened by the use of the interleaved method.
6 Conclusions and Recommendations
Randomnoiseinduced biases are inherent issues to the accurate derivation of secondorder statistical parameters (such as temperature, wind and species variances, momentum, heat and constituent fluxes, potential and kinetic energy densities of atmospheric waves, and power spectrum estimates) from lidar and radar measurements. As the boundaries of existing research expand, powerful techniques for removal of such biases must be developed to take full advantage of data collection campaigns. The variance subtraction, spectral proportion, and interleaved methods are all viable means to correct for the biases, yet the performance of each method varies depending on the conditions under which they are applied.
Based on the comparisons using the lidar observational datasets from Antarctica as well as the forwardmodeled cases, we draw the following conclusions. The variance subtraction method is best used with highSNR observations, as it is easily biasednegatively by noise in the data. It provides a precise, yet not always accurate, measurement of atmospheric variance even with a relativelysmall sample size. The spectral proportion method is more robust, yielding precise and accurate measurements of variance in significantly noisier data than the variance subtraction method, and also does not rely on a large sample size (Chu et al., 2018). However, it begins to display a positivebias under highnoise conditions. The interleaved method is the only method which will intrinsically not have a bias because it eliminates the randomnoiseinduced biases utilizing two statistically independent datasets that cover the same altitude range and time period (Gardner & Chu, 2020). However, such improved accuracy is attained at the price of reduced precision, necessitating a muchlarger sample size than the others even for highSNR measurements.
This work is the first demonstration of altitude/rangeinterleaved method for deriving secondorder statistics, following the original proposal by Gardner and Chu (2020). Interleaving in altitude (or range) bins provides two statistically independent samples over the same time period and altitude range even if the original raw data were not saved in high temporal resolutions but sufficiently high spatial (range) resolutions. Therefore, the altitude/rangeinterleaved method provides a suitable solution to many current and historic lidar and radar datasets for accurately deriving variances, fluxes, wave energy densities, and power spectrum estimates, etc.
Given the overall considerations we recommend applying the interleaved (either in time or in altitude/range bins) method and the spectral proportion method in real applications because they are superior to the noise subtraction method as demonstrated in this work. When the application goals are to derive statistically mean profiles with high accuracy and/or there are a large number of samples, the interleaved method would be the best choice because it inherently eliminates the noiseinduced biases to give the highest accuracy while the large sample size reduces uncertainties to ensure sufficient precision as well. However, if the application goals are to derive secondorder statistics within a small number of samples and then study the time evolution of such statistics over month, season, and/or year (i.e., nonstationary signals in longer time periods), the spectral proportion may be a better choice for its higher precision and the ability to handle small sample sizes with a caveat of potentially positivebiases in highnoise conditions. Applying the proper biasremoval method can unlock the full potential of a dataset, allowing retrieval of second or higherorder parameters into lowerSNR regions of the data. Additionally, it can reveal trends in the data that may otherwise be concealed by the bias, such as the seasonal asymmetry demonstrated prior, or altitudinal trends which have not yet been discovered.
Acknowledgments
We gratefully acknowledge the graduate students and research scientists who made contributions to the McMurdo lidar campaign, including winterover lidar scientists Zhibin Yu (2011), Brendan Roberts (2012), Weichun Fong (2013), Cao Chen (2014), Jian Zhao (2015), Ian Barry (2016), Zhengyu Hua (2017), Dongming Chang (2018), Zimu Li and Ian Geraghty (2019), and Xianxin Li and Cissi Lin (2020), and summer scientists Wentao Huang, Zhangjun Wang, John A. Smith, Xian Lu, Muzhou Lu, and Clare Miller. We are grateful to Richard Dean, Nikolas Sinkola, and Adam Godfrey for their engineering help and support. We sincerely appreciate the staff of the United States Antarctic Program, McMurdo Station, Antarctica, New Zealand, and Scott Base for their superb support over the years. This work was partially supported by NSF grants OPP1246405, OPP1443726, OPP2110428, AGS2029162, and AGS1452351. The work of X. Chu was partially supported by NASA LWS grant 80NSSC20K0002. J. Jandreau is grateful to the generous support of Cooperative Institute for Research in Environmental Sciences (CIRES) Graduate Student Research Award, George C. and Joan A. Reid Memorial Scholarship, and Aerospace Engineering Sciences Smead Scholars Program.
Appendix A: Considerations on Employing Temperature Versus Density for Rayleigh Lidar
Using Rayleigh lidar data, the gravity wave potential energy mass density can be calculated from the relative perturbations of atmospheric density or temperature. If done properly, the two will yield the same results. To derive temperatures using the Rayleigh integration technique (Chu et al., 2002) a seed temperature at the top of the altitude range (generally lower SNR) must be used to calculate each point along the altitude profile. The deviation of the empirical seed temperature from the actual value as well as the photon noise at the seeding altitude could result in noisy Rayleigh temperature profiles in the first several scale heights from the seeding point. In contrast, the relative density profile is derived using a bin at the bottom of the altitude range where the SNR is generally higher, reducing the chance of referencebinnoise propagation. More importantly, no seeding temperature is needed for the relative density profiling, thus further removing uncertainties. However, the knowledge of the background temperature profile is still necessary in order to calculate the speed of sound for the computation (see Equations 3 and 4 in Section 2).
While the atmospheric density may seem superior overall in the derivation, it is often preferred by many authors to use temperature for studies, as density (e.g., like minor species densities) can reflect highrate chemical fluctuations which add additional perturbations to the signal, obscuring the wave dynamics. Fortunately, Rayleigh lidars measure the total atmospheric number density, which is dominated by the chemically stable and ; therefore, the Rayleigh lidarmeasured number density does not suffer the chemical fluctuations. Both temperature and density are reasonable approaches for utilizing the Rayleigh lidar data for studies, as long as the above effects are considered and avoided. It can be helpful to calculate the results via both methods and compare the two, as they should agree if applied properly. This can be seen in Figure A1, where the results are plotted from the Fe Boltzmann/Rayleigh lidar. In this figure, it is clear that even though the uncorrected total values derived from relative density and temperature perturbations may not agree, the corrected waveinduced results derived from both parameters using either the spectral proportion method or the interleaved method generally agree with each other.
Appendix B: Development of Precision With Sample Size in the Interleaved Method
A primary downside of the interleaved method is the reduction of precision caused by splitting the samples into two groups. This is best countered by increasing the number of samples used for the measurements, which reduces the uncertainties roughly by a factor of , where is the number of independent samples used (Bevington & Robinson, 2003). Figure B1a demonstrates this by showing the development of interleaved profiles with the inclusion of additional samples, where it can clearly be seen that additional samples reduce the noiseinduced uncertainties, particularly those present at high altitudes where the SNRs are low. To quantify the uncertainty reduction, we define the rootmeansquare (RMS) errors as the difference between the derived from all samples and the derived from a smaller sample size. Such RMS errors are similar to the total uncertainties given in Table 1, which include both the statistical errors and the randomnoiseinduced errors. Figures B1b and B1c show that the RMS errors in the altitude ranges of 30–69 km and 30–50 km, respectively, decrease with the increasing sample length in hours at a trend proportional to , with only minor deviations from this trend due to nonuniform SNRs in individual samples. For our data in July at Antarctica, after 200 hr of data were averaged (i.e., 100, 2hr samples), the uncertainty was already decreased by ∼80%. For any observations, the exact number of samples needed to improve the measurement precision depends on the general quality of the actual data, especially its SNRs, as well as the correlation time of the measured atmospheric parameters (see Table 1).
Open Research
Data Availability Statement
The data shown in this work can be downloaded online from https://data.mendeley.com/datasets/5cryh29t67/3.