Volume 20, Issue 4 e2021SW002906
Research Article
Open Access

Real-Time 3-D Modeling of the Ground Electric Field Due To Space Weather Events. A Concept and Its Validation

Mikhail Kruglyakov

Corresponding Author

Mikhail Kruglyakov

University of Otago, Dunedin, New Zealand

Geoelectromagnetic Research Center, Institute of Physics of the Earth, Moscow, Russia

Correspondence to:

M. Kruglyakov,

[email protected]

Contribution: Methodology, Software, Validation, Writing - review & editing

Search for more papers by this author
Alexey Kuvshinov

Alexey Kuvshinov

Institute of Geophysics, ETH Zürich, Zürich, Switzerland

Contribution: Conceptualization, Resources, Writing - original draft, Writing - review & editing, Supervision

Search for more papers by this author
Elena Marshalko

Elena Marshalko

Institute of Physics of the Earth, Moscow, Russia

Geophysical Center, Moscow, Russia

Finnish Meteorological Institute, Helsinki, Finland

Contribution: Software, Validation, Writing - review & editing, Visualization

Search for more papers by this author
First published: 30 March 2022
Citations: 8

Abstract

We present a methodology that allows researchers to simulate in real time the spatiotemporal dynamics of the ground electric field (GEF) in a given 3-D conductivity model of the Earth based on continuously augmented data on the spatiotemporal evolution of the inducing source. The formalism relies on the factorization of the source by spatial modes (SM) and time series of respective expansion coefficients and exploits precomputed GEF kernels generated by corresponding SM. To validate the formalism, we invoke a high-resolution 3-D conductivity model of Fennoscandia and consider a realistic source built using the Spherical Elementary Current Systems (SECS) method as applied to magnetic field data from the International Monitor for Auroral Geomagnetic Effect network of observations. The factorization of the SECS-recovered source is then performed using the principal component analysis. Eventually, we show that the GEF computation at a given time instant on a 512 × 512 grid requires less than 0.025 s provided that GEF kernels due to pre-selected SM are computed in advance. Taking the 7–8 September 2017 geomagnetic storm as a space weather event, we show that real-time high-resolution 3-D modeling of the GEF is feasible. This opens a practical opportunity for GEF (and eventually geomagnetically induced currents) nowcasting and forecasting.

Key Points

  • We present the formalism of real-time modeling of the ground electric field (GEF) excited by temporally and spatially varying source

  • The formalism relies on the factorization of the source and exploits precomputed GEF kernels

  • Using Fennoscandia as a test region, we show that real-time 3-D modeling of the GEF takes less than 0.025 s

Plain Language Summary

The solar activity in the form of coronal mass ejections leads to abnormal fluctuations of the geomagnetic field. These fluctuations, in their turn, generate so-called geomagnetically induced currents (GIC) in electric power grids, which may pose a significant risk to the reliability and durability of such infrastructure. Forecasting GIC is one of the grand challenges of the modern space weather studies. The critical component of such forecasting is real-time simulation of the ground electric field (GEF), which depends on the electrical conductivity distribution inside the Earth and the spatiotemporal structure of geomagnetic field fluctuations. In this paper, we present and validate a methodology that allows researchers to simulate the GEF in fractions of a second (thus, in real time) irrespective of the complexity of the Earth's conductivity and geomagnetic field fluctuations models.

1 Introduction

As commonly recognized, geomagnetically induced currents (GIC) in electric power grids may pose a significant risk to the reliability and durability of such infrastructure (Bolduc, 2002; Love et al., 2018).

The ultimate goal of quantitative estimation of the hazard to power grids from abnormal geomagnetic disturbances (space weather events) is real-time and as realistic as practicable forecasting of GIC. Under GIC forecasting, we understand the time-domain computation of GIC using continuously augmented data on the spatiotemporal evolution of the source responsible for the geomagnetic disturbances. Specifically, to forecast GIC in the region of interest, one needs: (a) to adequately parameterize the source of geomagnetic disturbances; (b) to forecast the spatiotemporal evolution of the source in the region; (c) to specify/build a three-dimensional (3-D) electrical conductivity model of the Earth's subsurface; (d) to perform real-time modeling of the ground electric field (GEF) in a given 3-D conductivity model, that is, to compute as fast as feasible the spatiotemporal progression of the GEF from continuously augmented data on the spatiotemporal evolution of the forecasted source; (e) to convert the “forecasted” GEF into GIC.

It is well accepted that the decades of satellite observations of the solar wind parameters (plus observations of interplanetary magnetic field) at the L1 Lagrangian point are the most promising data for forecasting the spatiotemporal evolution of the source with algorithms known as neural networks (NN). Despite numerous studies that attempt to forecast the source evolution using different NN architectures quantitatively, the progress here is rather limited. This is, in particular, because the full potential of NN remains unexplored; the reader can find a rather exhaustive review of the literature on the subject in Tasistro-Hart et al. (2021). But even if the source forecasting will be feasible in the future, with the measurements at the L1 point, it is nearly impossible to forecast the source more than an hour in advance. This, in particular, means that forecasting GEF in a given 3-D conductivity model from continuously augmented data on the spatiotemporal evolution of the forecasted source should be performed “on the fly”, that is, within a few seconds, if one wishes to approach an ultimate goal of GIC forecasting in the region of interest—development of trustful alerting systems for the power industry. Note that once the GEF is forecasted, a conversion of the GEF into GIC is rather straightforward (Kelbert, 2020) and requires fractions of seconds provided the geometry of transmission lines and system design parameters are granted by power companies.

This paper presents and validates a methodology that allows researchers to simulate the spatiotemporal progression of the GEF in a 3-D conductivity model “on the fly”. The paper also details how the concept can be exploited for GEF nowcasting and forecasting.

2 Methodology

2.1 Governing Equations in the Frequency Domain

We start with the discussion of the problem in the frequency domain. Maxwell's equations govern electromagnetic (EM) field variations, and in the frequency domain, these equations are read as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0001(1)
urn:x-wiley:15427390:media:swe21301:swe21301-math-0002(2)
where μ0 is the magnetic permeability of free space; ω is angular frequency; jext(r, ω) is the extraneous (inducing) electric current density; B(r, ω; σ) and E(r, ω; σ) are magnetic and electric fields, respectively. σ(r) is the spatial distribution of electrical conductivity, r is a position vector, either in the spherical or Cartesian coordinates. Note that we neglected displacement currents and adopt the following Fourier convention
urn:x-wiley:15427390:media:swe21301:swe21301-math-0003(3)
Note that we will use the same notation for the fields in the time and frequency domain. We also assume that the current density, jext(r, ω), can be represented as a linear combination of spatial modes (SM) ji(r),
urn:x-wiley:15427390:media:swe21301:swe21301-math-0004(4)

Note also that the form of ji(r) (and their number, L) varies with application. For example, jext(r, ω) is parameterized via spherical harmonics in Püthe and Kuvshinov (2013), Honkonen et al. (2018), Guzavina et al. (2019), and Grayver et al. (2021), current loops in Sun et al. (2015), or eigenmodes from the Principal component analysis (PCA) of the physics-based models in Egbert et al. (2021) and Zenhausern et al. (2021).

By virtue of the linearity of Maxwell's equations with respect to the jext(r, ω), we can expand the total (i.e., inducing plus induced) electric field as a linear combination of individual fields Ei,
urn:x-wiley:15427390:media:swe21301:swe21301-math-0005(5)
where the Ei(r, ω; σ) is the “electric” solution of the following Maxwell's equations:
urn:x-wiley:15427390:media:swe21301:swe21301-math-0006(6)
urn:x-wiley:15427390:media:swe21301:swe21301-math-0007(7)

2.2 Governing Equations in the Time Domain

The transformation of Equation 5 into the time domain leads to the representation of the time-varying GEF as a sum of convolution integrals
urn:x-wiley:15427390:media:swe21301:swe21301-math-0008(8)
or equivalently
urn:x-wiley:15427390:media:swe21301:swe21301-math-0009(9)
where rs stands for the position vector at the surface of the Earth. The reader is referred to Appendix A for more details on the convolution integrals in Equations 8 and 9.

Since the radial component of the GEF is negligibly small (due to insulating air) and is not used in GIC calculations (Kelbert, 2020), we will confine ourselves to modeling the horizontal electric field solely; thus, hereinafter, Ei will stand for Ei = (Ex,i Ey,i).

2.3 Real-Time Modeling of the GEF. A Concept

Equation 9 shows how the GEF can be modeled using continuously augmented data on the temporal evolution of the nowcasted or forecasted ci (the reader is referred to Section 4.2 where nowcasting/forecasting of ci is briefly discussed). To make the formula ready for implementation, one needs: (a) to specify a set of SM, ji, i = 1, 2, …, L in the region, where GIC nowcasting/forecasting is required; we will discuss the construction of ji in Section 3.1; (b) to set up a 3-D conductivity model in this region; and (c) to estimate an upper limit of integrals in Equation 9, or, in other words, to estimate the integration time interval, T, above which Ei(rs, τ; σ) becomes negligibly small. The latter will allow us to rewrite Equation 9 as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0010(10)

Note that the upper limit in the integrals could be different for different SM, different components of the field, and different locations. However, one can choose a conservative approach, taking a single T irrespective of modes/components/locations as a maximum from all individual upper limit estimates. We will discuss the estimation of T in Sections 3.3 and 3.4.

The details of numerical calculation of the integrals in Equation 10 are presented in Appendix B. In short, assuming that ci(t), i = 1, 2, …, L are time series with the sampling interval Δt and T = NtΔt, we approximate E(rs, tk; σ) at tk = kΔt as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0011(11)
where di is defined as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0012(12)

The reasoning to represent the time-dependent part in Equation 11 in this form is given in Appendix B. Note also that quantities urn:x-wiley:15427390:media:swe21301:swe21301-math-0013 and urn:x-wiley:15427390:media:swe21301:swe21301-math-0014 are time-invariant, and for the given ji, i = 1, 2, …, L and 3-D conductivity model are calculated only once, then stored and used, when the calculation of E(rs, tk; σ) is required. Actual form and estimation of kernels urn:x-wiley:15427390:media:swe21301:swe21301-math-0015 and urn:x-wiley:15427390:media:swe21301:swe21301-math-0016 are also discussed in Appendix B.

Equation 11 is the essence of the real-time GEF calculation, showing that urn:x-wiley:15427390:media:swe21301:swe21301-math-0017 summations and multiplications are required to compute the GEF at a (current) time instant tk plus some overhead to read the precomputed urn:x-wiley:15427390:media:swe21301:swe21301-math-0018 and urn:x-wiley:15427390:media:swe21301:swe21301-math-0019 from the disc. Note that Ng is a number of points rs, at which the GEF is computed.

3 Real-Time Modeling of the GEF. Validation of the Concept

The validation of the presented concept will be performed using Fennoscandia as a test region. The choice of Fennoscandia is motivated by several reasons. First, it is a high-latitude region, where GIC are expected to be especially large. Second, there exists a 3-D electrical conductivity model of the region (Korja et al., 2002). Third, the regional magnetometer network (International Monitor for Auroral Geomagnetic Effect (IMAGE; Tanskanen, 2009)), allows us to build a realistic model of the source. Finally, the last but not the least consideration to choose this region is the fact that we have already performed a comprehensive 3-D EM model study in this region (Marshalko et al., 2021).

3.1 Building a Model of the Source

First, let us rewrite Equation 4 in the time domain
urn:x-wiley:15427390:media:swe21301:swe21301-math-0020(13)
We will further assume that the extraneous current jext(r, t) is divergence-free, it flows in a thin layer at the altitude of h = 90 km, and this layer is separated from the Earth by the insulating atmosphere. Following the Spherical Elementary Current Systems (SECS) method (Vanhamäki & Juusola, 2020), this current is represented as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0021(14)
where δ is Dirac's delta function, eϑ and eφ are unit vectors of the spherical coordinate system, r = (R, ϑ, φ), rm = (R, ϑm, φm), R = a + h, a is a mean radius of the Earth, rm is the location of the pole of the mth spherical elementary current system and Sm is the so-called scalar factor associated with the mth pole. Expressions for P(r, rm) and Q(r, rm) are presented in Appendix D. Note that in practice r and rm are usually taken as the nodes of two (similar) grids, which are slightly shifted with respect to each other (the reason for the shift is explained in Appendix D). Once Sm(t), m = 1…M are obtained by means of the SECS method as applied to some real data for some event, one can perform the PCA of Sm(t) expecting that the spatial structure of Sm(t) will be well approximated with a small number of principal components (PC) urn:x-wiley:15427390:media:swe21301:swe21301-math-0094, i = 1, 2, …L allowing to represent ji as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0022(15)

The aim of this section is to obtain urn:x-wiley:15427390:media:swe21301:swe21301-math-0143 and, consequently, ji (using Equation 15). To this end, we apply the SECS method to 10-s vector magnetic field data from all available (38) stations of the IMAGE network during the 7–8 September 2017 geomagnetic storm. Locations of IMAGE sites are shown in Figure 1. Considered (8-hr) time period is from 20:00:00 UT, 7 September 2017, to 03:59:50 UT, 8 September 2017, thus, including the onset and the main phase of the storm. S was estimated at 0.5° × 1° grid of 35.5° × 47° part of a sphere. Coordinates of the region are 47.75°–83.25°N and 0.5°W–46.5°E. This set up, in particular, means that S was computed at M = 72 × 48 = 3,456 grid points and N = 2,880 time instants. Note that the same event, region and grid were considered in our recent study (Marshalko et al., 2021).

Details are in the caption following the image

Location of sites from the International Monitor for Auroral Geomagnetic Effect magnetometer network. Credit: Finnish Meteorological Institute.

The PCA of Sm(t) is performed in a similar manner as it was done, for example, in Alken et al. (2017), Egbert et al. (2021), and Zenhausern et al. (2021). Specifically, we construct a matrix F as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0023(16)
where urn:x-wiley:15427390:media:swe21301:swe21301-math-0024 is Sm(t) estimated at the nth time instant at the mth grid point. Further, according to the PCA concept, we form an M × M covariance matrix R
urn:x-wiley:15427390:media:swe21301:swe21301-math-0025(17)
and apply an eigenvalue decomposition to R
urn:x-wiley:15427390:media:swe21301:swe21301-math-0026(18)
where Λ is a diagonal matrix containing the eigenvalues λi, i = 1, 2, …, M of R. The urn:x-wiley:15427390:media:swe21301:swe21301-math-0144 at M grid points is represented by ith column vector of V which is in its turn the eigenvector of R corresponding to the eigenvalue λi. Both V and Λ are matrices of the size M × M. The superscript T in Equation 17 denotes the transpose. Eigenvalues give the respective PC's variance contribution. The corresponding time series ci are calculated as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0027(19)
PC are usually sorted in order from the largest to the smallest eigenvalues. The PC corresponding to the largest eigenvalue will explain the most variance, followed by the second, third PC, etc… In practice, the PC corresponding to a few of the largest eigenvalues explain most of the analyzed fields' variance. The cumulative variance of L PC can be calculated as (Alken et al., 2017)
urn:x-wiley:15427390:media:swe21301:swe21301-math-0028(20)

Figure 2 presents the cumulative variance for the first 30 PC. Horizontal dashed line allows us to estimate the number of modes needed to explain 99% of the spatial variability of Sm(t). It is seen from the figure that one needs L = 21 PC to explain most (99%) of the variance. This is a dramatic reduction from the original M = 3,456 SECS poles. These 21 PC will be used in the further discussion of the real-time calculation of the GEF. Figure 3 shows ji corresponding to PC of different i, illustrating the fact that the modes with larger i capture smaller spatial structures of the source. The respective time series ci are presented in Figure 4. Figure 5 compares the maps of the original and the PCA-based sources (i.e., external equivalent currents) for two snapshots of the enhanced geomagnetic activity. The original source is built using the SECS method (Equation 14), whereas PCA-based source is calculated using Equations 13 and 15. It is seen that the agreement between the original and PCA-based sources is very good both in terms of the amplitude and spatial pattern. In addition, Figure 6 demonstrates the comparison of the time series of these sources for two exemplary sites (shown in Figure 5 as white circles): one is located in the region where the significant source current is observed (Jäckvik (JCK)), another—aside from this region (Tartu (TAR)). Again, we observe good agreement between the two sources, especially for the site above which the source current is large.

Details are in the caption following the image

Cumulative variance for the first 30 Principal Components. Red dashed line marks the 99% threshold.

Details are in the caption following the image

A selection of PCA-recovered ji, i = 1, 7, 14, 21. By color and arrows, the magnitude (in A/m) and direction of the corresponding ji are depicted. See details in the text.

Details are in the caption following the image

A selection of PCA-recovered ci, i = 1, 7, 14, 21. See details in the text.

Details are in the caption following the image

Left: the original external equivalent current. Right: the external equivalent current calculated using 21 spatial modes. The results (in A/m) are for two time instants: 23:16:00 (top row) and 23:52:00 (bottom row) UT on 7 September 2017.

Details are in the caption following the image

Time series of the original external equivalent current (black curves) and external equivalent current calculated using 15 (blue curves) and 21 spatial modes (red curves) above two exemplary sites (Jäckvik (JCK) and Tartu (TAR)). The results are in A/m. Left and right panels show x- and y-components of the currents, respectively. Note different scales in the panels. Locations of the sites are shown in Figure 5 as white circles.

3.2 3-D Conductivity Model of Fennoscandia

We took the 3-D conductivity model of the region from Marshalko et al. (2021), where it was constructed using the SMAP (Korja et al., 2002)—a set of maps of crustal conductances (vertically integrated electrical conductivities) of the Fennoscandian Shield, surrounding seas, and continental areas. The SMAP consists of six layers of laterally variable conductance. Each layer has a thickness of 10 km. The first layer comprises contributions from the seawater, sediments, and upper crust. The other five layers describe conductivity distribution in the middle and lower crust. SMAP covers an area 0°–50°E and 50°–85°N and has 5′ × 5′ resolution. We converted the original SMAP database into a Cartesian 3-D conductivity model of Fennoscandia with three layers of laterally variable conductivity of 10, 20, and 30 km thicknesses (Figures 7a–7c). This vertical discretization is chosen to be compatible with that previously used by Rosenqvist and Hall (2019), Dimmock et al. (2019), and Dimmock et al. (2020) for GIC studies in the region. Conductivities in the second and the third layer of this model are simple averages of the conductivities in the corresponding layers of the original conductivity model with six layers. To obtain the conductivities in Cartesian coordinates, we applied the transverse Mercator map projection (latitude and longitude of the true origin are 50°N and 25°E, correspondingly) to the original data, and then performed the interpolation to a laterally regular grid. Note that a similar procedure was invoked to convert ji from spherical to Cartesian coordinates. The lateral discretization and the size of the resulting 3-D part of the conductivity model of Fennoscandia were taken as 5 × 5 km2 and 2,550 × 2,550 km2, respectively. Deeper than 60 km, we used the 1-D conductivity profile obtained by Kuvshinov et al. (2021) (Figure 7d), which is an updated version of the 1-D profile from Grayver et al. (2017).

Details are in the caption following the image

Conductivity distribution [S/m] in the model of Fennoscandia: (a–c) Plane view on three layers of the 3-D part of the model; (d) global 1-D conductivity profile from Kuvshinov et al. (2021) used in this study. Locations of geomagnetic observatories Abisko (ABK), Uppsala (UPS), and Saint Petersburg (SPG) are marked with circles in plot (a).

Note that the lateral discretization and the size of the conductivity model of Fennoscandia imply that the GEF is calculated at a grid comprising Ng = 512 × 512 points.

3.3 Computation of Ei(rs, ω; σ)

To implement the real-time modeling concept one needs—as is seen from Equations B13 and C2—to compute Ei(rs, ω; σ) at a number of frequencies, or, in other words, to solve Maxwell's equations (Equations 6 and 7). These equations are numerically solved using the 3-D EM forward modeling code PGIEM2G (Kruglyakov & Kuvshinov, 2018), which is based on a method of volume integral equations with a contracting kernel (Pankratov & Kuvshinov, 2016). PGIEM2G exploits a piece-wise polynomial basis; in this study, PGIEM2G was run using the first-order polynomials in lateral directions and third-order polynomials in the vertical direction.

Figures 8-10 demonstrate Ei(rs, ω; σ) at locations of observatories Abisko (ABK), Uppsala (UPS), and Saint Petersburg (SPG), respectively. The results are for the excitations corresponding to the first, seventh, fourteenth and twenty-first SM and are shown for the frequency range from 10−5 to 1 Hz. From these figures, a few observations can be made. The behavior of Ei (with respect to frequency) varies with location and mode. Real and imaginary parts of Ei are comparable in magnitude. As expected, Ei are smooth functions with respect to the frequency; apparent non-smoothness of the results in some plots is due to the fact that absolute values of real and imaginary parts are shown.

Details are in the caption following the image

From left to right: absolute values of the real part, the imaginary part and the magnitude of Ei(rs, ω; σ) with respect to frequency, and for a number of spatial modes. Results are for observatory Abisko (ABK) located near the seashore (Figure 7a). Top and bottom rows show the results for Ex,i and Ey,i components (in mV/km), respectively.

Details are in the caption following the image

The same caption as in Figure 8 but for inland, Uppsala (UPS), geomagnetic observatory.

Details are in the caption following the image

The same caption as in Figure 8 but for Saint Petersburg (SPG) geomagnetic observatory.

Finally, it is important to note that Ei decrease—irrespective of the mode and location—as frequency decreases; specifically, the magnitude of Ei drops down more than two orders of magnitude as frequency decreases from 1 Hz down to 10−3 Hz. These plots suggest a value for T in Equation 10; recall, that useful rule of thumb is that the value for T corresponds to the inverse of frequency at which the field becomes small compared to the higher frequencies. Following this rule, T = 1,000 s seems to be a reasonable choice which will be further justified in the next section.

3.4 Model Study to Justify a Value for T

To justify the value for T we first calculate the reference (“true”) time-domain electric field for the chosen 8-hr event. This reference field was computed using the procedure presented in Ivannikova et al. (2018) and Marshalko et al. (20202021). Specifically, we calculate jext(t, r) using Equations 13 and 15 and taking 21 terms in Equation 13. Further, according to Marshalko et al. (2021), we calculate the reference electric field as follows:
  1. The source jext(t, r) is transformed from the time to the frequency domain with a fast Fourier transform (FFT).

  2. Frequency domain Maxwell's equations 1 and 2 are numerically solved using PGIEM2G at FFT frequencies between urn:x-wiley:15427390:media:swe21301:swe21301-math-0029 and urn:x-wiley:15427390:media:swe21301:swe21301-math-0030 where K is the length of the event, and Δt is the sampling rate of the considered time series. In this study, Δt is 10 s, and K is 8 hr.

  3. E(t, r) is obtained with an inverse FFT of the frequency domain field.

Then we calculate electric fields using Equation 11 with T = 900 s (15 min) and with T = 3,600 s (1 hr) and compare them with the reference field. Figures 11-13 show comparison of electric field time series, again, at locations of ABK, UPS, and SPG observatories. It is seen that both “real-time” (either calculated using T = 15 min or T = 1 hr) electric fields agree well with the reference electric field. Tables 1 and 2 confirm this quantitatively by presenting correlation coefficients between corresponding time series and the normalized root-mean-square errors; the latter are defined as
urn:x-wiley:15427390:media:swe21301:swe21301-math-0031(21)
where a and b are the GEF time series calculated exploiting real-time scheme and the reference GEF time series, respectively, ai and bi are elements of these time series, and N is the number of time instants. Since results for T = 15 min and T = 1 hr appear to be very similar, we present in the next section the estimates of computational loads for the case when T is taken as 15 min.
Details are in the caption following the image

Electric field components at the Abisko (ABK) geomagnetic observatory location calculated using 3-D electromagnetic modeling with 21 spatial modes for the whole 8 hr time interval (from 20:00:00 UT, 7 September 2017, to 03:59:50 UT, 8 September 2017) (red curves) and electric field components at the same observatory calculated using real-time 3-D ground electric field modeling approach with T = 15 min (blue curves) and T = 1 hr (green curves). The results are in mV/km.

Details are in the caption following the image

The same caption as in Figure 11 but for the Uppsala (UPS) geomagnetic observatory.

Details are in the caption following the image

The same caption as in Figure 11 but for the Saint Petersburg (SPG) geomagnetic observatory.

Table 1. Correlation Coefficients Between Reference Ground Electric Field (GEF) Components and GEF Components Calculated Using Real-Time 3-D GEF Modeling Approach With T = 15 min and T = 1 hr at Abisko (ABK), Uppsala (UPS), and Saint Petersburg (SPG) Geomagnetic Observatories
ABK UPS SPG
2017/09/07 20:00:00–2017/09/08 03:59:50
corr(Ex,15 min, Ex,ref) 0.984 0.991 0.989
corr(Ex,1 hr, Ex,ref) 0.984 0.995 0.995
corr(Ey,15 min, Ey,ref) 0.985 0.993 0.983
corr(Ey,1 hr, Ey,ref) 0.979 0.997 0.992
2015/03/17 00:00:00–2015/03/18 23:59:50
corr(Ex,15 min, Ex,ref) 0.986 0.992 0.988
corr(Ex,1 hr, Ex,ref) 0.986 0.996 0.995
corr(Ey,15 min, Ey,ref) 0.984 0.993 0.983
corr(Ey,1 hr, Ey,ref) 0.980 0.997 0.992
2003/10/29 00:00:00–2003/10/31 23:59:50
corr(Ex,15 min, Ex,ref) 0.983 0.991 0.989
corr(Ex,1 hr, Ex,ref) 0.984 0.994 0.994
corr(Ey,15 min, Ey,ref) 0.986 0.995 0.989
corr(Ey,1 hr, Ey,ref) 0.985 0.997 0.994
  • Note. The results are shown for three time intervals: from 20:00:00 UT, 7 September 2017, to 03:59:50 UT, 8 September 2017; from 00:00:00 UT, 17 March 2015, to 23:59:50 UT, 18 March 2015; from 00:00:00 UT, 29 October 2003, to 23:59:50 UT, 31 October 2003.
Table 2. Normalized Root Mean Square Errors Calculated Based on the Reference Ground Electric Field (GEF) Components and GEF Components Calculated Using Real-Time 3-D GEF Modeling Approach With T = 15 min and T = 1 hr at Abisko (ABK), Uppsala (UPS), and Saint Petersburg (SPG) Geomagnetic Observatories
ABK UPS SPG
2017/09/07 20:00:00–2017/09/08 03:59:50
nRMSE(Ex,15 min, Ex,ref) 0.237 0.167 0.181
nRMSE(Ex,1 hr, Ex,ref) 0.233 0.128 0.128
nRMSE(Ey,15 min, Ey,ref) 0.227 0.147 0.228
nRMSE(Ey,1 hr, Ey,ref) 0.238 0.112 0.161
2015/03/17 00:00:00–2015/03/18 23:59:50
nRMSE(Ex,15min, Ex,ref) 0.217 0.157 0.179
nRMSE(Ex,1 hr, Ex,ref) 0.211 0.122 0.122
nRMSE(Ey,15 min, Ey,ref) 0.232 0.143 0.214
nRMSE(Ey,1 hr, Ey,ref) 0.233 0.112 0.158
2003/10/29 00:00:00–2003/10/31 23:59:50
nRMSE(Ex,15 min, Ex,ref) 0.231 0.164 0.175
nRMSE(Ex,1 hr, Ex,ref) 0.224 0.136 0.128
nRMSE(Ey,15 min, Ey,ref) 0.215 0.130 0.188
nRMSE(Ey,1 hr, Ey,ref) 0.213 0.114 0.145
  • Note. The results are shown for three time intervals: from 20:00:00 UT, 7 September 2017, to 03:59:50 UT, 8 September 2017; from 00:00:00 UT, 17 March 2015, to 23:59:50 UT, 18 March 2015; from 00:00:00 UT, 29 October 2003, to 23:59:50 UT, 31 October 2003.

3.5 Computational Loads for the Real-Time GEF Calculation

Once urn:x-wiley:15427390:media:swe21301:swe21301-math-0032 and urn:x-wiley:15427390:media:swe21301:swe21301-math-0033 are computed and stored on the disc, GEF at a grid urn:x-wiley:15427390:media:swe21301:swe21301-math-0145  and time instant tk is computed using Equation 11. In accordance with this equation, the GEF calculation requires forecasting/nowcasting the L × Nt array c, reading the L × Nt × Ng array urn:x-wiley:15427390:media:swe21301:swe21301-math-0034 and L × Ng array urn:x-wiley:15427390:media:swe21301:swe21301-math-0035, and performing urn:x-wiley:15427390:media:swe21301:swe21301-math-0036 summations and multiplications. For our problem setup with Ng = 512 × 512, Nt = 90 and L = 21, the calculation of E(rs, tk; σ) takes from 0.00625 to 0.025 s, depending on the computational environment. Note that to store arrays for this setup one needs 7.25 Gigabytes of disc space.

4 Discussion

4.1 Further Justification of the Concept

So far, we have demonstrated the concept's validity on an example of a single space weather event. However, one can argue that ji(r) obtained for a specific event could be non-adequate for other events. To address this question, we performed the following modeling experiment. First, we built the “sources” (i.e., external equivalent currents) for two other space weather events—St. Patrick's Day geomagnetic storm on 17–18 March 2015 and Halloween storm on 29–31 October 2003—and then approximated corresponding sources using ji(r) obtained for the 7–8 September 2017 storm (Equation 13). Note that for each new event, ci(t) in Equation 13 are calculated using Equation 19, where urn:x-wiley:15427390:media:swe21301:swe21301-math-0146 are taken from the PCA of 7–8 September 2017 data, but Sm(t) are formed using new event data. To ensure that the spatiotemporal structure of the source for new events is different from that of the (reference) 7–8 September 2017 event, we took the new events' lengths as 48 and 72 hr, respectively; recall that the duration of 7–8 September 2017 event was taken as 8 hr. Figure 14 shows snapshots of the original and SM-based sources for St. Patrick's Day (top panels) and Halloween (bottom panels) geomagnetic storms. It is seen from the figure that the SM-based source (with SM obtained from another event) approximates very well the source of the other two events. Figure 15 confirms this inference by showing the agreement between the time series of the original and SM-based sources above exemplary site JCK, again, for Halloween (top panels) and St. Patrick's Day (bottom panels) storms. These results suggest that irrespective of the event (which corresponds to sources of different geometry), the spatial structure of these sources is well approximated by a finite number of SM obtained from the analysis of some specific event. The prerequisite to getting adequate set of SM is that the event to be used for SM estimation should be long enough and sufficiently energetically large and spatially complex.

Details are in the caption following the image

Left: the original external equivalent current. Right: the external equivalent current calculated using 21 spatial modes. The results are for two time instants: 23:45:40 UT on 17 March 2015 (top panels) and 20:08:20 UT on 30 October 2003 (bottom panels). Note that Jäckvik (JCK) site became a part of the International Monitor for Auroral Geomagnetic Effect network on 1 September 2010. Thus, its data were not used for the equivalent current calculation in case of the 29–31 October 2003 geomagnetic storm. The results are in A/m.

Details are in the caption following the image

Time series of the original external equivalent current (black curves) and external equivalent current calculated using 21 spatial modes (red curves) above Jäckvik magnetometer (JCK) for two time intervals: from 00:00:00 UT, 17 March 2015, to 23:59:50 UT, 18 March 2015 and from 00:00:00 UT, 29 October 2003, to 23:59:50 UT, 31 October 2003. The location of JCK is shown in Figure 14 as a white circle. Note that JCK magnetometer became a part of the International Monitor for Auroral Geomagnetic Effect network on 1 September 2010. Thus, its data were not used for the equivalent current calculation in case of the 29–31 October 2003 geomagnetic storm. The results are in A/m.

The linked question we also address is whether T = 15 min is a valid choice for the real-time modeling of the GEF during the above-discussed events. As in Section 3.4, we calculate electric fields using Equation 11 with T = 15 min and with T = 1 hr and compare them with the reference fields. Two top panels in Figures 16 and 17 show the comparison of electric field time series at the location of the ABK observatory for Halloween and St. Patrick's Day events, respectively. Similar to the 7–8 September 2017 event results, both “real-time” electric fields agree well with the reference electric field. Besides, bottom panels in Figures 16 and 17 show the difference between “real-time” electric fields calculated using T = 15 min and T = 1 hr. It is seen that this difference is small compared to signals themselves (cf., top panels in the same figures), once again confirming the fact that T = 15 min can be used for the real-time GEF modeling.

Details are in the caption following the image

The same caption as in Figure 11 but for a 48 hr time interval from 00:00:00 UT, 17 March 2015, to 23:59:50 UT, 18 March 2015. The bottom panel demonstrates absolute differences between ground electric field (GEF) components calculated using real-time 3-D GEF modeling approach with T = 15 min and T = 1 hr.

Details are in the caption following the image

The same caption as in Figure 11 but for a 72 hr time interval from 00:00:00 UT, 29 October 2003, to 23:59:50 UT, 31 October 2003. The bottom panel demonstrates absolute differences between ground electric field (GEF) components calculated using real-time 3-D GEF modeling approach with T = 15 min and T = 1 hr.

Tables 1 and 2 quantify the agreement for these two events at three geomagnetic observatories, as it was already done for 7–8 September 2017 event. It is seen that the agreement between results for two new events is as good as for the 7–8 September 2017 geomagnetic storm. Notably, throughout all three events, the correlation coefficient is lower, and nRMSE is larger for ABK than for UPS and SPG. This is probably because ABK is located in the region with a large lateral contrast of conductivity (Figure 7), where modeling results are expectedly less accurate.

4.2 Nowcasting and Forecasting GEF Using the Proposed Concept

In this section, we discuss how the proposed concept could be implemented for nowcasting/forecasting of the GEF. Specifically, a scheme to nowcast GEF could work as follows:
  1. Using magnetic field data collected at an observational network for historical (past) event/several events one obtains urn:x-wiley:15427390:media:swe21301:swe21301-math-0151 at rm. This is done by exploiting the procedure described in Section 3.1. These urn:x-wiley:15427390:media:swe21301:swe21301-math-0147 allow us to represent the source at any time instant t and at any position r via Equations 13-15. In this paper, we used IMAGE network of magnetic field data to obtain L = 21 urn:x-wiley:15427390:media:swe21301:swe21301-math-0148 and further ji(r). Using IMAGE data, we confine ourselves to Scandinavian region. If Canada, for example, is a region of interest, one would use the data from the Canadian networks of magnetic field observations, like CARISMA (Mann et al., 2008) and AUTUMNX (Connors et al., 2016).

  2. Once urn:x-wiley:15427390:media:swe21301:swe21301-math-0149 and, subsequently, ji(r) are obtained and stored, one estimates electric field at the current time instant, tk, using Equation 11. This, in particular, requires knowledge of coefficients ci at time instant tk and at a number of time instants in the past, tk − Δt, tk − 2Δt, …, tk − NtΔt. The coefficients at these instants are obtained by reusing Equations 16 and 19, namely.

urn:x-wiley:15427390:media:swe21301:swe21301-math-0037(22)
where Sm are computed from the available ground magnetic field data. Note that NtΔt = T, where T is either 15 min or 1 hr in our example. It is also important to stress that to model the GEF at the next time instant, tk + Δt, one needs to update only ci(tk + Δt).
As for forecasting of the GEF, the scheme could include the following steps:
  1. One obtains urn:x-wiley:15427390:media:swe21301:swe21301-math-0086 and, subsequently, ji(r) in a similar manner as it is done in case of the GEF nowcasting.

  2. One trains the neural network (NN) using as input data the time series of solar wind parameters collected by satellite(s) at the L1 Lagrangian point and ci(t) as output data. Time series ci(t) for the training period are obtained using Equations 16 and 19. There is a common understanding that the longer time series are used for the training phase, the better the quality of the forecasted results. Therefore, this period preferably should include multiple years of the L1 and ground magnetic field data; recall that ci(t) during the training phase are obtained from the ground magnetic field data.

  3. One forecasts GEF using the trained NN. Ideally, one has to forecast well ahead. However, given observations made at the L1 point, a geomagnetic disturbance is seen on the ground as fast as an hour ahead. This time latency can be further shrunk to half an hour or so, depending on the solar wind speed. This, in particular, advocates real-time modeling of the GEF which is a topic of this paper.

5 Conclusions

In this paper, we presented a formalism for real-time computation of the GEF in a given 3-D Earth's conductivity model excited by a continuously augmented spatially and temporally varying source responsible for a space weather event.

The formalism relies on the factorization of the source by SM and time series of respective expansion coefficients, and exploits precomputed GEF kernels generated by corresponding SM.

To validate the formalism, we invoked a high-resolution 3-D conductivity model of Fennoscandia and considered a realistic source built with the use of the SECS method as applied to magnetic field data from the IMAGE network of observations. Factorization of the SECS-recovered source is then performed using the PCA. Eventually, we show that the GEF computation at a given time instant on a 512 × 512 grid requires at most 0.025 s provided that GEF kernels due to the pre-selected SM are computed in advance. This opens a practical opportunity for GEF nowcasting, using ground magnetic field data, or even forecasting, using both ground magnetic field and L1 data.

We illustrate the concept on a Cartesian geometry problem setup. Global-scale implementation is rather straightforward; for this scenario, the source could be obtained either using magnetic field data from the global network of geomagnetic observatories or exploiting the results of the first-principle modeling of the global magnetosphere-ionosphere system.

Acknowledgments

M. Kruglyakov was supported by the New Zealand Ministry of Business, Innovation, and Employment through Endeavour Fund Research Programme contract UOOX2002. A. Kuvshinov was supported in the framework of Swarm DISC activities, funded by ESA contract no. 4000109587, with the support from EO Science for Society. E. Marshalko was supported by Grant 21-77-30010 from the Russian Science Foundation. The authors thank the institutes that maintain the IMAGE Magnetometer Array: Tromsø Geophysical Observatory of UiT, the Arctic University of Norway (Norway), Finnish Meteorological Institute (Finland), Institute of Geophysics Polish Academy of Sciences (Poland), GFZ German Research Center for Geosciences (Germany), Geological Survey of Sweden (Sweden), Swedish Institute of Space Physics (Sweden), Sodankylä Geophysical Observatory of the University of Oulu (Finland), and Polar Geophysical Institute (Russia). The authors thank INTERMAGNET (www.intermagnet.org) for promoting high standards of magnetic observatory practice.

    Appendix A: Properties of Transfer Functions and Impulse Responses

    The convolution integrals in Equation 9 represent the response of the medium to a time-varying extraneous current. These relations follow from the properties of a physical system we consider. We list these properties below and discuss implications. The presentation closely follows a more detailed analysis by Svetov (1991). Note that for the sake of clarity, we discuss the properties on an example of abstract scalar quantities and omit their dependence on spatial variables and electrical conductivity pertinent to our application.
    • 1.

      Linearity allows us to define the response, ζ(t), of the medium at time t to an extraneous forcing as

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0038(A1)
    • where χ is the extraneous forcing that depends on time t′ and urn:x-wiley:15427390:media:swe21301:swe21301-math-0039 is the medium Green's function.

    • 2.

      Stationarity implies that the response of the medium does not depend on the time of occurrence of the excitation. In this case urn:x-wiley:15427390:media:swe21301:swe21301-math-0040 and Equation A1 is rewritten as a convolution integral

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0041(A2)
    • where f(t) represents the impulse response of the medium. In the frequency domain, the convolution integral degenerates to

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0042(A3)
    • where urn:x-wiley:15427390:media:swe21301:swe21301-math-0043 is called the transfer function and we use tilde sign urn:x-wiley:15427390:media:swe21301:swe21301-math-0044 to denote complex-valued quantities. Equations A2 and A3 are related through the Fourier transform

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0045(A4)
    • 3.

      Since we work in the time domain with a real-valued forcing, the impulse response is also real. To see implications of this, let us define the inverse Fourier transform of urn:x-wiley:15427390:media:swe21301:swe21301-math-0046 as

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0047(A5)
    • For an impulse response to be real, the last term in the integral (Equation A5) has to vanish. This is possible only if fR(ω) and fI(ω) are even and odd functions of ω, respectively. Therefore, Equation A5 reduces to

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0048(A6)
    • 4.

      Impulse response is causal. This property implies that

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0049(A7)
    • Under this assumption, the convolution integral (Equation A2) is recast to

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0050(A8)
    • Also, due to causality (Equation A7), and exploiting Equation A6, one can write for t < 0

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0051(A9)
    • Further, using the fact that cos(ωt) and sin(ωt) are odd and even functions with respect of t, one obtains for t > 0

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0052(A10)
    • Using the latter equation and Equation A6 one can state that the impulse response is determined by using either only real or imaginary part of urn:x-wiley:15427390:media:swe21301:swe21301-math-0053:

    urn:x-wiley:15427390:media:swe21301:swe21301-math-0054(A11)

    Appendix B: Details of the Numerical Computation of the Real-Time GEF

    As discussed in the main text, to calculate the GEF in near-real time one needs to efficiently estimate integrals in the right-hand side (RHS) of the equation below
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0055(B1)
    It is important to stress that with finite T, one must account for a possibly substantial linear trend in time series ci(t). By removing the trend, we are forced to work with the following function
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0056(B2)
    Substituting Equation B2 into the RHS of Equation B1, and considering (for simplicity) only one term in the sum, we obtain
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0057(B3)
    Recall that T should be taken large enough to make approximation (Equation B1) valid; particularly, this means that
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0058(B4)
    But the integral in the RHS of the latter equation is zero since it corresponds to the electric field generated by the time-constant source. Then, Equation B3 can be approximated as
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0059(B5)
    where
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0060(B6)

    The integrals urn:x-wiley:15427390:media:swe21301:swe21301-math-0061 can be computed using the digital filter technique (see Appendix C), whereas first term in the RHS of Equation B5 is estimated as follows.

    Taking into account that we have ci(t) at discrete time instants, t = nΔt, n = 0, 1, … , we approximate di(t, τ; T) using the Whittaker-Shannon (sinc) interpolation formula
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0062(B7)
    where
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0063(B8)
    Recall that sinc interpolation is a method to construct a continuous band-limited function from a sequence of real numbers, in our case time series di at time instants t = nΔt, n = 0, 1, …. Note that in our context, the term “band-limited function” means that non-zero values of a Fourier transform of this function are confined to the frequencies
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0064(B9)
    Using the approximation (Equation B7) and taking into account that Ei(rs, τ; σ) = 0, τ < 0 (Appendix A), one obtains
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0065(B10)
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0066
    Thus, we can write
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0067(B11)
    where
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0068(B12)
    Further, following the properties of the Fourier transform as applied to sinc function, we obtain that
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0069(B13)
    Finally, substituting Equation B11 in Equation B5, and Equation B5 in the RHS of Equation B1 we obtain Equation 11
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0070
    where di(tk, nΔt; T), urn:x-wiley:15427390:media:swe21301:swe21301-math-0071, and urn:x-wiley:15427390:media:swe21301:swe21301-math-0072 are defined in Equations B2B6, and B13, respectively. Note that the estimation of the integral in the RHS of Equation B13 is performed using a suitable quadrature formula.

    An important note here is that, according to Equation B13, one does not need to compute Ei(rs, ω; σ) for urn:x-wiley:15427390:media:swe21301:swe21301-math-0073. This may be obvious, however, this is not the case if one uses piece-wise constant (PWC) approximation of ci(t) as it is done, for example, in Grayver et al. (2021). With PWC approximation, one is forced to compute the fields at very high frequencies irrespective of Δt value; this can pose a problem from the numerical point of view.

    Appendix C: Computation of urn:x-wiley:15427390:media:swe21301:swe21301-math-0074

    With the use of Equation A11, Ei(rs, τ; σ) can be written as
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0075(C1)
    Substituting the latter equation into Equation B6 and rearranging the order of integration, we write urn:x-wiley:15427390:media:swe21301:swe21301-math-0076 in the following form
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0077(C2)
    where Φ(ωT) reads
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0078(C3)
    Integrals in Equation C2 can be efficiently estimated using the digital filter technique. Specifically, one needs to construct a digital filter for the following integral transform
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0079(C4)
    To obtain filter's coefficients for this transform, we exploit the same procedure as in Werthmüller et al. (2019) using the following pair of output and input functions
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0080(C5)

    Appendix D: Formulas for P and Q

    The formulas for P(r, rm) and Q(r, rm) (in slightly different notations) are taken from Vanhamäki and Juusola (2020) (see their Sections 2.3 and 2.5) and are as follows
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0081(D1)
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0082(D2)
    where R = a + h, r = (R, ϑ, φ), rm = (R, ϑm, φm) and γ is an angle between r and rm; γ can be determined from the following spherical trigonometry formula
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0083(D3)
    and cos C and sin C are given as
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0084(D4)
    urn:x-wiley:15427390:media:swe21301:swe21301-math-0085(D5)

    From Equations D1 and D2, it is seen that P(r, rm) and Q(r, rm) tend to infinity as r tends to rm. The simplest way to deal with this issue is, as mentioned in Vanhamäki and Juusola (2020), to consider the grids for r and rm that are shifted with respect to each other. This approach is used in the current paper.

    Data Availability Statement

    The SMAP model is available via the EPOS portal (http://mt.bgs.ac.uk/EPOSMT/2019/MOD/EPOSMT2019_3D.mod.json).