Regional Ionospheric Parameter Estimation by Assimilating the LSTM Trained Results Into the SAMI2 Model

This paper presents a study on the possibility of predicting the regional ionosphere at midlatitude by assimilating the predicted ionospheric parameters from a neural network (NN) model into the Sami2 is Another Model of the Ionosphere (SAMI2). The NN model was constructed from the data set of Jeju ionosonde (33.43°N, 126.30°E) for the period of 1 January 2011 to 31 December 2015 by using the long‐short term memory (LSTM) algorithm. The NN model provides 24‐hr prediction of the peak density (NmF2) and peak height (hmF2) of the F2 layer over Jeju. The predicted NmF2 and hmF2 were used to compute two ionospheric drivers (total ion density and effective neutral meridional wind), which were assimilated into the SAMI2 model. The SAMI2‐LSTM model estimates the ionospheric conditions over the midlatitude region around Jeju on the same geomagnetic meridional plane. We evaluate the performance of the SAMI2‐LSTM by comparing predicted NmF2 and hmF2 values with measured values during the geomagnetic quiet and storm periods. The root‐mean‐square error values of NmF2 (hmF2) from Jeju ionosonde measurements are lower by 45% and 45% (30% and 11%) than those of the SAMI2 and IRI‐2016 models during the geomagnetic quiet periods. However, during the geomagnetic storm periods, the performance of the SAMI2‐LSTM model does not predict positive geomagnetic storms well. Comparing the quiet and storm periods for the SAMI2‐LSTM model, the root‐mean‐square error (RMSE) of the storm period was calculated to be 2.76 (3.2) times higher at Jeju (Icheon) than in the quiet period. From these results, we demonstrated that in this study, the combination of the NN‐LSTM model and physics‐based model could improve the ionosphere prediction of existing theoretical and empirical models for midlatitude regions, at least in geomagnetically quiet conditions. We strongly suggest that this attempt, which has not been reported before, could be used as one of the keys to advance the physics‐based model further.


Introduction
Ionospheric changes have a strong influence on GPS signal propagation, telecommunications, satellite communication, shortwave communication, geodetic, and traffic information, all of which are of high importance in our modern society. As such, the importance of monitoring and predicting the ionospheric environment is increasing. To accurately capture changes in the ionosphere, it is necessary to continuously observe and monitor its characteristics. Since the F2 layer has a major contribution to electron content in the ionosphere, and its maximum density (NmF2) and height (hmF2) can represent the ionospheric characteristics. If we observe and monitor the variation in these parameters, we may estimate the ionospheric impact on the transionospheric radio applications. For this reason, ionospheric conditions, including the F2 layer, have been monitored with various observation equipment, such as worldwide ionosonde networks (e.g., Galkin et al., 2006), incoherent scatter radars (e.g., Gordon, 1958), topside sounding (e.g., Chapman & Warren, 1968), and radio occultation from satellites (e.g., Hajj et al., 1994;Hardy et al., 1994). However, these observations not only have limitations in terms of time and spatial resolutions but also cannot predict future conditions. To overcome these shortcomings, many ionospheric models have been developed and studied.
Data assimilation techniques, such as Kalman filter algorithms (e.g., Chartier et al., 2016;Lee et al., 2012;Lin et al., 2015;Scherliess et al., 2004;Yue et al., 2011Yue et al., , 2012, 3D-Var (e.g., Aa et al., 2016;Bust et al., 2001Bust et al., , 2004Bust et al., , 2007, and 4D-Var techniques (e.g., Pi et al., 2003;Ssessanga et al., 2019;Wang et al., 2004), have a considerable advantage in improving the performance of ionospheric models. In particular, since the data assimilation is based on observations, it can be a useful tool for nowcasting, which can simulate the current state of the ionosphere. However, it is still difficult to predict the future state of the ionosphere with a data assimilation model. In our previous study , to simulate and predict the ionospheric state better, we devised a simple data assimilation method, which was based on studies by Richards (1991) and Dandenault (2018). In this method, we estimated two ionospheric drivers (the neutral meridional wind and the total ion density) from hmF2 and NmF2 parameters from ionosonde measurements, and then, we utilized them in a first principle physic model, the Sami2 is Another Model of the Ionosphere (SAMI2), to calculate the state of ionosphere about 15 min later. The SAMI2 code is a two-dimensional ionospheric model originally developed by Huba et al. (2000) and now is available as an open source. Kim et al. (2019) showed that their method improved significantly the short-term (~15 min) forecasts of NmF2 and hmF2 at midlatitude locations in the same meridional plane. Unfortunately, their data assimilation method is only useful for predicting short-term future conditions. It is because long-term predictions are needed for the point of view in space weather prediction and surveillance. Thus, other approaches and methods are required.
Recently, many studies have been conducted to predict the GPS total electron contents (TEC) map using artificial neural networks (ANNs) algorithm (e.g., Habarulema et al., 2007;Hernández-Pajares et al., 1997;Leandro & Santos, 2007;Maruyama, 2002;Tulunay et al., 2006;Zhukov et al., 2018). More advanced algorithms, collectively called as deep neural network (DNN), have been developed in many application fields, including a recurrent neural network (RNN) and long-short term memory (LSTM). The RNN technique can solve the problem of long-term prediction (Elman, 1990) but has a vanishing gradient problem because of the limited range of input values (Bengio et al., 1994;Habarulema et al., 2009). On the other hand, the LSTM technique has an advantage over RNN in scenarios where a long-term input value is required. The LSTM algorithm has recently been utilized in studies of predicting ionospheric TEC (e.g., Srivani et al., 2019;Sun et al., 2017). For the ionospheric F2 parameter prediction, numerous studies have been utilizing the ANN algorithm (e.g., Athieno et al., 2017;Fan et al., 2019;McKinnell & Poole, 2000;Nakamura et al., 2009;Poole & Poole, 2002;Williscroft & Poole, 1996;Wintoft & Cander, 2000), but only recent studies are beginning to adopt the LSTM algorithm. For example, Hu and Zhang (2018) attempted the hmF2 prediction using the LSTM and bi-LSTM algorithms. Most recently, Moon et al. (2020) conducted a study using the LSTM model to perform long-term prediction. They trained long-term data of Jeju ionosonde, performed predictions for the next 24 hr, and produced better prediction values than existing models.
In this study, to overcome the limitations of the short-term (15 min) prediction, we adopt the LSTM algorithm for the F2 parameter prediction. As a first step, we utilize the predictive values of NmF2 and hmF2 for a longer-term period (24 hr) using the LSTM algorithm in the previous study (Moon et al., 2020). We then estimate the ionospheric drivers (equivalent winds and electron density scale factors) from the predictive NmF2 and hmF2 for the next 24 hr. The ionospheric drivers then input to a first principle model, the SAMI2 to calculate the ionospheric states on the meridional plane for the 24 hr. In this way, the longer-term (24 hr) predictions of NmF2 and hmF2 can be made with a good level of accuracy, not only at a reference location where observation data are used but also at other locations that are nearly on the same meridional plane. The comparison of the predicted NmF2 and hmF2 with measured values are then made in the geomagnetic storm and quiet periods. We believe that this can be used as one of the keys to advance the physics-based model further because no such attempt (combining between deep-learning and physics-based model) has been reported before.
The paper consists of the following sections. In section 2, the design of the LSTM algorithm is explained and the detailed depiction of the data is provided for two cases of the geomagnetic activity. Section 3 introduces the assimilation version of the SAMI2 model. In section 4, the results of the assimilated SAMI2 model linked with the LSTM method are compared to those of the SAMI2 original and IRI-2016 models, with discussion on further improvement strategy. In section 5, conclusion and summary are given.

Design of LSTM
Since the RNN, which is in a larger category than LSTM, is known to work for the short-term prediction problem but not for the long-term dependency problem (Bengio et al., 1994;Habarulema et al., 2009), we decided to use the LSTM, which is more suitable for solving the long-term prediction problem. Basically, RNN consists of a repeating chain of neural networks (NNs), as does the LSTM. However, the LSTM has four unique interoperable structures rather than a single network layer. Figure 1 shows the repetitive structure of the conventional RNN and the unique structure of the LSTM. In Figure 1, the X t − 1 , X t , and X t+1 indicate the input values and the h t − 1 , h t , and h t+1 mean the values of the hidden layer. As shown in Figure 1, the RNN and LSTM algorithms differ in the A structures. The LSTM model adds a unique layer called the sigmoid layer (σ), which determines how much each component will affect. In other words, the sigmoid layer outputs a value of 0 or 1. It is responsible for deciding whether each component will be affected. The σ value of 0 means that a component does not affect future results. The σ value of 1 causes the data to flow, so that the component certainly affects future predictions. The RNN and LSTM models must use the hyperbolic tangent function (tanh), which is a nonlinear function, for the benefit of layering. More detailed description and discussion on the LSTM algorithm can be found in literature (e.g., Hochreiter & Schmidhuber, 1997;Moon et al., 2020).

Training and Validation Data Sets
In this study, we adopt the structure of LSTM made by Moon et al. (2020). Moon et al. (2020) designed the LSTM model using the Jeju (33.43°N, 126.30°E) ionosonde data ( foF2, hmF2) (https://spaceweather.rra. go.kr/observation/service/iono). In addition, they included the observed solar and geomagnetic indices such as the F10.7, sunspot number (SSN), and Kp index (https://omniweb.gsfc.nasa.gov/ow.html) because ionospheric parameters are strongly related to these indices. They trained the LSTM model with the data set for the period of 1 January 2011 to 31 December 2015 and validated it with the data set for the whole period in 2016.
To organize the training and validation data into 1-hr units, Moon et al. (2020) reconstructed the missing ionospheric data by filling in with the data observed in the same local time on the previous day. Since the time resolutions of the solar and geomagnetic indices are different, they used the running mean method of 81 days (2 days) for the sunspot number (ap index), and individual values for Kp and F10.7 itself in units of 1 hr. For the efficient performance of the algorithm, they set the ratio of the training and the validation data set to~8.3:1.7. Detailed specifications and numbers of data points for the data sets are summarized in Table 1. Moon et al. (2020) used the Deep learning Toolbox in Matlab-R2019b for training the LSTM model of Jeju ionospheric parameters. In the process, they conducted training verification using the collected training data to find the appropriate batch size, lookback, lookahead, and hidden layer count as the hyperparameters. Here, the batch size means the number of training examples utilized in one iteration, and the lookback size defines the number of recent data points to be used when predicting each future value in the time series. Conversely, the lookahead size means the number of data points to be predicted. For the foF2 LSTM model, they used 3 batch sizes, 24 lookback and lookahead sizes, and 21 hidden layers. For the hmF2 model, 24 batch sizes, 24 lookback and lookahead sizes, and 41 hidden layers were used. Therefore, in testing in this study, the lookback is 24, so past data from 24 hr ago to the present is used to predict values until the next 24 hr (lookahead = 24).

Test Sets
As shown in Table 1, we select the 3-day period of 6-8 September 2017 for geomagnetic storm periods and the 2-week period of 18-31 October 2018 for quiet periods to test the LSTM model. We chose these particular periods for the test because ionosonde data from all three comparing stations (Jeju, Icheon, and Okinawa) were available. Icheon (37.14°N, 127.54°E) and Okinawa (26.68°N, 128.15°E) are located near the same geomagnetic longitude as Jeju (33.43°N, 126.30°E), so that results from the assimilated SAMI2 model with the LSTM model can be compared directly. The Korean ionosonde data (Jeju and Icheon) are available from the Korean Space Weather Center homepage (https://spaceweather.rra.go.kr/observation/service/iono), and we used it at intervals of 7.5 min for tests. The Okinawa (26.68°N, 128.15°E) ionosonde data (30 min interval)

Space Weather
can be obtained from the website of the National Institute of Information and Communications (NICT) in Japan (http://wdc.nict.go.jp/IONO/HP2009/ISDJ/index-E.html). Figure 2 shows the geophysical condition data (IMF Bz component, SYM-H, Kp, F10.7, GOES X-ray data, and the observed NmF2 and hmF2 by Jeju ionosonde) during the storm test set. The geophysical condition data were obtained from the OMNI webpage mentioned in section 2.1, and the GOES data can be downloaded from NOAA NGDC (https://satdat.ngdc.noaa.gov/sem/goes/data/full/). As shown in Figure 2, 3 geomagnetic storms and 14 solar flares (>M class) occurred in succession during the storm test period. In particular, when the second storm occurred, the observed hmF2 and NmF2 values over Jeju station were abnormally increased. This is seen as a significant positive ionospheric effect due to the storm. On the other hand, although many solar flares occurred, there was no significant increase in electron density in the ionosphere over Jeju. Thus, we focus on the response to geomagnetic storms rather than the effects of solar flares. Figure 3 shows the geophysical conditions, GOES X-ray data, and the ionospheric parameters during the Figure 1. The system architecture of traditional recurrent neural network (top row) and LSTM (bottom row). X t − 1 , X t , and X t+1 (h t − 1 , h t , h t+1 ) are the input (hidden) values. A is the layering system, and σ is the sigmoid layer.

Assimilated SAMI2 Model
The trained LSTM model with the Jeju data calculates the 24-hr predicted values of hmF2 and NmF2, from which the ionospheric drivers (effective meridional winds and electron density scale factor) are computed. The ionospheric drivers are then assimilated into the revised SAMI2 model to calculate the ionospheric states on the meridional plane for the 24-hr period. The assimilation method for the revised SAMI2 model is described in Kim et al. (2019) and the references therein. The assimilation method is based on two assumptions: (1) the variation of hmF2 is linearly proportional to the variation of neutral meridional winds at the middle latitudes or electric fields (collectively called as equivalent meridional winds) and (2) all the ion density fields of the ionosphere on the meridional plane follow the same scaling factor between the model and measured NmF2 at the reference location.
Using the first assumption, Richards (1991) proposed the following simple Equation 1.

Space Weather
where α wind (t) is the proportionality constant at the time, t, and U is the equivalent meridional wind. Kim et al. (2019) computed α wind (t) for the SAMI2 model. Utilizing the computed α wind (t) in Equation 3, we can predict U at the next time step (~15 min) from the difference between the model and observed hmF2 values. The short term predicted U leads to short term predication of hmF2 fairly well, as demonstrated in Kim et al. (2019). In this study, we replace the term of hmF2 ionosonde with predicted values (hmF2 LSTM ) up to 24 hr by the trained LSTM model to achieve the longer-term prediction.
The second assumption is to adjust all uncertainty in the input parameters, such as neutral background density, solar extreme ultraviolet (EUV) flux, and transports of ions and other. The scale factor (α ion (t)) is derived from measured NmF2 at the reference location and then is applied to all the ion density fields, which are used as the initial ion densities in the SAMI2 model that runs to the next 15-min time step. The computed NmF2s at the next time step are reasonably close to the measured values at other locations, as demonstrated in Kim et al., 2019. In this study, the 24-hr predicted NmF2s, instead of measured NmF2 at the reference station, are used to compute the scale factor for the 24-hr period. Figure 4 shows the predicted values of ionospheric parameters over 24 hr from the LSTM model developed by Moon et al. (2020). We updated the ionospheric drivers using these long-term predicted values. We tested whether the long-term prediction was possible not only in the Jeju station but also in other regions by inputting these into the physics-based model. Also, we use the Flare Irradiance Spectral Model (FISM)  (Chamberlin et al., 2008) as the input energy source to the SAMI2 model. In addition, the Horizontal Wind Model (HWM14) (Drob et al., 2015) and U.S. Naval Research Laboratory Mass-Spectrometer-Incoherent-Scatter model (NRLMSISE-00) (Picone et al., 2002) are used as the thermosphere background model, and the E × B drift model (Scherliess & Fejer, 1999) is used as the basic electric field model.

Results and Discussions
To evaluate the performance of the assimilated SAMI2 model with the trained LSTM model (hereafter, SAMI2-LSTM), we compare the model NmF2's and hmF2's with measured values at the three ionosonde stations. The model performance is quantitatively evaluated with four measures: the correlation coefficient, root-mean-square error (RMSE), mean absolute percentage error (MAPE), and relative difference (RD). The correlation coefficient values are obtained using the REGRESS function of the IDL program, and the RMSE, MAPE, and RD are calculated using the following Equations 5-7, respectively. Smaller values for both RMSE and MAPE mean better performance.
The performance of the SAMI2-LSTM is then compared with those of the original SAMI2 modeland the widely used empirical model IRI-2016 (Bilitza et al., 2017). All the models calculate the prediction values at 15-min intervals during each test period. The SAMI2-LSTM model proposed by us uses an updated ionospheric driver. In contrast, the IRI-2016 model used the FORTRAN version provided on the web (irimodel.org/IRI-2016/), which does not update any ionospheric drivers. Thus, it may be an unfair comparison evaluation. However, the IRI-2016 model is an empirical model using global observation data, so we think it has a data assimilation effect. Also, since ionosonde data in the middle-low latitudes were used a lot, we believe it is similar to the purpose of using Jeju ionosonde in this study. Above all, since it is an international reference ionosphere model designated by COSPAR, which is widely used in ionosphere forecasting and surveillance in the globe, so it was utilized as a reference for comparison in this study.

Geomagnetically Quiet Case
First, we evaluate the performance of each model for 2 weeks from 18 October 2018 (doy = 291) to 31 October (doy = 304) when the geomagnetic activity was quiet. As described in section 2.2, since the SAMI2-LSTM model can calculate the ionospheric parameters at different locations on the longitudinal plane, we compare the results not only for Jeju station but also for other locations (Icheon and Okinawa). Figure 5 shows the observed and model values of NmF2 and hmF2 over each location. The solid black  To quantify the comparisons, we summarize the performance skill scores in Table 2. These results are the average values of all data for 2 weeks at each location. When interpreting this table, we should not make the mistake of evaluating only one score. For example, the correlation coefficient value represents only the correlating tendency of the model results toward the measured values, so the real difference between the model and measurement cannot be well evaluated by it. In the case of MAPE, the absolute values of each data are taken and averaged, so it is difficult to confirm in which direction the values are biased. Therefore, when interpreting this table, all performance scores should be considered together.
In  (Athieno et al., 2017;Fan et al., 2019;McKinnell & Poole, 2000). To reliably compare the performance of various NN models, it is necessary to pursue further study using the same location and period data.
Next, we compare the results of Icheon and Okinawa based on the use of the ionospheric drivers estimated from Jeju data. At these two locations, the IRI model yields better results for the correlation coefficient, while the SAMI2-LSTM model received the best ratings for the rest of the scores. The performance shown in  Figure 5 and Table 2 indicates that the long-term predictions of NmF2 and hmF2 at other locations on nearly the same meridional plane as the reference location can be simulated by the SAMI2-LSTM model with relatively high accuracy. The scores of Icheon, which is located slightly further north, are better than those of Jeju, while the scores of Okinawa, which is located at a lower latitude, are the lowest. We speculate that this result is due to the following causes. First, since the background electron density increases as the latitude gradually decreases, the RMSE value may increase as the latitude decreases. Second, Okinawa's geomagnetic latitude is 16.54°, close to the 10-15°region where the equatorial ionospheric anomaly peak exists. Therefore, the ionospheric parameters at the Okinawa station include the ionospheric variation associated with the equatorial anomaly, which cannot be seen in the midlatitude ionosphere during the quiet periods. Also, because there are more effects by the electric field as well as the neutral wind and density components at low latitudes, the RMSE values can be calculated higher. Consequently, the results of modeling the low-latitude ionosphere are less reliable.

Space Weather
We further calculate RD to examine the bias in model vs measurement differences that are difficult to identify in MAPE. As can be seen shown with the red bar plot in Figure 6, the SAMI2-LSTM model follows nearly the normal distributions of both NmF2 and hmF2 well centered on 0, implying little or no bias. However, the RD distributions of the IRI-2016 and SAMI2 original models are neither normal nor centered on 0 for both NmF2 and hmF2. Especially, all these model overestimate significantly hmF2 for all three locations. The SAMI2-LSTM model corrected the hmF2 bias fairly well by updating the effective meridional wind derived from the observations in the assimilation process. Because we obtained better hmF2 prediction not only at one location but also at another, we think our method for the assimilation is sufficiently applicable to compensate for the ignorance of neutral wind and electric field that are need in the physics model.
In the case of NmF2, the SAMI2 original model tends to overestimate slightly at all three locations, whereas the IRI model shows different patterns depending on the location. The problem of electron density overestimation in the SAMI2 model has been reported in some other research (Kim et al., 2016;Klenzing et al., 2013). This may be due to the overestimation in the neutral atmospheric density of the MSIS model (Emmert et al., 2010;Liu et al., 2017). To quantify the bias between the model and measurement, we calculated the proportion of positive vs negative RDs, as listed in Table 3. The SAMI2-LSTM model shows the least bias proportions for both NmF2 and hmF2 among all the models. Therefore, we verified that the LSTM model could improve the results of the physical model through longer-term predictions, not only for one location where the model had been trained with data but also at other locations. Obviously, it can also be a good criterion if someone develops assimilation devices and applies them to the IRI model. However, these tasks were beyond the scope of this study and could not be done.

Geomagnetic Storm Case
We evaluated the performance of each model for 3 days from 6 September 2017 (doy = 249) through 8 September 2017 (doy = 251), a geomagnetically disturbed period. Figure 7 shows the results for the geomagnetic storm days in the same format as in Figure 5. The blue arrows, numbers, and the vertical dashed lines indicate the time when solar flare events occurred, respectively.
First, it seems that there was a weak increase in electron density in the observations of Jeju and Icheon, which may be the effect of solar flares. Indeed, because the sixth, seventh, tenth, eleventh, and twelfth solar flares occurred during the daytime, it can be expected that they would have affected the ionosphere changes. However, none of the SAMI2-LSTM, SAMI2 original, and IRI-2016 models simulated the ionospheric responses due to solar flares. Indeed, the SAMI2-LSTM model uses the flared version of the FISM solar radiation model (Chamberlin et al., 2008). Although the effect of solar flares was expected, the SAMI2-LSTM model did not produce visible increase in the F2 layer. We speculate that the moderate M class flares affected very little the F2 layer in the model. Studies of the ionosphere response to solar flares have shown that the ionosphere impact is considerably imperceptible when a flare of class M or lower occurs (Barta et al., 2019). In addition, because its response depends not only on the solar flare class but also on the location of its occurrence (Donnelly, 1971;Donnelly & Puga, 1990). Since the SAMI2 original and the IRI-2016 models do not input flare increases of EUV and X-ray at all, any ionospheric responses to the solar flare are not expected. It may be possible to train an NN model for ionospheric increases due to solar flares once the ionosonde data set and solar flare data set are prepared appropriately. This idea will be probed in the future.

Space Weather
In regard to geomagnetic storms, NmF2 and hmF2 observed in Jeju and Icheon show a significant increase during the main phase of the second storm, which may be called a positive ionospheric storm.
The SAMI2-original model seems to simulate NmF2 increases on all 3 days, whereas the observed values increased only during the first and second storm periods. In addition, the timing of peak increases does not match with the observation. Before the first storm, the SAMI2-LSTM model predicted closely the observed NmF2s, which is similar to the quiet case. However, during the first and second geomagnetic storm periods, the performance of the SAMI2-LSTM model does not predict positive geomagnetic storms well. Table 4 summarizes the performance skill scores for the geomagnetically disturbed period. The performance analysis for Okinawa was Figure 7. The model and observed F2 parameters during a geomagnetically disturbed period. Each line color means the same in Figure 5. In addition, the red arrows, numbers, and the vertical dashed lines indicate the solar flare events.  Note. NmF2 ( foF2) unit = #/cm 3 × 10 5 (MHz); hmF2 unit = km. Figure 8. Same as Figure 6, but during the storm period except for Okinawa location.

Space Weather
and hmF2 on Day 251. Therefore, despite better performance scores of the IRI-2016 and SAMI2-LSTM models, in consideration of no storm-related ionospheric increase in these models, we conclude that they, too, failed ionospheric prediction during the geomagnetically disturbed periods.
Our analysis identified two reasons for the poor performance to predict geomagnetic storm-related ionospheric changes using the LSTM model. First, we trained the model using only a 5-year-long data set from the ionosonde in Jeju, which includes only 57 cases of geomagnetic storms. In other words, the model was trained mostly under the geomagnetically quiet condition, resulting in reasonable performance for quiet days but not for stormy days. For better performance during geomagnetic storms, it is necessary to collect and include much more ionospheric data for the storm case in the training of the LSTM model. The second reason is from the fact that we trained the LSTM model with a simple geomagnetic index input of Kp. Wintoft and Cander (2000) suggested that in the prediction study of foF2 using NN models during the storm events, the model should be designed to extend the AL or AU index rather than the simple AE index. In addition, Nakamura et al. (2009) discussed that the Dst and the local K index should be included in order to predict the storm case well. Thus, there is a need to create a new LSTM model with inputs of appropriate geomagnetic indices. We plan to develop and apply a new deep-learning model for storms in the near future to overcome these problems.
In Figure 8, we plot the histogram that shows the NmF2 and hmF2 differences between predicted and observed values at Jeju and Icheon stations, similar to Figure 6. The analysis for Okinawa station was difficult to interpret due to a lack of data. The SAMI2-LSTM and IRI-2016 models tend to underestimate the NmF2, but the SAMI2 original model shows overestimation. In the case of hmF2, the SAMI2-LSTM and IRI-2016 models follow a fairly normal distribution, while the other two models have a broad error range.

Summary and Conclusion
In this study, we utilized a deep-learning algorithm, LSTM, in an attempt to address the limitation in the short-term prediction of the assimilated SAMI2 model proposed by Kim et al. (2019). The LSTM algorithm is useful for time series data analysis to make forecast values reflecting the past data. We trained the LSTM model with a 5-year-long data set of the ionospheric F2 layer parameters observed from the Jeju ionosonde and the observed space environment indices. Utilizing the predicted values of the F2 parameters (NmF2 and hmF2) from the trained LSTM model, we estimated the ionospheric drivers (effective meridional winds and electron density scale factor), which were then input to the SAMI2 model as the assimilation procedure. The assimilated SAMI2-LSTM model calculates the prediction values of the ionosphere on the meridional plane crossing Jeju station for the 24-hr period. We evaluate the performance of the SAMI2-LSTM model for both geomagnetically quiet (2 weeks) and disturbed (3 days) periods by comparing the predicted NmF2 and hmF2 with observed values at Jeju, Icheon, and Okinawa stations that are nearly the same longitude.
The principal findings of this study are as follows: 1. On quiet geomagnetic days, the SAMI2-LSTM model showed a level of accuracy that was approximately 45% and 45% higher than the SAMI2 original model and IRI-2016 model, respectively, based on the RMSE values of NmF2 at the Jeju station. The model also improves the accuracy by 49% and 34% (37% and 38%) from those of the SAMI2 original model and IRI-2016 model, respectively, at the Icheon (Okinawa) station. 2. On quiet geomagnetic days, the SAMI2-LSTM model shows the RMSE improvements of hmF2 over Jeju, Icheon, and Okinawa by about 30%, 37%, and 30%, respectively, from the SAMI2 original model, and by about 11%, 28%, and 5%, respectively, from the IRI-2016 model. 3. The RDs in NmF2 and hmF2 between the SAMI2-LSTM model and observation are significantly closer to the normal distribution centered at 0, compared to other models, indicating little bias in the predicted values. 4. Although the SAMI2-LSTM model shows better RMSE performance in NmF2 and hmF2 prediction than other comparing models for the geomagnetically disturbed period, the model fails to predict positive ionospheric effects from either geomagnetic storms or solar flares.
The above-mentioned improvement stems from the fact that the SAMI2-LSTM model utilizes ionospheric drivers computed from the LSTM model that were trained by an observed data set at one location. In 10.1029/2020SW002590

Space Weather
conclusion, we have demonstrated that the combination of an NN model with a physics-based model can improve ionospheric predictions of existing theoretical and empirical models for the midlatitude region, at least under the geomagnetically quiet condition. Thus, this method of combining a deep-learning model with a physics-based model opens the opportunity to addressing the respective weaknesses of an empirical model and a theoretical model in the ionosphere forecasting.

Data Availability Statement
The Jeju and Icheon ionosonde data were obtained from the Korean Space Weather Center homepage (https://spaceweather.rra.go.kr/observation/service/iono). The Okinawa data were provided from the webpage of the National Institute of Information and Communications (NICT) in Japan (http://wdc.nict.go.jp/ IONO/HP2009/ISDJ/index-E.html). The solar and geomagnetic indices data can be downloaded from the OMNI online system (https://omniweb.gsfc.nasa.gov/ow.html). The GOES X-ray data can be accessed through NOAA NGDC (https://satdat.ngdc.noaa.gov/sem/goes/data/full/