>Now we have talked about the topic of 'monitoring'. The idea is
>
>- to pass the 'reviews' of the product, some requirements are defined
>and metrics to verify that the requirements are met
>("validation")
>
>- when data is already available, the products are generated for a time and
>metrics are applied. It is checked whether the requirements are passed or not
>("validation"), and therefore whether they pass the "review" or not. If they pass the
>review, the 'operational' production of the product begins
>
>- from the moment the product is "operational", they continue
>applying the metrics of “validation,” but now it’s done formally
>continued and is called "monitoring".
>
>
>We will obviously participate in the validation part, and also the
>monitoring. There is a part of all this that will be in near-real-time
>(NRT), and this will surely be done by the DMI itself. It will be the part with
>less relation to "accuracy", and more to type: product volume
>generated, delays in production, some initial quality flag (for
>example, if we define an expected range of values ​​for each product, i
>it turns out that a profile gives values ​​out of range), perhaps some
>standard-deviation easy and fast to produce and that can give info if
>something is too noisy, etc.). All that has to be in
>near-real time will do so in the DMI itself.
>
>Our part, then, would be more related to accuracy, and perhaps
>comparing it with other ion / space weather models. NRT should not be done
>but equally automatically and continuously, e.g., daily, gold
>weekly, or once per month ... (TBD). The idea is that we compare with
>some other ion data source. You have your own model,
>right? Do you have plans to have it up and running by ~ 2024? do you think it would have
>meaning that the validation and monitoring part is done by giving the
>comparison with your model? If you have access to other models, or models
>advanced iono-climatological conditions, perhaps it could also be done with these (or
>instead of yours) ...
>
>How do you see it? complicated or feasible?
>

I see it as feasible but requiring significant or very significant effort. My thoughts "extrapolate" the experience in the service we provide for global maps from VTEC (GIMs) to IGS 
(since 1998, with latencies of 1 day or more, and since 2014 in real time) and for the civil aviation organization, ICAO (since one year ago in real time).

I explain myself by the validation (and largely by the monitoring) of the EPS-SG mission, orbiting at ~ 800 km altitude, we would be talking about:

1) Electronic density profiles, obtained with truncated GNSS ROs data, only available below 500 km of impact parameter.
2) The POD-ROVI (Rate Of VTEC Index) as a simple scintillation index with GNSS data (POD) above the LEO.

To validate 1) we can consider:

1st- "Precision": consistency between EPS-SG profiles approximately placed, with an RMS between them that would degrade depending on the distance). It should be characterized what is the expected correlation between the profiles as a function of distance in a previous study that should be done at the beginning of the activity, and then it would seem relatively easy to implement, also by monitoring.

1b- "Accuracy": Here we have the egg yolk. What really gives confidence at this point is to find an external observational source (independent and much more accurate than the climate models of the ionosphere), independent of our observational source (GNSS RO EPS-SG), reliable enough, and with "biases" well characterized in a previous study that should be done at the beginning of the new activity. Here are at least three potential external sources, being likely the most convenient the first one:

1bi- COSMIC-2 GNSS ROs, which complete (i.e. no-truncated) ROs would be routinely processed at IEEC, starting during a first phase of CDOP-4. The EPS-SG collocated ROs would be compared, taking as reference the complete ones processed from COSMIC-2.

1bii- Ion probe (ionosondes) measurements: give the profile "bottomside" but with the possibility that they are affected by significant errors (see Araujo et al. 2019), especially with the heights if they are not manually scaled. Another limitation is the relatively small number of these facilities around the world, but also with a European network in NRT (mid-high latitude, direct contact with the coordinator), and we have direct contacts for example in Brazil (low latitude). The "reliability" of the Nm density at the peak of the profile (typically the F2 layer and sporadically the E layer) should also be studied, presumably more reliable including automatically, and if it were the case likely to be also serve in monitoring (TBC). All this would involve a previous study to have it well characterized.

1biii- Compare “EPS-SG” profiles, in the domain of the new low-resolution tomography with independent measures “multi-Space-Geodesy” (and this would be another previous study, in the first phase of the project). Indeed, in the manuscript Hernández-Pajares et al. 2020 (accepted two days ago at Journal of Geodesy) it is shown by the first time the benefit for the generation of VTEC Global Ionospheric Maps by combining:

- GNSS data from land (and from an available vessel),

- with GNSS POD data from LEOs at different heights (SWARM, Sentinel ...)

- and non-GNSS data (DORIS) with ground transmitters and receivers in LEOs, including known altimetric missions.

This has made it possible to double the vertical resolution of the model and we have so far run with two layers with ground-only ground GNSS data, and that is what works slightly better than all IGS (see GIM UQRG in Roma-Dollase et al. 2018). Well, this will predictably be the subject of a new student's doctoral thesis who has been working with me at UPC for almost a year on a Chinese scholarship, and which should be completed by 2023. The goal the benefit of this approach in different solar conditions will be characterized, and to generate a new operative product, GIM, but also average electronic densities of each layer. And the latter would be another independent source for comparing EPS-SG electronic density profiles and likely to be applied to monitoring as well.

Regarding scintillation (point 2), we have less experience here. What we did at CDOP3, and it seems to work reasonably well. is to compare the POD-ROVI (proposed at CDOP-3, and presented at the Copenhagen meeting) with the ROTI of ground data, which have appeared quite correlated at high latitude (TBC extensively in another initial study during CDOP- 4 if deemed appropriate). They are not fully equivalent, especially at low latitudes when there may be scintillation below the LEO, for example due to the “plasma bubbles” that are often below (see Prol et al. 2018).

I think so, that we can make a very significant contribution in the design and study of validation, and with the consequences, in designing monitoring in the early years of CDOP-4. And then in the execution of the products and monitoring, with us as "advisors" (for example monthly or weekly) of the 24/7 service that I understand would be the responsibility of DMI.

In summary, I see the problem very interesting, with a contribution that can be decisive from IEEC to CDOP-4 in iono. (the above points can contribute to the preparation of the proposal), and which can be perfectly equivalent to 3 or 4 projects of 1 year, one for each of the points mentioned. But I understand that would not be a problem thanks to the 5-year "timeline" of what is available in CDOP-4 if I do not misunderstand.

Then there is the practical issue, very important, of the human team. Haixia is the right person for supporting it at full time, jointly with my part-time support, if the approach looks good and comes out approved. 

>
> do you think that
>
>    1) we could help to define which set of parameters of the products
>    could ion generated be used to do NRT monitoring in DMI?
>    (expected ranges, expected sigma-max, etc.)
>

Yes, from a radio-occultations' database with complete GNSS ROs data (extending that of ROPE from COSMIC-1 in another study at the beginning of the activity, including possibly COSMIC-2 data).
 
>
>    2) we could do it offline (i.e. not in real time) but in form
>    'continues' quantification of how profiles are compared to other sources
>    of ion data?
>

Yes, see points 1a, 1bi, 1bii, 1biii and 2 in my text above.


References:


Araujo-Pradere, E., Weatherhead, E. C., Dandenault, P. B., Bilitza, D., Wilkinson, P., Coker, C., ... & Hernández-Pajares, M. (2019). Critical issues in ionospheric data quality and implications for scientific studies. Radio Science, 54(5), 440-454.

Hernández-Pajares, M., Lyu, H., Garcia-Fernandez, M. & R. Orús-Perez (2020). A new way of improving Global Ionospheric Maps by ionospheric tomography: consistent combination of multi-GNSS and multi-Space Geodetic dual-frequency measurements gathered from vessel-, LEO- and ground-based receivers. Journal of Geodesy (accepted on July 1st 2020).

Prol, F. dos Santos, Hernández-Pajares, M., Tadeu de Assis Honorato Muella, M., & de Oliveira Camargo, P. (2018). Tomographic imaging of ionospheric plasma bubbles based on GNSS and radio occultation measurements. Remote Sensing, 10(10), 1529.

Roma-Dollase, D., Hernández-Pajares, M., Krankowski, A., Kotulak, K., Ghoddousi-Fard, R., Yuan, Y., ... & Feltens, J. (2018). Consistency of seven different GNSS global ionospheric mapping techniques during one solar cycle. Journal of Geodesy, 92(6), 691-706.