The minimizing sets method for trend detection in time series of noisy measurement data

This article discusses the problem of trend detection in time series generated by technical devices. The solution   to this problem is closely related to the problem of detecting coarse measurements (outliers), which negatively impact the accuracy of estimates of various physical quantities. These are crucial in many applications in various scientific fields in which the input data are observations, such as space geodynamics, geodesy, and others. Previously, the author proposed   a trend-detecting method based on the condition of maximizing the amount of data cleared of outliers and used in further   processing. The reference values used for trend construction are determined as a result of a completely convergent iterative process, the core of which is the minimizing sets method developed earlier by the author. This paper deals with the aspects of trend construction in the class of harmonic functions with unknown frequencies, phases and amplitudes.The main problem of trend construction in such a functional class is the nonlinear dependence of harmonics on the desired parameters, which does not allow to reduce the problem of trend search to the solution of a system of linear equations. The search for harmonics approximating the measurement data is carried out by the conjugate gradients method generalized to nonlinear problems. The efficiency of the method was tested on the test problem of trend construction in the data obtained by computer simulation.

1. Dach R., Beutler G., Hugentobler U. et al. Time transfer using GPS carrier phase: error propagation and results. Journal of Geodesy, 77(1-2), 1–14 (2003). https://doi.org/10.1007/s00190-002-0296-z

2. Donchenko S. I., Blinov I. Y., Norets I. B. et al., Characteristics of the long-term instability of the new-generation hydrogen frequency and time standards. Measurement Techniques, 63(1), 34–37 (2020). https://doi.org/10.1007/s11018-020-01746-8

3. Bernese GNSS Software Version 5.2. Dach R., Lutz S., Walser P., Fridez P. (eds.) Astronomical Institute, University of Bern, Bern Open Publishing, Bern (2015). https://doi.org/10.7892/boris.72297

4. Pasynok S. L. Increase in the determination accuracy of the Earth’s rotation parameters by combining the results of various types of measurements in the Main Metrological Center of the State service of time, frequency, and earth’s rotation parameters evaluation. Measurement Techniques, 63(1), 38–44 (2020). https://doi.org/10.1007/s11018-020-01747-7

5. Zharov V. E., Pasynok S. L. SAI-VNF VLBI Analysis Center in 2019–2020. International VLBI Service for Geodesy and Astrometry 2019+2020 Biennial Report, Behrend D., Armstrong K. L., Baver K. D. (eds.), NASA/TP-20210021389, 258–259 (2021). https://ivscc.gsfc.nasa.gov/publications/br2019+2020/acsai-vniiftri.pdf

6. Schubert E., Weiler M., Zimek A. Outlier detection and trend detection: two sides of the same coin. Proceedings 2015 IEEE International Conference on Data Mining Workshop (ICDMW), Atlantic City, NJ, USA, рр. 40–46 (2015). https://doi.org/10.1109/ICDMW.2015.79

7. Blazquez-Garcia A., Conde A., Mori U., Lozano J. A., A review on outlier/anomaly detection in time series data. ACM Computing Surveys, 54(3), 1–33 (2021). https://doi.org/10.1145/3444690

8. Vaniček P. Approximate spectral analysis by least-squares fit. Astrophysics and Space Science, 4(4), 387–391 (1969). https://doi.org/10.1007/BF0065134

9. Erten E., Reigber A., Hellwich O. Generation of three-dimensional deformation maps from InSAR data using spectral diversity techniques. ISPRS Journal of Photogrammetry and Remote Sensing, (65), 388–394 (2010). https://doi.org/10.1016/j.isprsjprs.2010.04.005

10. Sosnovsky A. V. InSAR data processing in digital elevation models creating tasks: state-of-art and issues. Ural Radio Engineering Journal, 4(2), 198–233 (2020). (In Russ.) https://doi.org/10.15826/urej.2020.4.2.004

11. Bernardi M.S., Africa P.C., de Falco C., et al. On the use of interferometric synthetic aperture radar data for monitoring and forecasting natural hazards. Mathematical Geosciences, (53), 1781–1812 (2021). https://doi.org/10.1007/s11004-021-09948-8

12. Ma P., Zheng Y., Zhang Z. et al. Building risk monitoring and prediction using integrated multi-temporal InSAR and numerical modeling techniques. International Journal of Applied Earth Observation and Geoinformation, (114), 103076 (2022). https://doi.org/10.1016/j.jag.2022.103076

13. Venmans A. A. M., op de Kelder M., de Jong J. et al. Reliability of InSAR satellite monitoring of buildings near inner city quay walls. Proceedings of the International Association of Hydrological Sciences, (382), 195–199 (2020). https://doi.org/10.5194/piahs-382-195-2020

14. Li M., Wu H., Yang M. et al. Trend classification of InSAR displacement time series using SAE–CNN. Remote Sensing, 16(1), 54 (2023). https://doi.org/10.3390/rs16010054

15. Zhou C., Gong H., Chen B. et al. InSAR time-series analysis of land subsidence under different land use types in the Eastern Beijing Plain, China. Remote Sensing, 9(4), 380 (2017). https://doi.org/10.3390/rs9040380

16. Kermarrec G., Maddanu F., Klos A. et al. Modeling trends and periodic components in geodetic time series: a unified approach. Journal of Geodesy, 98, 17 (2024). https://doi.org/10.1007/s00190-024-01826-5

17. Chang K.-L., Schultz M. G., Lan X. et al. Trend detection of atmospheric time series: incorporating appropriate uncertainty estimates and handling extreme events. Elementa: Science of the Anthropocene, 9(1), 00035 (2021). https://doi.org/10.1525/elementa.2021.00035

18. Melek W. W., Lu Z., Kapps A., Fraser W. D. Comparison of trend detection algorithms in the analysis of physiological time series data. IEEE Transactions on Biomedical Engineering, 52(4), 639–651 (2005). https://doi.org/10.1109/TBME.2005.844029

19. Mredula M. S., Dey N., Rahman M. S. et al. A review on the trends in event detection by analyzing social media platforms’ data. Sensors, (22), 4531 (2022). https://doi.org/10.3390/s22124531

20. Bezmenov I. V., Naumov A. V., Pasynok S. L. An effective algorithm for elimination of outliers from data measurements of Global Navigation Satellite Systems. Measurement Techniques, 61(9), 878–884 (2018). https://doi.org/10.1007/s11018-018-1518-y

21. Bezmenov I. V. Method of cleaning outliers from measurement data: search for the optimal solution with the minimum number of rejected measured data. Measurement Techniques, 66(1), 14–23 (2023). https://doi.org/10.1007/s11018-023-02184-y

22. Bezmenov I. V. Fast algorithm for cleaning highly noisy measurement data from outliers, based on the search for the optimal solution with the minimum number of rejected measurement data. Metascience in Aerospace, 1(1), 110–129 (2024). https://doi.org/10.3934/mina.2024005

23. Bezmenov I. V. Effective algorithms for detection outliers and cycle slip repair in GNSS data measurements. In: Satellite Systems – Design, Modeling, Simulation and Analysis, ed. Nguyen T. M. IntechOpen, London, UK, рр. 177–209 (2021). https://doi.org/10.5772/intechopen.92658

24. Bezmenov I. V. Trend detection in time series of measurement data in solving problems in space geodynamics and other research areas. Metascience in Aerospace, 1(3), 268–291 (2024). https://doi.org/10.3934/mina.2024012

25. Ignatenko I. Yu., Bezmenov I. V. Application of stable estimation algorithms in laser location measurement processing. Metrology of time and space. Reports of the XI International Symposium, Mendeleevo, Russia, 27–29 September 2023, Mendeleevo, FSUE “VNIIFTRI”, рр. 71–81 (2024). (In Russ.)

26. Bezmenov I. V., Ignatenko I. Yu., Pasynok S. L. New methods for achieving advanced accuracy in time and coordinate measurements. Transactions of IAA RAS, (60), 12–20 (2022). (In Russ.) https://doi.org/10.32876/ApplAstron.60.12-20

27. Bezmenov I. V., Drozdov A. E., Pasynok S. L. A Strategy for finding outliers in noisy data series including an unknown trend. Measurement Techniques, 65(5), 339–345 (2022). https://doi.org/10.1007/s11018-022-02085-6

28. Samarskii A. A., Nikolaev E. S. Numerical methods for grid equations, Birkhäuser Verlag, Basel (1989). https://doi.org/10.1007/978-3-0348-9142-4

29. Tsyba E. N., Volkova O. A., Vostrukhov N. A. Experience of using autoencoders in solving problems of detecting anomalies in time series of measurement information. Al’manac of Modern Metrology, 2(38), 150–160 (2024). (In Russ.)

30.