Generate QR code
Open Access
Subscription Access
A strategy for finding outliers in noisy data series including an unknown trend
https://doi.org/10.32446/0368-1025it.2022-5-29-34
Abstract
One of the problems related to detection of coarse measurements (outliers) in automated processing of data series measured in technical devices has been considered. A modifi cation of the strategy developed by the fi rst author for detecting outliers in time series of noisy data containing an unknown trend is proposed. The previously developed strategy consists of two steps: fi nding a trend and applying to the residues obtained after subtracting the found trend from the measurement data, an algorithm for fi nding the optimal solution. The search for a trend is carried out in the power polynomials class by means of the least squares method using sets with preset number of reference values. The trend search algorithm is carried out using a completely convergent iterative process and is based on the method of minimizing sequences (sets). The disadvantage of this strategy is the need to set a priori the total number of reference values by which the trend is built, which can lead to a distorted determination of the trend and incorrect detection of outliers. In the proposed strategy, the number of reference values is selected from the condition of minimizing the number of detected outliers on the one hand, and from the condition of maximizing the numberof reference values on the other. The results of numerical testing on real data for satellite laser range fi nding measurements are given. The proposed strategy can be used to detect and eliminate outliers from time series of data at the stage of their preliminary processing.
About the Authors
I. V. Bezmenov
Russian Metrological Institute of Technical Physics and Radio Engineering
Russian Federation
Russian Federation
Igor V. Bezmenov
Mendeleevo, Moscow Region
A. E. Drozdov
Russian Metrological Institute of Technical Physics and Radio Engineering
Russian Federation
Russian Federation
Aleksey E. Drozdov
Mendeleevo, Moscow Region
S. L. Pasynok
Всероссийский научно-исследовательский институт физико-технических и радиотехнических измерений
Russian Federation
Russian Federation
Sergey L. Pasynok
Mendeleevo, Moscow Region
References
1. International GNSS Service, available at: http://www.igs.org/network (accessed: 05.04.2022).
2. Pasynok S. L., Bezmenov I. V., Ignatenko I. Yu., Ivanov V. S., Tsyba E. N., Zharov V. E., Transactions IAA RAS, 2021, iss. 57, pp. 28–33. (In Russ.) https://doi.org/10.32876/ApplAstron.57.28-33
3. Bernese GNSS Software Version 5.2. User manual, ed. Dach R., Lutz S., Walser P., Fridez P., Astronomical Institute, University of Bern, Bern Open Publishing, 2015. https://doi.org/10.7892/boris.72297
4. Bezmenov I. V., Blinov I. Yu., Teoreticheskie osnovy postroenija modelej dlja opisanija sovremennyh shkal vremeni i standartov chastoty [Theoretical foundations for model construction to describe modern time scales and frequency standards], Mendeleevo, VNIIFTRI Publ., 2015, 528 p. (In Russ.)
5. Mehlstäubler T. E., Grosche G., Lisdat C., Schmidt P. O., Denker H., Reports on Progress in Physics, 2018, vol. 81, no. 6, 064401. https://doi.org/10.1088/1361-6633/aab409
6. GGOS Requirements for Core Sites (Revision 2) Draft 3.4, available at: https://cddis.nasa.gov/docs/2015/SiteRecDoc_Rev2_D3.4.pdf (accessed: 16.04.2022).
7. Blazquez-Garcia A., Conde A., Mori U., Lozano J. A., ACM Computing Surveys, 2021, vol. 54, iss. 3, pp. 1–33. https://doi.org/10.1145/3444690
8. Fox A. J., Outliers in Time Series, Journal of the Royal Statistical Society: Series B (Methodological), 1972, vol. 34, no. 3, pp. 350–363.
9. Basu S., Meckesheimer M., Knowl. Inf. Syst, 2007, vol. 11, no. 2, pp. 137–154. https://doi.org/10.1007/s10115-006-0026-6
10. Mehrang S., Helander E., Pavel M., Chieh A., Korhonen I., 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Washington, DC, USA, 2015, 9–12 November, IEEE, 2015, pp. 1489–1496. https://doi.org/10.1109/BIBM.2015.7359896
11. J. Chen, W. Li, A. Lau, J. Cao, and K. Wang, IEEE Transactions on Smart Grid, 2010, vol. 1, no. 2, pp. 213–221. https://doi.org/10.1109/TSG.2010.2053052
12. Bezmenov I. V., Naumov A. V., Pasynok S. L., Measurement Techniques, 2018, vol. 61, pp. 878–884. https://doi.org/10.1007/s11018-018-1518-y
13. Sinclair A. T., Data Screening and Normal Point Formation, Re-Statement of Herstmonceux Normal Point Recommendation, available at: https://ilrs.cddis.eosdis.nasa.gov/data_and_products/ data/npt/npt_algorithm.html (accessed: 06.04.22).
14. Bezmenov I. V., Effective Algorithms for Detection Outliers and Cycle Slip Repair in GNSS Data Measurements, ch. 9 in book: Satellite Systems: Design, Modeling, Simulation and Analysis,. ed. Tien M. Nguyen, IntechOpen, London, UK, 2021, pp. 177–209. https://doi.org/10.5772/intechopen.73789
Review
For citations:
Bezmenov I.V., Drozdov A.E., Pasynok S.L. A strategy for finding outliers in noisy data series including an unknown trend. Izmeritel`naya Tekhnika. 2022;(5):29-34. (In Russ.) https://doi.org/10.32446/0368-1025it.2022-5-29-34
Views:
181
Email this article 