Modified estimation of the Pearson correlation coefficient: normalization of random variables according to the distribution mode

The problems of structural analysis of statistical data and construction of linear models of stochastic dependencies under a priori uncertainty are considered. A modified estimate of the correlation coefficient is proposed and investigated, which forms the basis for the structural analysis of statistical data. Unlike the traditional Pearson correlation coefficient, the modified estimator is based on normalizing random variables by the modes of their probability densities. To find modes of distribution laws, kernel estimates of probability densities of the random variables being analyzed are used. The choice of the blurriness coefficients for the kernel functions of nonparametric probability density estimates is based on the condition of maximizing the likelihood function. An alternative approach to choosing blurriness coefficients is to minimize the standard deviations of nonparametric probability density estimates. Estimates of traditional and modified correlation coefficients are examined. Their application in constructing linear approximations of statistical relationships is discussed. For this purpose, remote sensing data from a test forest area damaged by the Siberian silk moth was used. Two sets of spectral feature pairs were identified, differing in the large and small values of the correlation coefficient estimates under consideration. The correlation coefficient estimates were compared, and the corresponding nonparametric probability density estimates for the spectral features were analyzed. In the analysis of errors in linear approximations of dependencies between spectral features, the conditions for the advantage of traditional and modified estimates of correlation coefficients were determined. The obtained results can be used in the synthesis of algorithms for structural analysis of remote sensing data of natural objects.

1. Lapko A. V., Lapko V. A. Procedure for decomposing the values of two-dimensional spectral features of remote sensing based on the analysis of correlation coefficient components. Izmeritel’naya Tekhnika, 73(6), 12–17 (2024). (In Russ.) https://doi.org/10.32446/0368-1025it.2024-6-12-17 ; https://www.elibrary.ru/pphujk

2. Lapko A. V., Lapko V. A., Im S. T. Decomposition of spectral features of remote sensing based on the correlation coefficient components. Optoelectronics, Instrumentation and Data Processing, 61(3), 310–317 (2025). https://doi.org/10.3103/S8756699025700360 ; https://elibrary.ru/vmeflu

3. Lapko A. V., Lapko V. A. A method for evaluating the informative value of arguments of a nonparametric stochastic dependence model with their specific values. Automatic Documentation and Mathematical Linguistics, 59(4), 252–255 (2025). https://doi.org/10.3103/S0005105525700694 ; https://elibrary.ru/hkwzqv

4. Johnston J., Dinardo J. Econometric Methods. McGraw-Hill, New York (1997).

5. Gentle J. E., Härdle W. K., Mori Y. Handbook of computational statistics. Springer, Berlin, Heidelberg (2012). https://doi.org/10.1007/978-3-642-21551-3

6. Härdle W. K., Lu H. H. S., Shen X. Handbook of big data analytics. Springer, Berlin, Heidelberg (2018). https://doi.org/10.1007/978-3-319-18284-1

7. Lu H. H. S., Schölkopf B., Wells M. T., Zhao H. Handbook of statistical bioinformatics. Springer, Berlin, Heidelberg (2022). https://doi.org/10.1007/978-3-662-65902-1

8. Parzen E. On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33(3), 1065–1076 (1962). https://doi.org/10.1214/aoms/1177704472

9. Epanechnikov V. A. Non-parametric estimation of a multivariate probability density. Theory of Probability & Its Applications, 14(1), 156–161 (1969). https://doi.org/10.1137/1114019

10. Sharueva A. V., Lapko A. V., Lapko V. A. Nonparametric methods for hypotheses testing about distributions of random variables in the analysis of remote sensing data. SB RAS, Novosibirsk (2024). (In Russ.) https://doi.org/10.53954/9785604990094 ; https://elibrary.ru/dfbrbi

11. Duin R. P. W. On the choice of smoothing parameters for Parzen estimators of probability density functions. IEEE Transactions on Computers, C-25(11), 1175–1179 (1976). https://doi.org/10.1109/TC.1976.1674577

12. Botev Z. I., Kroese D. P. Non-asymptotic bandwidth selection for density estimation of discrete data. Methodology and Computing in Applied Probability, 10(3), 435–451 (2008). https://doi.org/10.1007/s11009-007-9057-z ; https://elibrary.ru/pljmag

13. Lapko A. V., Lapko V. A. Analysis of optimization methods for nonparametric estimation of the probability density with respect to the blur factor of kernel functions. Izmeritel’naya Tekhnika, (6), 3–8 (2017). https://www.elibrary.ru/yzjoon

14. Rudemo M. Empirical choice of histogram and kernel density estimators. Scandinavian Journal of Statistics, (9), 65–78 (1982).

15. Bowman A. W. A comparative study of some kernel-based non-parametric density estimators. Journal of Statistical Computation and Simulation, 21, 313–327 (1982).

16. Hall P. Large-sample optimality of least squares cross-validation in density estimation. Annals of Statistics, 11, 1156–1174 (1983).

17. Jiang M., Provost S. B. A hybrid bandwidth selection methodology for kernel density estimation. Journal of Statistical Computation and Simulation, 84(3), 614–627 (2014).

18. Dutta S. Cross-validation Revisited. Communications in Statistics - Simulation and Computation, 45(2), 472–490 (2016).

19. Guan Y. A Composite likelihood cross-validation approach in selecting bandwidth for the estimation of the pair correlation function. Scandinavian Journal of Statistics, 34(2), 336–346 (2007). https://doi.org/10.1111/j.1467-9469.2006.00533.x

20. Silverman B. W. Density Estimation for Statistics and Data Analysis. Chapman & Hall, London (1986).

21. Scott D. W. Multivariate Density Estimation: Theory, Practice, and Visualization. John Wiley & Sons, New Jersey (2015).

22. Sheather S. J. Density estimation. Statistical Science, 19(4), 588–597 (2004). https://doi.org/10.1214/088342304000000297

23. Kharuk V. I., Im S. T., Soldatov V. V. Siberian silkmoth outbreaks surpassed geoclimatic barrier in Siberian Mountains. Journal of Mountain Science, 17, 1891–1900 (2020). https://doi.org/10.1007/s11629-020-5989-3 ; https://elibrary.ru/xetekq

24. Kharuk V. I., Im S. T., Ranson K. J., Yagunov M. N. Climate-induced northerly expansion of Siberian silkmoth range. Forests, 8(8), 301 (2017). https://doi.org/10.3390/f8080301 ; https://elibrary.ru/xnoowe

25. Ponomarev E. I., Yakimov N. D., Tretyakov P. D., Sultson S. M. Estimation of defoliation features in dark coniferous tree stands after the impact of siberian silk moth according to remote data. Cosmic Research, 62(S1), S73–S80 (2024). https://doi.org/10.1134/S0010952524601208

26. Lapko A. V., Lapko V. A. Im S. T., Yuronen Yu. P. Modified method of structural analysis of remote sensing data. Izmeritel’naya Tekhnika, 74(6), 4–12 (2025). (In Russ.) https://doi.org/10.32446/0368-1025it.2025-6-4-12 ; https://www.elibrary.ru/vvchcc