Comparison of the approximation properties of integral and traditional nonparametric probability density estimates

Nonparametric estimates of the probability density of a random variable are considered, for example, those used in pattern recognition and automatic classification algorithms with a priori uncertainty. With small and limited sample sizes of random variables, the variance of the traditional nonparametric probability density estimation increases. Under these conditions, it is recommended to use an integral estimate of the probability density to reduce the variance. To evaluate the areas of competence of the integral probability density estimation and nonparametric Rosenblatt-Parsen statistics, their approximation properties are compared.

The method of synthesizing an integral estimate of the probability density of a random variable is based on smoothing the traditional kernel estimate by using an integral operator. The peculiarity of the integral probability density estimate is the implicitly defined kernel function, the form of which depends on its blur coefficient and smoothing parameter. When restoring the probability density under specific statistical data conditions, the properties of the integral estimate of the probability density, in contrast to the traditional nonparametric estimate, are determined not only by the blur coefficient of the kernel function, but also by its type. To justify the effectiveness of the integral probability density estimate, the conditions of its asymptotic convergence are studied, which are compared with the traditional Rosenblatt–Parzen kernel estimate. It is found that the asymptotic values of the mean square deviation and the variance of the integral probability density estimate are less than those of the traditional nonparametric estimate. A procedure has been developed for consistently selecting the optimal blur coefficient for the integral estimation of the probability density and the smoothing parameter. The procedure for sequentially selecting the optimal blur coefficient of the integral probability density estimate and the smoothing parameter is considered. The proposed method for synthesizing an integral estimate of the probability density of a random variable can be applied in conditions of multidimensional random variables. Under these conditions, the integral probability density estimation has an advantage over the traditional nonparametric Rosenblatt-Parsen statistics. The results obtained can be used in the synthesis of pattern recognition and automatic classification algorithms in conditions of limited and small samples of statistical data.

1. Lapko A. V., Lapko V. A. Kernel probability density estimates and their application. Reshetnev University, Krasnoyarsk (2021). (In Russ) https://elibrary.ru/pycmjn

2. Lapko A. V., Lapko V. A. Nonparametric estimate of a parzen-type probability density with an implicitly specified form of the kernel. Izmeritel’naya Tekhnika, (6), 14–17 (2016). (In Russ) https://elibrary.ru/wmcfcl

3. Parzen E. On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33(3), 1065-1076 (1962). https://doi.org/10.1214/aoms/1177704472

4. Epanechnikov V. A. Non-parametric estimation of a multivariate probability density. Theory of Probability & Its Applications, 14(1), 156–161 (1969). https://doi.org/10.1137/1114019

5. Rudemo M. Empirical choice of histogram and kernel density estimators. Scandinavian Journal of Statistics, (9), 65–78 (1982).

6. Bowman A. W. A comparative study of some kernel-based non-parametric density estimators. Journal of Statistical Computation and Simulation, 21(3-4), 313–327 (1985). https://doi.org/10.1080/00949658508810822

7. Hall P. Large-sample optimality of least squares cross-validation in density estimation. Annals of Statistics, 11, 1156–1174 (1983). https://doi.org/10.1214/aos/1176346329

8. Jiang M., Provost S. B. A hybrid bandwidth selection methodology for kernel density estimation. Journal of Statistical Computation and Simulation, 84(3), 614–627 (2014). https://doi.org/10.1080/00949655.2012.721366

9. Dutta S. Cross-validation Revisited. Communications in Statistics – Simulation and Computation, 45(2), 472–490 (2016). https://doi.org/10.1080/03610918.2013.862275