izmertechИзмерительная техникаIzmeritel`naya Tekhnika0368-10252949-5237ФГУП "ВНИИФТРИ"10.32446/0368-1025it.2024-7-60-69izmertech-2197Research ArticleАКУСТИЧЕСКИЕ ИЗМЕРЕНИЯACOUSTIC MEASUREMENTSДвухэтапный алгоритм спектрального анализа для систем автоматического распознавания речиTwo-stage algorithm of spectral analysis for automatic speech recognition systemshttps://orcid.org/0000-0003-3045-3337СавченкоВ. В.SavchenkoV. V.

Владимир Васильевич Савченко

Нижний Новгород

Vladimir V. Savchenko

Nizhny Novgorod

vvsavchenko@yandex.ruhttps://orcid.org/0000-0002-2776-5471СавченкоЛ. В.SavchenkoL. V.

Людмила Васильевна Савченко

Нижний Новгород

Lyudmila V. Savchenko

Nizhny Novgorod

vvsavchenko@yandex.ruНациональный исследовательский университет «Высшая школа экономики»РоссияNational Research University Higher School of EconomicsRussian Federation202405092024076069Copyright © ФГУП "ВНИИФТРИ", 20242024ФГУП "ВНИИФТРИ"ФГУП "ВНИИФТРИ"https://www.izmt.ru/jour/about/submissions#copyrightNoticehttps://www.izmt.ru/jour/article/view/2197

В рамках динамично развивающегося направления исследований в области акустических измерений рассмотрена задача спектрального анализа речевых сигналов в системах автоматического распознавания речи. Отмечена низкая по сравнению с человеческим восприятием устной речи эффективность указанных систем в неблагоприятных условиях речепроизводства (шумы, недостаточная разборчивость звуков речи). Для повышения эффективности систем автоматического распознавания речи предложен двухэтапный алгоритм спектрального анализа речевых сигналов. Первый этап обработки речевого сигнала – параметрический спектральный анализ с использованием авторегрессионной модели голосового тракта условного диктора. Второй этап обработки – преобразование (модификация) полученной спектральной оценки по принципу частотно-избирательного усиления амплитуды основных формант внутрипериодного спектра мощности. Описана программная реализация предложенного алгоритма на базе вычислительной процедуры быстрого преобразования Фурье. С применением авторского программного обеспечения проведён натурный эксперимент: исследована аддитивная смесь гласных звуков речи контрольного диктора с белым гауссовым шумом. По результатам эксперимента сделан вывод об усилении на 10–20 дБ амплитуды основных формант речевого сигнала и, соответственно, существенном улучшении разборчивости звуков речи. Разработанный алгоритм можно применять в системах автоматического распознавания речи, основанных на обработке речевого сигнала в частотной области, в том числе, с использованием искусственных нейросетей.

Within the framework of a dynamically developing direction of research in the field of acoustic measurements, the task of spectral analysis of speech signals in automatic speech recognition systems is considered. The low efficiency of the systems in unfavorable speech production conditions (noise, insufficient intelligibility of speech sounds) compared to human perception of oral speech is noted. To improve the efficiency of automatic speech recognition systems, a two-stage algorithm for spectral analysis of speech signals is proposed. The first stage of speech signal processing consists of its parametric spectral analysis using an autoregressive model of the vocal tract of a conditional speaker. The second stage of processing is the transformation (modification) of the obtained spectral estimate according to the principle of frequency-selective amplification of the amplitude of the main formants of the intra-periodic power spectrum. The software implementation of the proposed algorithm based on the high-speed computational procedure of the fast Fourier transform is described. Using the author’s software, a full-scale experiment was carried out: an additive mixture of vowel sounds of the control speaker’s speech with white Gaussian noise was studied. Based on the results of the experiment, it was concluded that the amplitude of the main speech signal formants were amplified by 10–20 dB and, accordingly, a significant improvement in the speech sounds intelligibility. The scope of possible application of the developed algorithm covers automatic speech recognition systems based on speech signal processing in the frequency domain, including the use of artificial neural networks.

речевой сигналспектральный анализголосовой трактавторегрессионная модельискусственная нейронная сетьаугментация данныхspeech signalspectral analysisvocal tractautoregressive modelall-pole modelartificial neural networkdata augmentationReferencesTernström S. Special issue on current trends and future directions in voice acoustics measurement. Applied Sciences, 13(6), 3514 (2023). https://doi.org/10.3390/app13063514Ternström S. Special issue on current trends and future directions in voice acoustics measurement. Applied Sciences, 13(6), 3514 (2023). https://doi.org/10.3390/app13063514Mishra J., Sharma R. Vocal tract acoustic measurements for detection of pathological voice disorders. Journal of Circuits, Systems and Computers, 2450173 (2024). https://doi.org/10.1142/S0218126624501731Mishra J., Sharma R. Vocal tract acoustic measurements for detection of pathological voice disorders. Journal of Circuits, Systems and Computers, 2450173 (2024). https://doi.org/10.1142/S0218126624501731Li S. A., Liu Y. Y., Chen Y. C., Feng H. M., Shen P. K., Wu Y. C. Voice interaction recognition design in real-life scenario mobile robot applications. Applied Sciences, 13(5), 3359 (2023). https://doi.org/10.3390/app13053359Li S. A., Liu Y. Y., Chen Y. C. et al. Voice interaction recognition design in real-life scenario mobile robot applications. Applied Sciences, 13(5), 3359 (2023). https://doi.org/10.3390/app13053359Савченко А. В., Савченко В. В. Метод измерений интенсивности потока гласных звуков речи для аудиовизуальных диалоговых информационных систем. Измерительная техника, (3), 65–72 (2022). https://doi.org/10.32446/0368-1025it.2022-3-65-72Savchenko A. V., Savchenko V. V. Method for measurement the intensity of speech vowel sounds fow for audiovisual dialogue information systems. Measurement Techniques, 65(3), 219–226 (2022). https://doi.org/10.1007/s11018-022-02072-xO’Shaughnessy D. Trends and developments in automatic speech recognition research. Computer Speech and Language, 83(12) (2024). https://doi.org/10.1016/j.csl.2023.101538O’Shaughnessy D. Trends and developments in automatic speech recognition research. Computer Speech and Language, 83(12) (2024). https://doi.org/10.1016/j.csl.2023.101538Yu D., Deng L. Automatic speech recognition. A Deep Learning Approach. Vol. 1. Springer, London (2016). https://doi.org/10.1007/978-1-4471-5779-3Yu D., Deng L. Automatic speech recognition. A Deep Learning Approach. Vol. 1. Springer, London (2016). https://doi.org/10.1007/978-1-4471-5779-3Савченко В. В. Дивергенция Итакуры-Саито как элемент информационной теории восприятия речи. Радиотехника и электроника, 64(6), 585–592 (2019). https://doi.org/10.1134/S0033849419060093Savchenko V. V. Itakura–Saito Divergence as an element of the information theory of speech perception. Journal of Communications Technology and Electronics, 64(6), 590–596 (2019). https://doi.org/10.1134/S1064226919060093Kathiresan Th., Maurer D., Suter H., Dellwo V. Formant pattern and spectral shape ambiguity in vowel synthesis: The role of fundamental frequency and formant amplitude. The Journal of Acoustical Society of America, 143(3), 1919–1920 (2018). https://doi.org/10.1121/1.5036258Kathiresan Th., Maurer D., Suter H., Dellwo V. Formant pattern and spectral shape ambiguity in vowel synthesis: The role of fundamental frequency and formant amplitude. The Journal of Acoustical Society of America, 143(3), 1919–1920 (2018). https://doi.org/10.1121/1.5036258Fu M., Wang X., Wang J. Polynomial-Decomposition-Based LPC for Formant Estimation. IEEE Signal Processing Letters, 29, 1392–1396 (2022). https://doi.org/10.1109/LSP.2022.3181523Fu M., Wang X., Wang J. Polynomial-decomposition-based LPC for formant estimation. IEEE Signal Processing Letters, 29, 1392–1396 (2022). https://doi.org/10.1109/LSP.2022.3181523Савченко В. В. Мера различий речевых сигналов по тембру голоса. Измерительная техника, (10), 63–69 (2023). https://doi.org/10.32446/0368-1025it.2023-10-63-69Savchenko V. V. A measure of differences in speech signals by the voice timbre. Measurement Techniques, 66(10) 803–812 (2024). https://doi.org/10.1007/s11018-024-02294-1Tokuda I. The source–flter theory of speech. Oxford Research Encyclopedia of Linguistics (2021). https://doi.org/10.1093/acrefore/9780199384655.013.894Tokuda I. The source–flter theory of speech. Oxford Research Encyclopedia of Linguistics (2021). https://doi.org/10.1093/acrefore/9780199384655.013.894Kim H. S. Linear predictive coding is all-pole resonance modeling. Center for Computer Research in Music and Acoustics, Stanford University (2023). https://ccrma.stanford.edu/~hskim08/lpc/lpc.pdfKim H. S. Linear predictive coding is all-pole resonance modeling. Center for Computer Research in Music and Acoustics, Stanford University (2023). https://ccrma.stanford.edu/~hskim08/lpc/lpc.pdfButenko I., Slavnov N., Stroganov Yu., Kvasnikov A. Phonetic-acoustic database of trigrams for Russian dialects speech recognition. AIP Conference Proceeding, 2833(1) (2023). https://doi.org/10.1063/5.0151706Butenko I., Slavnov N., Stroganov Yu., Kvasnikov A. Phonetic-acoustic database of trigrams for Russian dialects speech recognition. AIP Conference Proceedings, 2833(1) (2023). https://doi.org/10.1063/5.0151706Shumway R. H., Stoffer D. S. Spectral analysis and fltering. In: Time series analysis and its applications. Springer Texts in Statistics. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-52452-8_4Shumway R. H., Stoffer D. S. Spectral analysis and fltering. In: Time series analysis and its applications. Springer Texts in Statistics. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-52452-8_4Marple S. L. Digital Spectral Analysis with Applications. 2-nd ed. Dover Publications, Mineola, New York (2019).Marple S. L. Digital Spectral Analysis with Applications. 2 ed. Dover Publications, Mineola, New York (2019).Савченко В. В., Савченко Л. В. Метод асинхронного анализа голосового источника речи на основе двухуровневой авторегрессионной модели речевого сигнала. Измерительная техника, 73(2), 55–62 (2024). https://doi.org/10.32446/0368-1025it. 2024-2-55-62Savchenko V. V., Savchenko L. V. Method for asynchronous analysis of a glottal source based on a two-level autoregressive model of the speech signal. Izmeritel’naya Tekhnika, 73(2), 55–62 (2024). (In Russ.) https://doi.org/10.32446/0368-1025it.2024-2-55-62Савченко В. В., Савченко Л. В. Метод тестирования устойчивости и корректировки параметров авторегрессионной модели речевого тракта. Измерительная техника, 73(5), 54–63 (2024). https://doi.org/10.32446/0368-1025it.2024-5-54-63Savchenko V. V., Savchenko L. V. Method for testing stability and adjusting parameters of an autoregressive model of the vocal tract. Izmeritel’naya Tekhnika, 73(5), 54–63 (2024). (In Russ.) https://doi.org/10.32446/0368-1025it.2024-5-54-63Савченко В. В. Метод авторегрессионного моделирования речевого сигнала с использованием огибающей периодограммы Шустера в качестве опорного спектрального образца. Радиотехника и электроника, 68(2), 138–145 (2023). https://doi.org/10.31857/S0033849423020122Savchenko V. V. A method for autoregression modeling of a speech signal using the envelope of the schuster periodogram as a reference spectral sample. Journal of Communications Technology and Electronics, 68(2), 121–127 (2023). https://doi.org/10.1134/S1064226923020122Savchenko V. V. Method for reduction of speech signal autoregression model for speech transmission systems on lowspeed communication channels. Radioelectronics and Communications Systems, 64(11), 592–603 (2021). https://doi.org/10.3103/S0735272721110030Savchenko V. V. Method for reduction of speech signal autoregression model for speech transmission systems on lowspeed communication channels. Radioelectronics and Communications Systems, 64(11), 592–603 (2021). https://doi.org/10.3103/S0735272721110030Савченко В. В. Гибридный метод спектрального анализа речевых сигналов на основе авторегрессионной модели и периодограммы Шустера. Измерительная техника, (3), 61–66 (2023). https://doi.org/10.32446/0368-1025it.2023-3-61-66Savchenko V. V. Hybrid method of speech signals spectral analysis based on the autoregressive model and Schuster periodogram. Measurement Techniques, 66(3), 203–210 (2023). https://doi.org/10.1007/s11018-023-02211-yСавченко В. В. Совершенствование методики измерения показателя точности авторегрессионной модели речевого сигнала. Измерительная техника, (10), 58–63 (2022). https://doi.org/10.32446/0368-1025it.2022-10-58-63Savchenko V. V. Improving the method for measuring the accuracy indicator of a speech signal autoregression model. Measurement Techniques, 65(10), 769–775 (2023). https://doi.org/10.1007/s11018-023-02150-8Rabiner L. R., Schafer R. W. Theory and Applications of Digital Speech Processing. Prentice Hall (2010).Rabiner L. R., Schafer R. W. Theory and Applications of Digital Speech Processing. Prentice Hall (2010).Alku P., Kadiri S. R., Gowda D. Refning a deep learning-based formant tracker using linear prediction methods. Computer Speech & Language, 81, 101515 (2023). https://doi.org/10.1016/j.csl.2023.101515Alku P., Kadiri S. R., Gowda D. Refning a deep learning-based formant tracker using linear prediction methods. Computer Speech & Language, 81, 101515 (2023). https://doi.org/10.1016/j.csl.2023.101515Kuhn K., Kersken V., Reuter B., Egger N., Zimmermann G. Measuring the accuracy of automatic speech recognition solutions. ACM Transactions on Accessible Computing, 16(4), 1–23 (2024). https://doi.org/10.1145/3636513Kuhn K., Kersken V., Reuter B., Egger N., Zimmermann G. Measuring the accuracy of automatic speech recognition solutions. ACM Transactions on Accessible Computing, 16(4), 1–23 (2024). https://doi.org/10.1145/3636513Candan С. Making linear prediction perform like maximum likelihood in gaussian autoregressive model parameter estimation. Signal Processing, 166, 107256 (2020). https://doi.org/10.1016/j.sigpro.2019.107256Candan С. Making linear prediction perform like maximum likelihood in Gaussian autoregressive model parameter estimation. Signal Processing, 166, 107256 (2020). https://doi.org/10.1016/j.sigpro.2019.107256Боровков А. А. Математическая статистика. Дополнительные главы. Наука. Физматлит, Москва (1984).Borovkov A. A. Matematicheskaya statistika. Dopolnitel’nye glavy. Nauka. Fizmatlit, Moscow (1984). (In Russ.)Jolad B., Khanai R. An approach for speech enhancement with dysarthric speech recognition using optimization based machine learning frameworks. International Journal of Speech Technology, 26, 287–305 (2023). https://doi.org/10.1007/s10772-023-10019-yJolad B., Khanai R. An approach for speech enhancement with dysarthric speech recognition using optimization based machine learning frameworks. International Journal of Speech Technology, 26, 287–305 (2023). https://doi.org/10.1007/s10772-023-10019-yKolbæk M., Tan Z.-H., Jensen S. H., Jensen J. On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28, 825–838 (2020). https://doi.org/10.1109/TASLP.2020.2968738Kolbæk M., Tan Z.-H., Jensen S. H., Jensen J. On Loss Functions for Supervised Monaural Time-Domain Speech Enhancement. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28, 825–838 (2020). https://doi.org/10.1109/TASLP.2020.2968738Савченко В. В., Савченко Л. В. Метод измерений показателя разборчивости речевого сигнала в информационной метрике Кульбака-Лейблера. Измерительная техника, (9), 59–64 (2019). https://doi.org/10.32446/0368-1025it.2019-9-59-64Savchenko V. V., Savchenko L. V. Method for measuring the intelligibility of speech signals in the Kullback-Leibler information metric. Measurement Techniques, 62(9), 832–839 (2019). https://doi.org/10.1007/s11018-019-01702-1Feng S., Halpern B. M., Kudina O., Scharenborg O. Towards inclusive automatic speech recognition. Computer Speech & Language, 84, 101567 (2024). https://doi.org/10.1016/j.csl.2023.101567Feng S., Halpern B. M., Kudina O., Scharenborg O. Towards inclusive automatic speech recognition. Computer Speech & Language, 84, 101567 (2024). https://doi.org/10.1016/j.csl.2023.101567Esfandiari M., Vorobyov S. A., Karimi M. New estimation methods for autoregressive process in the presence of white observation noise. Signal Processing, 171, 107480 (2020). https://doi.org/10.1016/j.sigpro.2020.107480Esfandiari M., Vorobyov S. A., Karimi M. New estimation methods for autoregressive process in the presence of white observation noise. Signal Processing, 171, 107480 (2020). https://doi.org/10.1016/j.sigpro.2020.107480Ngo Th., Kubo R., Akagi M. Increasing speech intelligibility and naturalness in noise based on concepts of modulation spectrum and modulation transfer function. Speech Communication, 135, 11–24 (2021). https://doi.org/10.1016/j.specom.2021.09.004Ngo Th., Kubo R., Akagi M. Increasing speech intelligibility and naturalness in noise based on concepts of modulation spectrum and modulation transfer function. Speech Communication, 135, 11–24 (2021). https://doi.org/10.1016/j.specom.2021.09.004O’Shaughnessy D. Speech enhancement – a review of modern methods. IEEE Transactions on Human-Machine Systems, 54(1), 110–120 (2024). https://doi.org/10.1109/THMS.2023.3339663O’Shaughnessy D. Speech enhancement – a review of modern methods. IEEE Transactions on Human-Machine Systems, 54(1), 110–120 (2024). https://doi.org/10.1109/THMS.2023.3339663Gustafsson Ph. U., Laukka P., Lindholm T. Vocal characteristics of accuracy in eyewitness testimony. Speech Communication, 146, 82–92 (2023). https://doi.org/10.1016/j.specom.2022.12.001Gustafsson Ph. U., Laukka P., Lindholm T. Vocal characteristics of accuracy in eyewitness testimony. Speech Communication, 146, 82–92 (2023). https://doi.org/10.1016/j.specom.2022.12.001Alex A, Wang L, Gastaldo P., Cavallaro A. Data augmentation for speech separation. Speech Communication, 152, 102949 (2023). https://doi.org/10.1016/j.specom.2023.05.009Alex A., Wang L., Gastaldo P., Cavallaro A. Data augmentation for speech separation. Speech Communication, 152, 102949 (2023). https://doi.org/10.1016/j.specom.2023.05.009Aldarmaki H., Ullah A., Ram S., Zaki N. Unsupervised automatic speech recognition: a review. Speech Communication, 139, 76–91 (2022). https://doi.org/10.1016/j.specom.2022.02.005Aldarmaki H., Ullah A., Ram S., Zaki N. Unsupervised automatic speech recognition: A review. Speech Communication, 139, 76–91 (2022). https://doi.org/10.1016/j.specom.2022.02.005Shahnawazuddin S. Developing children’s ASR system under low-resource conditions using end-to-end architecture. Digital Signal Processing, 146, 104385 (2024). https://doi.org/10.1016/j.dsp.2024.104385Shahnawazuddin S. Developing children’s ASR system under low-resource conditions using end-to-end architecture. Digital Signal Processing, 146, 104385 (2024). https://doi.org/10.1016/j.dsp.2024.104385Wei S., Zou S., Liao F. A comparison on data augmentation methods based on deep learning for audio classifcation. Journal of Physics: Conference Series, 1453(1), 012085 (2020). https://doi.org/10.1088/1742-6596/1453/1/012085Wei S., Zou S., Liao F. A comparison on data augmentation methods based on deep learning for audio classifcation. Journal of Physics: Conference Series, 1453(1), 012085 (2020). https://doi.org/10.1088/1742-6596/1453/1/012085

The authors declare that there are no conflicts of interest present.