A Speech-Driven Framework for Automated Uzbek-Language Customer Service

Mualliflar

  • Mukhamadiyev Abdinabi Research Professor, Department of Computer Engineering, Gachon University, Sujeong-gu, Seongnam-si 13120, Korea ##default.groups.name.author##
  • Khujakulov Toshtemir Associate Professor, t.f.f.D (PhD), University of Tashkent for Applied Sciences, Department of "Computer Engineering" ##default.groups.name.author##
  • Majidova Yulduz University of Tashkent for Applied Sciences, Department of "Computer Engineering" ##default.groups.name.author##

##semicolon##

https://doi.org/10.65164/xmq9an31

##semicolon##

speech technologies; call center; speech corpus; Uzbek language; speech-to-text; textto-speech; speech recognition; speech synthesis; IVR; public services

Annotatsiya

Contact centers in banking and public services increasingly struggle to combine
continuous availability with acceptable waiting times, and this problem is especially acute in lowresource-language markets such as Uzbekistan, where mature commercial speech technology is
unavailable. To address this gap, we design, implement, and evaluate UzAssistant, a fully speechdriven virtual operator that conducts an entire service dialogue in Uzbek without human
intervention. The framework chains four neural components. First, a Gaussian-Mixture-Model
classifier operating on Mel-Frequency Cepstral Coefficient (MFCC) features verifies the caller's
identity against enrolled voice profiles, with a rejection threshold for unregistered speakers. Second,
an end-to-end recurrent architecture based on Deep Speech 2 transcribes the spoken request into
standardized Uzbek text. Third, an attention-augmented sequence-to-sequence summarizer
condenses the transcription, after which a fine-tuned multilingual sentence transformer embeds the
condensed query and selects the semantically closest entry from a structured question-answer
knowledge base through cosine similarity. Finally, the retrieved answer is vocalized by a WaveNet
neural vocoder coupled with a Tacotron 2 decoder and explicit phoneme alignment, producing
natural Uzbek speech in real time. Trained and evaluated on a purpose-built 108-hour banking
speech corpus of 863 speakers, the framework reached 96.4% speaker-verification accuracy with
56 live participants, 85.27% average sentence-embedding accuracy over four Uzbek datasets, and
82.5% end-to-end response accuracy on paraphrased customer queries; the strongest recognition
configuration-Deep Speech 2 with language-model decoding and threefold data augmentationyielded a 5.22% character error rate and a 13.8% word error rate. These findings indicate that operator workload in Uzbek-language call centers can be reduced by an estimated 20-25% while extending service to a 24/7 schedule.

##submission.citations##

##submission.downloads##

##submissions.published##

2026-09-25