Abstract:
Today, the Transformer model, which allows parallelization and also has its own internal attention,
has been widely used in the feld of speech recognition. The great advantage of this architecture is the
fast learning speed, and the lack of sequential operation, as with recurrent neural networks. In this
work, Transformer models and an end-to-end model based on connectionist temporal classifcation
were considered to build a system for automatic recognition of Kazakh speech. It is known that
Kazakh is part of a number of agglutinative languages and has limited data for implementing speech
recognition systems. Some studies have shown that the Transformer model improves system
performance for low-resource languages. Based on our experiments, it was revealed that the joint
use of Transformer and connectionist temporal classifcation models contributed to improving the
performance of the Kazakh speech recognition system and with an integrated language model it
showed the best character error rate 3.7% on a clean dataset.
