Temporally-aware acoustic unit discovery for zerospeech 2019 challenge


Yusuf B., Gök A., Gundogdu B., Kose O. D., SARAÇLAR M.

20th Annual Conference of the International Speech Communication Association: Crossroads of Speech and Language, INTERSPEECH 2019, Graz, Avusturya, 15 - 19 Eylül 2019, cilt.2019-September, ss.1098-1102, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası: 2019-September
  • Doi Numarası: 10.21437/interspeech.2019-1430
  • Basıldığı Şehir: Graz
  • Basıldığı Ülke: Avusturya
  • Sayfa Sayıları: ss.1098-1102
  • Anahtar Kelimeler: self-organizing map, gated recurrent unit, sparse recurrent autoencoder, correspondence autoencoder
  • Boğaziçi Üniversitesi Adresli: Evet

Özet

Zero-resource speech processing efforts focus on unsupervised discovery of sub-word acoustic units. Common approaches work with spatial similarities between the acoustic frame representations within Bayesian or neural network-based frameworks. We propose two methods that utilize the temporal proximity information in addition to the acoustic similarity for clustering frames into acoustic units. The first approach uses a temporally biased self-organizing map (SOM) to discover such units. Since the SOM unit indices are correlated with (vector) spatial distance, we pool neighboring units and then train a recurrent neural network to predict each pooled unit. The second approach incorporates temporal awareness by training a recurrent sparse autoencoder, in which unsupervised clustering is done on the intermediate softmax layer. This network is then fine-tuned using aligned pairs of acoustically similar sequences obtained via unsupervised term discovery. Our approaches outperform the provided baseline system on two main metrics of the Zerospeech 2019 challenge, ABX-discriminability and bitrate of the quantized embeddings, both for English and the surprise language. Furthermore, the temporal-awareness and the post-filtering techniques adopted in this work resulted in an enhanced continuity of the decoding, yielding low bitrates.