Compositional neural network language models for agglutinative languages


Arisoy E., SARAÇLAR M.

17th Annual Conference of the International Speech Communication Association, INTERSPEECH 2016, California, Amerika Birleşik Devletleri, 8 - 16 Eylül 2016, cilt.08-12-September-2016, ss.3494-3498, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası: 08-12-September-2016
  • Doi Numarası: 10.21437/interspeech.2016-1239
  • Basıldığı Şehir: California
  • Basıldığı Ülke: Amerika Birleşik Devletleri
  • Sayfa Sayıları: ss.3494-3498
  • Anahtar Kelimeler: Language modeling, long short-term memory, sub-word-based language modeling, agglutinative languages
  • Boğaziçi Üniversitesi Adresli: Evet

Özet

Continuous space language models (CSLMs) have been proven to be successful in speech recognition. With proper training of the word embeddings, words that are semantically or syntactically related are expected to be mapped to nearby locations in the continuous space. In agglutinative languages, words are made up of concatenation of stems and suffixes and, as a result, compositional modeling is important. However, when trained on word tokens, CSLMs do not explicitly consider this structure. In this paper, we explore compositional modeling of stems and suffixes in a long short-term memory neural network language model. Our proposed models jointly learn distributed representations for stems and endings (concatenation of suffixes) and predict the probability for stem and ending sequences. Experiments on the Turkish Broadcast news transcription task show that further gains on top of a state-of-theart stem-ending-based n-gram language model can be obtained with the proposed models.