Continuous space discriminative language modeling


Xu P., Khudanpur S., Lehr M., Prud'hommeaux E., Glenn N., Karakos D., ...Daha Fazla

2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012, Kyoto, Japonya, 25 - 30 Mart 2012, ss.2129-2132, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/icassp.2012.6288332
  • Basıldığı Şehir: Kyoto
  • Basıldığı Ülke: Japonya
  • Sayfa Sayıları: ss.2129-2132
  • Anahtar Kelimeler: Discriminative language modeling, neural network
  • Boğaziçi Üniversitesi Adresli: Evet

Özet

Discriminative language modeling is a structured classification problem. Log-linear models have been previously used to address this problem. In this paper, the standard dot-product feature representation used in log-linear models is replaced by a non-linear function parameterized by a neural network. Embeddings are learned for each word and features are extracted automatically through the use of convolutional layers. Experimental results show that as a stand-alone model the continuous space model yields significantly lower word error rate (1% absolute), while having a much more compact parameterization (60%-90% smaller). If the baseline scores are combined, our approach performs equally well. © 2012 IEEE.