Continuous space discriminative language modeling
2012 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2012, Kyoto, Japonya, 25 - 30 Mart 2012, ss.2129-2132, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/icassp.2012.6288332
- Basıldığı Şehir: Kyoto
- Basıldığı Ülke: Japonya
- Sayfa Sayıları: ss.2129-2132
- Anahtar Kelimeler: Discriminative language modeling, neural network
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Discriminative language modeling is a structured classification problem. Log-linear models have been previously used to address this problem. In this paper, the standard dot-product feature representation used in log-linear models is replaced by a non-linear function parameterized by a neural network. Embeddings are learned for each word and features are extracted automatically through the use of convolutional layers. Experimental results show that as a stand-alone model the continuous space model yields significantly lower word error rate (1% absolute), while having a much more compact parameterization (60%-90% smaller). If the baseline scores are combined, our approach performs equally well. © 2012 IEEE.