Multi-stream long short-term memory neural network language model
16th Annual Conference of the International Speech Communication Association, INTERSPEECH 2015, Dresden, Almanya, 6 - 10 Eylül 2015, cilt.2015-January, ss.1413-1417, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Cilt numarası: 2015-January
- Basıldığı Şehir: Dresden
- Basıldığı Ülke: Almanya
- Sayfa Sayıları: ss.1413-1417
- Anahtar Kelimeler: Language modeling, Long short-term memory, Sub-word-based language modeling
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Long Short-Term Memory (LSTM) neural networks are recurrent neural networks that contain memory units that can store contextual information from past inputs for arbitrary amounts of time. A typical LSTM neural network language model is trained by feeding an input sequence. i.e., a stream of words, to the input layer of the network and the output layer predicts the probability of the next word given the past inputs in the sequence. In this paper we introduce a multi-stream LSTM neural network language model where multiple asynchronous input sequences are fed to the network as parallel streams while predicting the output word sequence. For our experiments, we use a sub-word sequence in addition to a word sequence as the input streams, which allows joint training of the LSTM neural network language model using both information sources.