Discriminative language modeling with conditional random fields and the perceptron algorithm


Roark B., SARAÇLAR M., Collins M., Johnson M.

42nd Annual Meeting of the Association for Computational Linguistics, ACL 2004, Barcelona, İspanya, 21 - 26 Temmuz 2004, ss.47-54, (Tam Metin Bildiri)

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Basıldığı Şehir: Barcelona
  • Basıldığı Ülke: İspanya
  • Sayfa Sayıları: ss.47-54
  • Boğaziçi Üniversitesi Adresli: Evet

Özet

This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the perceptron algorithm, and a method based on conditional random fields (CRFs). The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The perceptron algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. However, using the feature set output from the perceptron algorithm (initialized with their weights), CRF training provides an additional 0.5% reduction in word error rate, for a total 1.8% absolute reduction from the baseline of 39.2%.