Subword Contextual Embeddings for Languages with Rich Morphology
19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020, Florida, Amerika Birleşik Devletleri, 14 - 17 Aralık 2020, ss.994-1001, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/icmla51294.2020.00161
- Basıldığı Şehir: Florida
- Basıldığı Ülke: Amerika Birleşik Devletleri
- Sayfa Sayıları: ss.994-1001
- Anahtar Kelimeler: deep learning, named entity recognition, nlp, transfer learning
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Morphological information is important for many sequence labeling tasks in Natural Language Processing (NLP). Yet, existing approaches rely heavily on manual annotations or external software to capture this information. In this study, we propose using subword contextual embeddings for languages with rich morphology. Evaluated on Dependency Parsing (DEP) and Named Entity Recognition (NER) tasks, which are shown to benefit highly from morphological information, subword contextual embeddings consistently outperformed other approaches on all languages tested (Hungarian, Finnish, Czech and Turkish). Our proposed method enables achieving state-of-the-art results with little annotation requirements compared to the previous work. Besides, the novel network architecture we propose, coupled with a Bayesian hyperparameter optimization suite, achieved state-of-the-art results for both tasks for the Turkish language. Finally, we experimented with different multi-task learning architectures to analyze the effect of jointly learning the two tasks.