Keyword search for sign language
2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020, Barcelona, İspanya, 4 - 08 Mayıs 2020, cilt.2020-May, ss.8184-8188, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Cilt numarası: 2020-May
- Doi Numarası: 10.1109/icassp40776.2020.9054678
- Basıldığı Şehir: Barcelona
- Basıldığı Ülke: İspanya
- Sayfa Sayıları: ss.8184-8188
- Anahtar Kelimeler: Keyword search, sign language, weakly supervised learning
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Keyword search is the search for a written query in an archive, which is often assumed to be a collection from a spoken language. Yet, the main languages of the Deaf, i.e. sign languages, are mostly neglected in this definition due to being visual languages. In this paper, we propose a keyword search (KWS) system for a sign language. In this technique, we first extract body and hand joints from the frames of a sign language sentence and represent it with a unified spatio-temporal graph of skeleton joints. We then train our graph-convolutional-network encoder, query embedding, and selection mechanism together in a weakly supervised, end-to-end fashion. Experimental results are reported on RWTH-PHOENIX-Weather 2014T dataset.