Unsupervised discovery of fingerspelled letters in sign language videos Işaret dili videolarinda parmak alfabesiyle gösterilen harflerin gözetimsiz olarak keşfi
29th IEEE Conference on Signal Processing and Communications Applications, SIU 2021, Virtual, Istanbul, Türkiye, 9 - 11 Haziran 2021, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/siu53274.2021.9477880
- Basıldığı Şehir: Virtual, Istanbul
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: Adjusted mutual information, Fingerspelling, Gaussian Mixture Model, K-means clustering, Sign language, Unsupervised discovery
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Automatic discovery and indexing in sign language videos would provide many valuable services such as search and retrieval in deaf social and news media which consists of sign language videos often with no associated texts available. As some of the context words in sign language conversations are fingerspelled, the discovery of fingerspelled letters plays a significant role. In this study, k-means clustering and Gaussian Mixture Model methods are used for the unsupervised discovery of fingerspelled letters. The clusers which mostly contain hand shapes corresponding to transitions between letters are eliminated based on their distortion level and Bayes information criterion values. The performance of the clustering methods are evaluated with the help of Adjusted Mutual Information metric. According to the results obtained with this metric, using a Gaussian Mixture Model for clustering yields better results than k-means clustering.