Joint Distance and Representation Learning for Sign Language Videos Isaret Dili Videolari icin Gosterim ve Uzaklik Metrigi Ogrenimi
28th Signal Processing and Communications Applications Conference, SIU 2020, Gaziantep, Türkiye, 5 - 07 Ekim 2020, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/siu49456.2020.9302207
- Basıldığı Şehir: Gaziantep
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: automatic speech recognition, diagonal matching, dynamic time warping, same-different word discrimination, Siamese neural networks, sign language, video processing
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Scientific progress in sign language applications has a crucial importance, since it can increase the quality of the lives of deaf and mute people. In this study, frame level distance metric and representation learning problems are studied to represent and classify sign language video frame sequences in an efficient way. For the joint representation and distance metric learning, a Siamese deep neural network is employed. This network is trained with the examples extracted from the automatically acquired transcription of the speech signal found in the sign language videos. The performance of the proposed system is measured by utilizing the same-different sign-discrimination task. Our method considerably outperforms the considered baseline systems that are based on the dynamic time warping (DTW) algorithm used with common distance metrics. This success can be considered as a first step towards an efficient query-by-sign search system.