Cross-Lingual Adaptation of Phoneme-Motion Memory for Turkish Talking Head Generation Türkçe Konuşan Yüz Üretimi için Fonem-Hareket Hafizasinin Diller Arasi Uyarlanmasi
34th Signal Processing and Communications Applications Conference, SIU 2026, İstanbul, Türkiye, 7 - 10 Temmuz 2026, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/siu71813.2026.11636676
- Basıldığı Şehir: İstanbul
- Basıldığı Ülke: Türkiye
- Anahtar Kelimeler: 3D Turkish facial motion dataset, cross-lingual transfer, facial motion memory, phoneme-motion alignment, Speech-driven 3D facial animation
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Speech-driven 3D facial animation is largely developed on English datasets, leaving cross-lingual generalization underexplored. We study the transferability of memory-based motion representations to Turkish using MemoryTalker, a two-stage architecture with a Facial Motion Memory and audio-guided stylization. The model is trained on VOCASET and evaluated zero-shot on TDV, a Turkish 3D facial motion dataset. Results reveal a significant cross-lingual gap, especially in lip articulation, while fine-tuning with limited Turkish data effectively adapts the motion memory, achieving near in-domain performance without sacrificing temporal consistency. These findings show that memory-based motion representations enable efficient cross-lingual transfer and rapid adaptation to language-specific articulation.