TIMERES: A Turkish Benchmark For Evaluating Temporal Understanding of Large Language Models
19th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2026, Rabat, Fas, 24 - 29 Mart 2026, cilt.4, ss.910-920, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Cilt numarası: 4
- Doi Numarası: 10.18653/v1/2026.eacl-srw.67
- Basıldığı Şehir: Rabat
- Basıldığı Ülke: Fas
- Sayfa Sayıları: ss.910-920
- Açık Arşiv Koleksiyonu: AVESİS Açık Erişim Koleksiyonu
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Temporal information is an essential part of communication, and understanding language requires processing it effectively. Despite recent advances, Large Language Models (LLMs) still struggle with temporal understanding. Existing benchmarks primarily focus on English and underexplore how linguistic structure contributes to temporal meaning. As a result, temporal understanding in languages other than English remains largely understudied. In this paper, we introduce TIMERES, a Turkish benchmark for evaluating temporal understanding of LLMs. TIMERES aims to investigate comprehension of Reichenbach's temporal points and reported speech through date arithmetic. Our dataset includes 4,600 questions across 4 tasks at two levels of complexity, and presents a paired question formulation to distinguish temporal discourse understanding from temporal arithmetic capabilities. We evaluated six LLMs, and demonstrated that models struggle to resolve reported speech and fail to generalize across word order variations. Code and data are available at https://github.com/yagizdemir/timeres.