Generating Domain Models with LLMs Using Instruction Tuning: A Research Preview
31st International Working Conference on Requirements Engineering: Foundation for Software Quality, REFSQ 2025, Barcelona, İspanya, 7 - 10 Nisan 2025, cilt.15588 LNCS, ss.157-165, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Cilt numarası: 15588 LNCS
- Doi Numarası: 10.1007/978-3-031-88531-0_11
- Basıldığı Şehir: Barcelona
- Basıldığı Ülke: İspanya
- Sayfa Sayıları: ss.157-165
- Anahtar Kelimeler: Generative AI, Large language models, Domain models, Instruction Tuning
- Boğaziçi Üniversitesi Adresli: Evet
Özet
[Context and Motivation] Domain models are valuable tools for facilitating software development by providing structured insights into the problem space. [Question/problem] Building domain models requires time, domain, and modeling expertise; therefore, it is an expensive task. Due to this cost, domain models are often overlooked in agile development, where the primary form of documentation is user stories that capture actors, their desires, and, optionally, the rationale. [Principal ideas] Automated domain model generation from user stories can support the software development process in agile settings. Recent advances in large language models offer promise in automating such tasks through improved language understanding and reasoning. This research preview investigates the potential of open-source large language models to automatically generate domain models from natural language requirements, employing instruction tuning on a substantial dataset of user stories and their corresponding domain models. By comparing instruction-tuned and pre-trained models, we aim to assess the specific impact of instruction tuning on model generation quality. [Contributions] This study is, to our knowledge, the first to apply instruction tuning on large language models specifically for domain model generation. Through qualitative and quantitative evaluations of factors like completeness and correctness, we assess the performance and quality of the instruction-tuned models.