Assessing ChatGPT’s logical reasoning quality in physics problem solving
Discover Education, cilt.5, sa.1, 2026 (Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 5 Sayı: 1
- Basım Tarihi: 2026
- Doi Numarası: 10.1007/s44217-026-01464-3
- Dergi Adı: Discover Education
- Derginin Tarandığı İndeksler: Scopus, EBSCO Education Source, Education Abstracts, ERIC (Education Resources Information Center), Directory of Open Access Journals, Education Source Ultimate (EBSCO)
- Anahtar Kelimeler: AI in education, ChatGPT, Language comparison, Large language models (LLMs), Physics education
- Eskişehir Osmangazi Üniversitesi Adresli: Evet
Özet
This study examines ChatGPT-4o’s process of solving calculus-based physics problems, particularly questions related to Gauss’s Law and Electric Field, not only in terms of its success in arriving at the correct answer but also in terms of how it structures this process and constructs its logical reasoning chain. A qualitative case study design was employed. Six multiple-choice questions (four on Gauss’s Law and two on Electric Field) were posed to ChatGPT-4o in both Turkish and English; the responses were recorded via screenshots and evaluated by four academics, three specializing in physics education and one in assessment, under the Cognitive Levels framework (Understanding, Application, Analysis, and Computation). The academics focused on whether the system understood the question, how it established connections between concepts, and whether it demonstrated consistent reasoning in the solution process. Findings based on expert evaluations reveal that the model performs better on English questions at all cognitive levels; however, it experiences a significant performance decline on Turkish questions that require multi-step reasoning and context-based interpretation. This highlights that language selection in AI-supported teaching environments is not only a means of communication but also an important factor determining the quality of the learning process.