A Criteria-Based Evaluation of AI-Generated Reading Texts for 5th Grade Students: A Comparative Analysis of ChatGPT, Gemini, and Claude

Abstract

The purpose of this study is to examine reading texts at the 5th grade level of middle school produced using ChatGPT 5.2 (OpenAI), Gemini 3.1 Pro (Google DeepMind), and Claude Opus 4.6 (Anthropic) artificial intelligence tools through an evaluation form created based on the Ministry of National Education (MoNE) text qualities (2019) and TTKB electronic content evaluation criteria (2026). In this context, the document analysis method was employed in the study. The data sources for this study consist of six narrative texts generated using three different AI tools. In the evaluation of these documents, an assessment form consisting of 51 items across 7 categories was used. This form was developed by selecting criteria from the MoNE (2019) and Board of Education (2026) guidelines that are appropriate for the characteristics of the story genre. The data collection tool in question was reviewed and approved by two subject matter experts prior to implementation. The evaluations revealed that all three AI tools used in the study largely met the specified criteria, but Gemini received the highest score. It was found that all three AI tools received full marks in the categories of character and content consistency, values and inclusivity, and literary and aesthetic quality. However, no text received a perfect score in terms of language and expression; while punctuation errors were found in all tools, spelling errors were also observed in ChatGPT and Claude. It was concluded that Gemini only produced texts close to the target class level in terms of text length, while Claude lagged behind other tools in terms of visual and text consistency.