Employing a Hierarchical Rater Models for Automated Scoring: Scope Review on the Application in Educational Assessment

Akif Avcu — Wed, 16 Apr 2025 00:00:00 +0300

This scope-review presents the milestones of how Hierarchical Rater Models (HRMs) become operable to used in automated essay scoring (AES) to improve instructional evaluation. Although essay evaluations—a useful instrument for evaluating higher-order cognitive abilities—have always depended on human raters, concerns regarding rater bias, inconsistency, and scalability have motivated the development of automated systems. By modeling rater behaviors, task complexity, and interaction effects, HRMs handle these issues and offer a strong base to minimize biases and increase dependability. Advances in machine learning and natural language processing (NLP) have helped HRMs to be included into AES systems. Leveraging HRMs to rectify rater biases and guarantee fairness, these systems include language analysis, semantic evaluation, and contextual understanding. This review also includes how Signal Detection Theory (SDT) is being included into HRMs to improve their capacity to assess rater sensitivity and bias and provide understandable results. By including HRMs into AES, feedback quality is improved as well as score accuracy, hence facilitating more exact formative evaluations become possible. Still present, though, are difficulties like computing complexity, dataset availability, and algorithmic bias. The paper emphasizes the possibilities of HRMs in creating fair, high-quality, scalable evaluation systems and supports ongoing research to improve these approaches for various educational environments.

The Effect of Video Magazines on Secondary School Students' Academic Achievement and Digital Technology Attitudes: An Experimental Study

İlknur Bilgi-Kurt, Osman Soner — Wed, 16 Apr 2025 00:00:00 +0300

Video magazines can make students' learning experience more effective by adding a different dimension to the learning process. This learning experience can create changes in students' attitudes and behaviors. This study aimed to investigate the impact of video magazines in the experimental group. It printed magazines in the control group on middle school pupils' academic performance and attitudes toward digital technology. The research study group comprises 5th-grade children enrolled in a state-affiliated secondary school in the Arnavutköy district of Istanbul during the academic year 2023-2024. One of the two identical fifth-grade classes at the school was designated as the control group, while the other was given the experimental group. Both the control group and the experimental group consisted of 27 students each. The study employed a quasi-experimental approach and used quantitative analysis protocols. An examination of the accomplishment test revealed a notable disparity in the post-test scores between the experimental and control groups. Consequently, it was concluded that the experimental group received instruction using video magazines and achieved superior scores. Upon analysis of the post-tests of the attitude scale between the experimental and control groups, a notable disparity was observed in the sub-dimensions of "social networks," "instructional technology use," "technology for personal use," and "use for entertainment purposes." The use of video magazines in teaching is of significant importance. The results were deliberated over in relationship to the existing body of knowledge, and recommendations were put forward.

Malaysian Online Journal of Educational Technology

Employing a Hierarchical Rater Models for Automated Scoring: Scope Review on the Application in Educational Assessment

The Effect of Video Magazines on Secondary School Students' Academic Achievement and Digital Technology Attitudes: An Experimental Study