A new meta-analysis published in Humanities and Social Sciences Communications has found that ChatGPT has a positive effect on student learning performance, with more moderate effects on student perception and higher-order thinking.
The study, conducted by researchers at Hangzhou Normal University, analyzed data from 51 experimental and quasi-experimental studies carried out between November 2022 and February 2025.
The analysis focused on three key outcomes: learning performance, learning perception, and higher-order thinking. ChatGPT showed a large effect size on learning performance (g = 0.867), and smaller but positive effects on learning perception (g = 0.456) and higher-order thinking (g = 0.457).
The study found that the tool’s effectiveness varied depending on how it was used. The strongest learning gains were reported in problem-based learning environments and in skill- or competency-driven courses. ChatGPT’s influence on higher-order thinking was more pronounced when used as an intelligent tutor.
The duration of use was also a critical factor. Students benefited most from ChatGPT when it was used over a four to eight-week period. Shorter interventions produced weaker results, and longer-term use occasionally showed signs of diminishing returns, potentially due to over-reliance.
Performance gains were most significant in STEM-related courses, while learning perception outcomes were less consistent. ChatGPT was less effective in project-based learning scenarios, which the authors note may demand more collaborative or open-ended engagement than the tool currently supports.
The authors reviewed 6,621 studies using PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) standards. Only English-language studies that used experimental or quasi-experimental designs with measurable learning outcomes were included. Studies focusing on student intent to use ChatGPT, or those using other AI tools, were excluded.
Sample populations ranged from primary school to university level, although most studies focused on college students. Primary and early childhood education were significantly underrepresented, with only one qualifying study involving primary learners.
The study recommends embedding ChatGPT within structured pedagogical models, such as Bloom’s taxonomy, to help students apply and evaluate content, rather than rely solely on it for recall or summarization. It also emphasizes the importance of pairing generative AI tools with formative assessment and instructional design.
The authors argue that ChatGPT’s value lies in its role as a supplemental learning tool, not a replacement for traditional teaching. They call for further studies that include younger learners and mixed-methods designs to offer a more complete view of the tool’s educational impact.
The full article, “The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis”, is available in Humanities and Social Sciences Communications. Download the article here.
© Paxton Media Ltd 2025. All rights reserved. No part of this website, including text, graphics, logos, images, and software, may be reproduced, distributed, or transmitted in any form or by any means without the prior written permission of Paxton Media Ltd.
Leave a Comment